AtHeartEngineer/rfc-index
1
0
Fork 0
mirror of https://github.com/vacp2p/rfc-index.git synced 2026-01-07 23:04:09 -05:00
Code Issues Packages Projects Releases Wiki Activity

Fix Files for Linting (#94)

Browse Source
This commit is contained in:
Jimmy Debe
2024-09-13 10:51:09 -04:00
committed by GitHub
parent 0b4d151638
commit 3ab314d87d
77 changed files with 6814 additions and 3371 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
waku/.DS_Store vendored Normal file
View File

Binary file not shown.

6
waku/README.md
View File

@@ -1,5 +1,7 @@
# Waku RFCs
Waku builds a family of privacy-preserving, censorship-resistant communication protocols for web3 applications.
Waku builds a family of privacy-preserving,
censorship-resistant communication protocols for web3 applications.
Contributors can visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.
Contributors can visit [Waku RFCs](https://github.com/waku-org/specs)
for new Waku specifications under discussion.

BIN
waku/deprecated/.DS_Store vendored Normal file
View File

Binary file not shown.

369
waku/deprecated/16/rpc.md
View File

@@ -9,13 +9,17 @@ editor: Hanno Cornelius <hanno@status.im>
## Introduction
This specification describes the JSON-RPC API that Waku v2 nodes MAY adhere to. Refer to the [Waku v2 specification](../10/waku2.md) for more information on Waku v2.
This specification describes the JSON-RPC API that Waku v2 nodes MAY adhere to.
Refer to the [Waku v2 specification](../10/waku2.md)
for more information on Waku v2.
## Wire Protocol
### Transport
Nodes SHOULD expose an accessible [JSON-RPC](https://www.jsonrpc.org/specification) API. The JSON-RPC version SHOULD be `2.0`. Below is an example request:
Nodes SHOULD expose an accessible
[JSON-RPC](https://www.jsonrpc.org/specification) API.
The JSON-RPC version SHOULD be `2.0`. Below is an example request:
```json
{
@@ -37,7 +41,10 @@ Nodes SHOULD expose an accessible [JSON-RPC](https://www.jsonrpc.org/specificati
### Types
In this specification, the primitive types `Boolean`, `String`, `Number` and `Null`, as well as the structured types `Array` and `Object`, are to be interpreted according to the [JSON-RPC specification](https://www.jsonrpc.org/specification#conventions). It also adopts the same capitalisation conventions.
In this specification, the primitive types `Boolean`, `String`,
`Number` and `Null`, as well as the structured types `Array` and `Object`,
are to be interpreted according to the [JSON-RPC specification](https://www.jsonrpc.org/specification#conventions).
It also adopts the same capitalisation conventions.
The following structured types are defined for use throughout the document:
@@ -57,23 +64,27 @@ Refer to [`Waku Message` specification](../14/message.md) for more information.
## Method naming
The JSON-RPC methods in this document are designed to be mappable to HTTP REST endpoints. Method names follow the pattern `<method_type>_waku_<protocol_version>_<api>_<api_version>_<resource>`
The JSON-RPC methods in this document are designed to be mappable to HTTP REST endpoints.
Method names follow the pattern `<method_type>_waku_<protocol_version>_<api>_<api_version>_<resource>`
- `<method_type>`: prefix of the HTTP method type that most closely matches the JSON-RPC function. Supported `method_type` values are `get`, `post`, `put`, `delete` or `patch`.
- `<method_type>`:
prefix of the HTTP method type that most closely matches the JSON-RPC function.
Supported `method_type` values are `get`, `post`, `put`, `delete` or `patch`.
- `<protocol_version>`: Waku version. Currently **v2**.
- `<api>`: one of the listed APIs below, e.g. `store`, `debug`, or `relay`.
- `<api_version>`: API definition version. Currently **v1** for all APIs.
- `<resource>`: the resource or resource path being addressed
The method `post_waku_v2_relay_v1_message`, for example, would map to the HTTP REST endpoint `POST /waku/v2/relay/v1/message`.
The method `post_waku_v2_relay_v1_message`, for example,
would map to the HTTP REST endpoint `POST /waku/v2/relay/v1/message`.
## Debug API
### Types
Types
The following structured types are defined for use on the Debug API:
#### WakuInfo
### WakuInfo
`WakuInfo` is an `Object` containing the following fields:
@@ -82,9 +93,7 @@ The following structured types are defined for use on the Debug API:
| `listenAddresses` | `Array`[`String`] | mandatory | Listening addresses of the node |
| `enrUri` | `String` | optional | ENR URI of the node |
#### WakuInfo
### `get_waku_v2_debug_v1_info`
`get_waku_v2_debug_v1_info`
The `get_waku_v2_debug_v1_info` method retrieves information about a Waku v2 node
@@ -96,94 +105,120 @@ none
- [**`WakuInfo`**](#wakuinfo) - information about a Waku v2 node
### `get_waku_v2_debug_v1_version`
The `get_waku_v2_debug_v1_version` method retrieves the version of a Waku v2 node as a string.
The `get_waku_v2_debug_v1_version` method retrieves the version of a Waku v2 node
as a string.
The version SHOULD follow [semantic versioning](https://semver.org/).
In case the node's current build is based on a git commit between semantic versions,
the retrieved version string MAY contain the git commit hash alone or in combination with the latest semantic version.
the retrieved version string MAY contain the git commit hash alone or
in combination with the latest semantic version.
#### Parameters
Parameters
none
#### Response
Response
- **`string`** - represents the version of a Waku v2 node
## Relay API
Refer to the [Waku Relay specification](../11/relay.md) for more information on the relaying of messages.
Refer to the [Waku Relay specification](../11/relay.md)
for more information on the relaying of messages.
### `post_waku_v2_relay_v1_message`
`post_waku_v2_relay_v1_message`
The `post_waku_v2_relay_v1_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
The `post_waku_v2_relay_v1_message` method publishes a message to be relayed on a
[PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The `message` being relayed |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `post_waku_v2_relay_v1_subscriptions`
`post_waku_v2_relay_v1_subscriptions`
The `post_waku_v2_relay_v1_subscriptions` method subscribes a node to an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `post_waku_v2_relay_v1_subscriptions` method subscribes a node to an array of
[PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being subscribed to |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `delete_waku_v2_relay_v1_subscriptions`
`delete_waku_v2_relay_v1_subscriptions`
The `delete_waku_v2_relay_v1_subscriptions` method unsubscribes a node from an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `delete_waku_v2_relay_v1_subscriptions` method unsubscribes a node from an array
of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being unsubscribed from |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_relay_v1_messages`
`get_waku_v2_relay_v1_messages`
The `get_waku_v2_relay_v1_messages` method returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
The `get_waku_v2_relay_v1_messages` method returns a list of messages
that were received on a subscribed
[PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
after the last time this method was called.
The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object)
if no subscription exists for the polled `topic`.
If no message has yet been received on the polled `topic`,
the server SHOULD return an empty list.
This method can be used to poll a `topic` for new messages.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
#### Response
Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Array`[[`WakuMessage`](#wakumessage)]** -
the latest `messages` on the polled `topic` or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Relay Private API
The Private API provides functionality to encrypt/decrypt `WakuMessage` payloads using either symmetric or asymmetric cryptography. This allows backwards compatibility with [Waku v1 nodes](../../legacy/6/waku1.md).
It is the API client's responsibility to keep track of the keys used for encrypted communication. Since keys must be cached by the client and provided to the node to encrypt/decrypt payloads, a Private API SHOULD NOT be exposed on non-local or untrusted nodes.
The Private API provides functionality to encrypt/decrypt `WakuMessage` payloads
using either symmetric or asymmetric cryptography.
This allows backwards compatibility with [Waku v1 nodes](../../legacy/6/waku1.md).
It is the API client's responsibility to keep track of the keys
used for encrypted communication.
Since keys must be cached by the client and
provided to the node to encrypt/decrypt payloads,
a Private API SHOULD NOT be exposed on non-local or untrusted nodes.
### Types
Types
The following structured types are defined for use on the Private API:
#### KeyPair
### KeyPair
`KeyPair` is an `Object` containing the following fields:
@@ -194,33 +229,40 @@ The following structured types are defined for use on the Private API:
### `get_waku_v2_private_v1_symmetric_key`
Generates and returns a symmetric key that can be used for message encryption and decryption.
Generates and returns a symmetric key that can be used for message encryption and
decryption.
#### Parameters
Parameters
none
#### Response
Response
- **`String`** - A new symmetric key as hex encoded data string
### `get_waku_v2_private_v1_asymmetric_keypair`
Generates and returns a public/private key pair that can be used for asymmetric message encryption and decryption.
Generates and returns a public/private key pair
that can be used for asymmetric message encryption and decryption.
#### Parameters
Parameters
none
#### Response
Response
- **[`KeyPair`](#keypair)** - A new public/private key pair as hex encoded data strings
### `post_waku_v2_private_v1_symmetric_message`
`post_waku_v2_private_v1_symmetric_message`
The `post_waku_v2_private_v1_symmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `post_waku_v2_private_v1_symmetric_message` method publishes a message
to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
Before being relayed, the message payload is encrypted using the supplied symmetric key. The client MUST provide a symmetric key.
Before being relayed,
the message payload is encrypted using the supplied symmetric key.
The client MUST provide a symmetric key.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
@@ -228,17 +270,22 @@ Before being relayed, the message payload is encrypted using the supplied symmet
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The (unencrypted) `message` being relayed |
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload encryption. This field MUST be included if symmetric key cryptography is selected |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `post_waku_v2_private_v1_asymmetric_message`
`post_waku_v2_private_v1_asymmetric_message`
The `post_waku_v2_private_v1_asymmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `post_waku_v2_private_v1_asymmetric_message` method publishes a message
to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
Before being relayed, the message payload is encrypted using the supplied public key. The client MUST provide a public key.
Before being relayed,
the message payload is encrypted using the supplied public key.
The client MUST provide a public key.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
@@ -246,66 +293,88 @@ Before being relayed, the message payload is encrypted using the supplied public
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The (unencrypted) `message` being relayed |
| `publicKey` | `String` | mandatory | The hex encoded public key to use for payload encryption. This field MUST be included if asymmetric key cryptography is selected |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_private_v1_symmetric_messages`
The `get_waku_v2_private_v1_symmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
The `get_waku_v2_private_v1_symmetric_messages` method decrypts and
returns a list of messages that were received on a subscribed
[PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
after the last time this method was called.
The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object)
if no subscription exists for the polled `topic`.
If no message has yet been received on the polled `topic`,
the server SHOULD return an empty list.
This method can be used to poll a `topic` for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied symmetric key. The client MUST provide a symmetric key.
Before returning the messages,
the server decrypts the message payloads using the supplied symmetric key.
The client MUST provide a symmetric key.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload decryption. This field MUST be included if symmetric key cryptography is selected |
#### Response
Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Array`[[`WakuMessage`](#wakumessage)]** -
the latest `messages` on the polled `topic` or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_private_v1_asymmetric_messages`
`get_waku_v2_private_v1_asymmetric_messages`
The `get_waku_v2_private_v1_asymmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
The `get_waku_v2_private_v1_asymmetric_messages` method decrypts and
returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
after the last time this method was called.
The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object)
if no subscription exists for the polled `topic`.
If no message has yet been received on the polled `topic`,
the server SHOULD return an empty list.
This method can be used to poll a `topic` for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied private key. The client MUST provide a private key.
Before returning the messages,
the server decrypts the message payloads using the supplied private key.
The client MUST provide a private key.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
| `privateKey` | `String` | mandatory | The hex encoded private key to use for payload decryption. This field MUST be included if asymmetric key cryptography is selected |
#### Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
Response
- **`Array`[[`WakuMessage`](#wakumessage)]** -
the latest `messages` on the polled `topic` or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Store API
Refer to the [Waku Store specification](../13/store.md) for more information on message history retrieval.
### Types
Refer to the [Waku Store specification](../13/store.md)
for more information on message history retrieval.
The following structured types are defined for use on the Store API:
#### StoreResponse
### StoreResponse
`StoreResponse` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `messages` | `Array`[[`WakuMessage`](#wakumessage)] | mandatory | Array of retrieved historical messages |
| `pagingOptions` | [`PagingOptions`](#pagingOptions) | [conditional](#get_waku_v2_store_v1_messages) | Paging information from which to resume further historical queries |
| `pagingOptions` | [`PagingOptions`](#pagingoptions) | [conditional](#get_waku_v2_store_v1_messages) | Paging information from which to resume further historical queries |
#### PagingOptions
`PagingOptions` is an `Object` containing the following fields:
`pagingOptions` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: |:-----------------:| :---: |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@@ -322,7 +391,7 @@ The following structured types are defined for use on the Store API:
| `digest` | `String` | mandatory | A hash for the message at this [`Index`](#index) |
| `receivedTime` | `Number` | mandatory | UNIX timestamp in nanoseconds at which the message at this [`Index`](#index) was received |
#### ContentFilter
ContentFilter
`ContentFilter` is an `Object` containing the following fields:
@@ -330,11 +399,19 @@ The following structured types are defined for use on the Store API:
| ----: | :---: | :---: |----------- |
| `contentTopic` | `String` | mandatory | The content topic of a [`WakuMessage`](#wakumessage) |
### `get_waku_v2_store_v1_messages`
`get_waku_v2_store_v1_messages`
The `get_waku_v2_store_v1_messages` method retrieves historical messages on specific content topics. This method MAY be called with [`PagingOptions`](#pagingoptions), to retrieve historical messages on a per-page basis. If the request included [`PagingOptions`](#pagingoptions), the node MUST return messages on a per-page basis and include [`PagingOptions`](#pagingoptions) in the response. These [`PagingOptions`](#pagingoptions) MUST contain a `cursor` pointing to the [`Index`](#index) from which a new page can be requested.
The `get_waku_v2_store_v1_messages` method retrieves historical messages
on specific content topics.
This method MAY be called with [`PagingOptions`](#pagingoptions),
to retrieve historical messages on a per-page basis.
If the request included [`PagingOptions`](#pagingoptions),
the node MUST return messages on a per-page basis and
include [`PagingOptions`](#pagingoptions) in the response.
These [`PagingOptions`](#pagingoptions) MUST contain a `cursor` pointing
to the [`Index`](#index) from which a new page can be requested.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
@@ -344,19 +421,21 @@ The `get_waku_v2_store_v1_messages` method retrieves historical messages on spec
| `endTime` | `Number` | optional | The inclusive upper bound on the [`timestamp`](../14/message.md/#message-attributes) of queried [`WakuMessage`s](#wakumessage). This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. |
| `pagingOptions` | [`PagingOptions`](#pagingoptions) | optional | Pagination information |
#### Response
Response
- [**`StoreResponse`**](#storeresponse) - the response to a `query` for historical messages.
- [**`StoreResponse`**](#storeresponse) -
the response to a `query` for historical messages.
## Filter API
Refer to the [Waku Filter specification](../12/filter.md) for more information on content filtering.
Refer to the [Waku Filter specification](../12/filter.md)
for more information on content filtering.
### Types
Types
The following structured types are defined for use on the Filter API:
#### ContentFilter
### ContentFilter
`ContentFilter` is an `Object` containing the following fields:
@@ -366,57 +445,73 @@ The following structured types are defined for use on the Filter API:
### `post_waku_v2_filter_v1_subscription`
The `post_waku_v2_filter_v1_subscription` method creates a subscription in a [light node](../12/filter.md/#rationale) for messages that matches a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `post_waku_v2_filter_v1_subscription` method creates a subscription in a
[light node](../12/filter.md/#rationale) for messages that matches a content filter
and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentFilters` | `Array`[[`ContentFilter`](#contentfilter)] | mandatory | Array of content filters being subscribed to |
| `topic` | `String` | optional | Message topic |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** - `true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `delete_waku_v2_filter_v1_subscription`
`delete_waku_v2_filter_v1_subscription`
The `delete_waku_v2_filter_v1_subscription` method removes subscriptions in a [light node](../12/filter.md/#rationale) matching a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
The `delete_waku_v2_filter_v1_subscription` method removes subscriptions
in a [light node](../12/filter.md/#rationale) matching a content filter and,
optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentFilters` | `Array`[[`ContentFilter`](#contentfilter)] | mandatory | Array of content filters being unsubscribed from |
| `topic` | `String` | optional | Message topic |
#### Response
Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_filter_v1_messages`
The `get_waku_v2_filter_v1_messages` method returns a list of messages that were received on a subscribed content `topic` after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled content `topic`. If no message has yet been received on the polled content `topic`, the server SHOULD respond with an empty list. This method can be used to poll a content `topic` for new messages.
The `get_waku_v2_filter_v1_messages` method returns a list of messages
that were received on a subscribed content `topic`
after the last time this method was called.
The server MUST respond with an
[error](https://www.jsonrpc.org/specification#error_object)
if no subscription exists for the polled content `topic`.
If no message has yet been received on the polled content `topic`,
the server SHOULD respond with an empty list.
This method can be used to poll a content `topic` for new messages.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentTopic` | `String` | mandatory | The content topic to poll for the latest messages |
#### Response
Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled content `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **`Array`[[`WakuMessage`](#wakumessage)]** -
the latest `messages` on the polled content `topic` or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Admin API
The Admin API provides privileged accesses to the internal operations of a Waku v2 node.
### Types
The Admin API provides privileged accesses
to the internal operations of a Waku v2 node.
The following structured types are defined for use on the Admin API:
#### WakuPeer
### WakuPeer
`WakuPeer` is an `Object` containing the following fields:
@@ -428,13 +523,19 @@ The following structured types are defined for use on the Admin API:
### `get_waku_v2_admin_v1_peers`
The `get_waku_v2_admin_v1_peers` method returns an array of peers registered on this node. Since a Waku v2 node may open either continuous or ad hoc connections, depending on the negotiated protocol, these peers may have different connected states. The same peer MAY appear twice in the returned array, if it is registered for more than one protocol.
The `get_waku_v2_admin_v1_peers` method returns an array of peers
registered on this node.
Since a Waku v2 node may open either continuous or ad hoc connections,
depending on the negotiated protocol,
these peers may have different connected states.
The same peer MAY appear twice in the returned array,
if it is registered for more than one protocol.
#### Parameters
Parameters
none
- none
#### Response
Response
- **`Array`[[`WakuPeer`](#wakupeer)]** - Array of peers registered on this node
@@ -442,30 +543,36 @@ none
The `post_waku_v2_admin_v1_peers` method connects a node to a list of peers.
#### Parameters
Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `peers` | `Array`[`String`] | mandatory | Array of peer `multiaddrs` to connect to. Each `multiaddr` must contain the [location and identity addresses](https://docs.libp2p.io/concepts/addressing/) of a peer. |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
Response
- **`Bool`** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Example usage
### Store API
Store API
#### `get_waku_v2_store_v1_messages`
### `get_waku_v2_store_v1_messages`
This method is part of the `store` API and the specific resources to retrieve are (historical) `messages`. The protocol (`waku`) is on `v2`, whereas the Store API definition is on `v1`.
This method is part of the `store` API and
the specific resources to retrieve are (historical) `messages`.
The protocol (`waku`) is on `v2`, whereas the Store API definition is on `v1`.
1. `get` *all* the historical messages for content topic **"/waku/2/default-content/proto"**; no paging required
1.`get` *all* the historical messages for content topic
**"/waku/2/default-content/proto"**; no paging required
#### Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":["", [{"contentTopic":"/waku/2/default-content/proto"}]]}' --header "Content-Type: application/json" http://localhost:8545```
```curl
curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":["", [{"contentTopic":"/waku/2/default-content/proto"}]]}' --header "Content-Type: application/json" http://localhost:8545
```
```jsonrpc
{
@@ -481,7 +588,7 @@ This method is part of the `store` API and the specific resources to retrieve ar
}
```
#### Response
Response
```jsonrpc
{
@@ -513,11 +620,16 @@ This method is part of the `store` API and the specific resources to retrieve ar
---
2. `get` a single page of historical messages for content topic **"/waku/2/default-content/proto"**; 2 messages per page, backward direction. Since this is the initial query, no `cursor` is provided, so paging will be performed from the end of the list.
2.`get` a single page of historical messages for content topic **"/waku/2/default-content/proto"**;
2 messages per page, backward direction.
Since this is the initial query, no `cursor` is provided,
so paging will be performed from the end of the list.
#### Request
Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
```curl
curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545
```
```jsonrpc
{
@@ -537,7 +649,7 @@ This method is part of the `store` API and the specific resources to retrieve ar
}
```
#### Response
Response
```jsonrpc
{
@@ -571,11 +683,14 @@ This method is part of the `store` API and the specific resources to retrieve ar
---
3. `get` the next page of historical messages for content topic **"/waku/2/default-content/proto"**, using the cursor received above; 2 messages per page, backward direction.
3.`get` the next page of historical messages for content topic **"/waku/2/default-content/proto"**,
using the cursor received above; 2 messages per page, backward direction.
#### Request
Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"cursor":{"digest":"abcdef","receivedTime":1605887187000000000},"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
```curl
curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"cursor":{"digest":"abcdef","receivedTime":1605887187000000000},"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545
```
```jsonrpc
{
@@ -599,7 +714,7 @@ This method is part of the `store` API and the specific resources to retrieve ar
}
```
#### Response
Response
```jsonrpc
{

129
waku/deprecated/18/swap.md
View File

@@ -9,21 +9,49 @@ contributor: Ebube Ud <ebube@status.im>
## Abstract
This specification outlines how we do accounting and settlement based on the provision and usage of resources, most immediately bandwidth usage and/or storing and retrieving of Waku message. This enables nodes to cooperate and efficiently share resources, and in the case of unequal nodes to settle the difference through a relaxed payment mechanism in the form of sending cheques.
This specification outlines how we do accounting and settlement based on the provision
and usage of resources, most immediately bandwidth usage and/or
storing and retrieving of Waku message.
This enables nodes to cooperate and efficiently share resources,
and in the case of unequal nodes to settle the difference
through a relaxed payment mechanism in the form of sending cheques.
**Protocol identifier***: `/vac/waku/swap/2.0.0-beta1`
## Motivation
The Waku network makes up a service network, and some nodes provide a useful service to other nodes. We want to account for that, and when imbalances arise, settle this. The core of this approach has some theoretical backing in game theory, and variants of it have practically been proven to work in systems such as Bittorrent. The specific model use was developed by the Swarm project (previously part of Ethereum), and we re-use contracts that were written for this purpose.
The Waku network makes up a service network, and
some nodes provide a useful service to other nodes.
We want to account for that, and when imbalances arise, settle this.
The core of this approach has some theoretical backing in game theory, and
variants of it have practically been proven to work in systems such as Bittorrent.
The specific model use was developed by the Swarm project
(previously part of Ethereum), and
we re-use contracts that were written for this purpose.
By using a delayed payment mechanism in the form of cheques, a barter-like mechanism can arise, and nodes can decide on their own policy as opposed to be strictly tied to a specific payment scheme. Additionally, this delayed settlement eases requirements on the underlying network in terms of transaction speed or costs.
By using a delayed payment mechanism in the form of cheques,
a barter-like mechanism can arise, and
nodes can decide on their own policy
as opposed to be strictly tied to a specific payment scheme.
Additionally, this delayed settlement eases requirements
on the underlying network in terms of transaction speed or costs.
Theoretically, nodes providing and using resources over a long, indefinite, period of time can be seen as an iterated form of [Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma). Specifically, and more intuitively, since we have a cost and benefit profile for each provision/usage (of Waku Message's, e.g.), and the pricing can be set such that mutual cooperation is incentivized, this can be analyzed as a form of donations game.
Theoretically, nodes providing and using resources over a long,
indefinite, period of time can be seen as an iterated form of
[Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma).
Specifically, and more intuitively,
since we have a cost and benefit profile for each provision/usage
(of Waku Message's, e.g.), and
the pricing can be set such that mutual cooperation is incentivized,
this can be analyzed as a form of donations game.
## Game Theory - Iterated prisoner's dilemma / donation game
What follows is a sketch of what the game looks like between two nodes. We can look at it as a special case of iterated prisoner's dilemma called a [Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game) where each node can cooperate with some benefit `b` at a personal cost `c`, where `b>c`.
What follows is a sketch of what the game looks like between two nodes.
We can look at it as a special case of iterated prisoner's dilemma called a
[Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game)
where each node can cooperate with some benefit `b` at a personal cost `c`,
where `b>c`.
From A's point of view:
@@ -32,7 +60,11 @@ A/B | Cooperate | Defect
Cooperate | b-c | -c
Defect | b | 0
What this means is that if A and B cooperates, A gets some benefit `b` minus a cost `c`. If A cooperates and B defects she only gets the cost, and if she defects and B cooperates A only gets the benefit. If both defect they get neither benefit nor cost.
What this means is that if A and B cooperates,
A gets some benefit `b` minus a cost `c`.
If A cooperates and B defects she only gets the cost,
and if she defects and B cooperates A only gets the benefit.
If both defect they get neither benefit nor cost.
The generalized form of PD is:
@@ -50,18 +82,32 @@ And the following holds:
In our case, this means `b>b-c>0>-c` and `2(b-c)> b-c` which is trivially true.
As this is an iterated game with no clear finishing point in most circumstances, a tit-for-tat strategy is simple, elegant and functional. To be more theoretically precise, this also requires reasonable assumptions on error rate and discount parameter. This captures notions such as "does the perceived action reflect the intended action" and "how much do you value future (uncertain) actions compared to previous actions". See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation) for more details. In specific circumstances, nodes can choose slightly different policies if there's a strong need for it. A policy is simply how a node chooses to act given a set of circumstances.
As this is an iterated game with no clear finishing point in most circumstances,
a tit-for-tat strategy is simple, elegant and functional.
To be more theoretically precise,
this also requires reasonable assumptions on error rate and discount parameter.
This captures notions such as
"does the perceived action reflect the intended action" and
"how much do you value future (uncertain) actions compared to previous actions".
See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
for more details.
In specific circumstances,
nodes can choose slightly different policies if there's a strong need for it.
A policy is simply how a node chooses to act given a set of circumstances.
A tit-for-tat strategy basically means:
- cooperate first (perform service/beneficial action to other node)
- defect when node stops cooperating (disconnect and similar actions), i.e. when it stops performing according to set parameters re settlement
- defect when node stops cooperating (disconnect and similar actions),
i.e. when it stops performing according to set parameters re settlement
- resume cooperation if other node does so
This can be complemented with node selection mechanisms.
## SWAP protocol overview
We use SWAP for accounting and settlement in conjunction with other request/reply protocols in Waku v2,
We use SWAP for accounting and
settlement in conjunction with other request/reply protocols in Waku v2,
where accounting is done in a pairwise manner.
It is an acronym with several possible meanings (as defined in the Book
of Swarm), for example:
@@ -71,39 +117,63 @@ of Swarm), for example:
- send waiver as payment
- start without a penny
This approach is based on communicating payment thresholds and sending cheques as indications of later payments.
This approach is based on communicating payment thresholds and
sending cheques as indications of later payments.
Communicating payment thresholds MAY be done out-of-band or as part of the handshake.
Sending cheques is done once payment threshold is hit.
See [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth) section 3.2. on Peer-to-peer accounting etc., for more context and details.
See [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth)
section 3.2. on Peer-to-peer accounting etc., for more context and details.
### Accounting
Nodes perform their own accounting for each relevant peer based on some "volume"/bandwidth metric. For now we take this to mean the number of `WakuMessage`s exchanged.
Nodes perform their own accounting for each relevant peer
based on some "volume"/bandwidth metric.
For now we take this to mean the number of `WakuMessage`s exchanged.
Additionally, a price is attached to each unit. Currently, this is simply a "karma counter" and equal to 1 per message.
Additionally, a price is attached to each unit.
Currently, this is simply a "karma counter" and equal to 1 per message.
Each accounting balance SHOULD be w.r.t. to a given protocol it is accounting for.
NOTE: This may later be complemented with other metrics, either as part of SWAP or more likely outside of it. For example, online time can be communicated and attested to as a form of enhanced quality of service to inform peer selection.
NOTE: This may later be complemented with other metrics,
either as part of SWAP or more likely outside of it.
For example, online time can be communicated and
attested to as a form of enhanced quality of service to inform peer selection.
### Flow
Assuming we have two store nodes, one operating mostly as a client (A) and another as server (B).
Assuming we have two store nodes,
one operating mostly as a client (A) and another as server (B).
1. Node A performs a handshake with B node. B node responds and both nodes communicate their payment threshold.
2. Node A and B creates an accounting entry for the other peer, keep track of peer and current balance.
3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`. Based on the number of WakuMessages in the response, both nodes update their accounting records.
4. When payment threshold is reached, Node A sends over a cheque to reach a neutral balance. Settlement of this is currently out of scope, but would occur through a SWAP contract (to be specified). (mock and hard phase).
1. Node A performs a handshake with B node.
B node responds and both nodes communicate their payment threshold.
2. Node A and B creates an accounting entry for the other peer,
keep track of peer and current balance.
3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`.
Based on the number of WakuMessages in the response,
both nodes update their accounting records.
4. When payment threshold is reached,
Node A sends over a cheque to reach a neutral balance.
Settlement of this is currently out of scope,
but would occur through a SWAP contract (to be specified).
(mock and hard phase).
5. If disconnect threshold is reached, Node B disconnects Node A (mock and hard phase).
Note that not all of these steps are mandatory in initial stages, see below for more details. For example, the payment threshold MAY initially be set out of bounds, and policy is only activated in the mock and hard phase.
Note that not all of these steps are mandatory in initial stages,
see below for more details.
For example, the payment threshold MAY initially be set out of bounds,
and policy is only activated in the mock and hard phase.
### Protobufs
We use protobuf to specify the handshake and signature. This current protobuf is a work in progress. This is needed for mock and hard phase.
We use protobuf to specify the handshake and signature.
This current protobuf is a work in progress.
This is needed for mock and hard phase.
A handshake gives initial information about payment thresholds and possibly other information. A cheque is best thought of as a promise to pay at a later date.
A handshake gives initial information about payment thresholds and
possibly other information.
A cheque is best thought of as a promise to pay at a later date.
```protobuf
@@ -126,11 +196,13 @@ message Cheque {
## Incremental integration and roll-out
To incrementally integrate this into Waku v2, we have divided up the roll-out into three phases:
To incrementally integrate this into Waku v2,
we have divided up the roll-out into three phases:
- Soft - accounting only
- Mock - send mock cheques and take word for it
- Hard Test - blockchain integration and deployed to public testnet (Goerli, Optimism testnet or similar)
- Hard Test - blockchain integration and deployed to public testnet
(Goerli, Optimism testnet or similar)
- Hard Main - deployed to a public mainnet
An implementation MAY support any of these phases.
@@ -140,7 +212,8 @@ An implementation MAY support any of these phases.
In the soft phase only accounting is performed, without consequence for the
peers. No disconnect or sending of cheques is performed at this tage.
SWAP protocol is performed in conjunction with another request-reply protocol to account for its usage.
SWAP protocol is performed in conjunction with another request-reply protocol
to account for its usage.
It SHOULD be done for [13/WAKU2-STORE](../../core/13/store.md)
and it MAY be done for other request/reply protocols.
@@ -151,10 +224,12 @@ and SHOULD indicate when a peer is out of bounds (either of its thresholds met).
In the mock phase, we send mock cheques and send cheques/disconnect peers as appropriate.
- If a node reaches a disconnect threshold, which MUST be outside the payment threshold, it SHOULD disconnect the other peer.
- If a node reaches a disconnect threshold,
which MUST be outside the payment threshold, it SHOULD disconnect the other peer.
- If a node is within payment balance, the other node SHOULD stay connected to it.
- If a node receives a valid Cheque it SHOULD update its internal accounting records.
- If any node behaves badly, the other node is free to disconnect and pick another node.
- If any node behaves badly, the other node is free to disconnect and
pick another node.
- Peer rating is out of scope of this specification.
### Hard phase

431
waku/deprecated/5/waku0.md
View File

@@ -11,11 +11,25 @@ contributors:
- Kim De Mey <kimdemey@status.im>
---
This specification describes the format of Waku messages within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic messages (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
This specification describes the format of Waku messages within the ÐΞVp2p Wire Protocol.
This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627).
Waku is a fork of the original Whisper protocol that enables better usability
for resource restricted devices,
such as mostly-offline bandwidth-constrained smartphones.
It does this through (a) light node support,
(b) historic messages (with a mailserver)
(c) expressing topic interest for better bandwidth usage and
(d) basic rate limiting.
## Motivation
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku messages in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
Waku was created to incrementally improve in areas that Whisper is lacking in,
with special attention to resource restricted devices.
We specify the standard for Waku messages
in order to ensure forward compatibility of different Waku clients,
backwards compatibility with Whisper clients,
as well as to allow multiple implementations of Waku and its capabilities.
We also modify the language to be more unambiguous, concise and consistent.
## Definitions
@@ -29,23 +43,47 @@ Waku was created to incrementally improve in areas that Whisper is lacking in, w
### Use of DevP2P
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
For nodes to communicate, they MUST implement devp2p and run RLPx.
They MUST have some way of connecting to other nodes.
Node discovery is largely out of scope for this spec,
but see the appendix for some suggestions on how to do this.
### Gossip based routing
In Whisper, messages are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Messages are eligible for retransmission until their TTL expires. A node SHOULD relay messages to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [mailserver](./mailserver.md) response. A node SHOULD NOT send a message to a peer that it has already sent before.
In Whisper, messages are gossiped between peers.
Whisper is a form of rumor-mongering protocol
that works by flooding to its connected peers based on some factors.
Messages are eligible for retransmission until their TTL expires.
A node SHOULD relay messages to all connected nodes
if an envelope matches their PoW and bloom filter settings.
If a node works in light mode, it MAY choose not to forward envelopes.
A node MUST NOT send expired envelopes,
unless the envelopes are sent as a [mailserver](./mailserver.md) response.
A node SHOULD NOT send a message to a peer that it has already sent before.
## Wire Specification
### Use of RLPx transport protocol
All Waku messages are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
All Waku messages are sent as devp2p RLPx transport protocol,
version 5[^1] packets.
These packets MUST be RLP-encoded arrays of data containing two objects:
packet code followed by another object (whose type depends on the packet code).
See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and
the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf)
for more details on RLP.
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
Waku is a RLPx subprotocol called `waku` with version `0`.
The version number corresponds to the major version in the header spec.
Minor versions should not break compatibility of `waku`,
this would result in a new major.
(Some exceptions to this apply in the Draft stage
of where client implementation is rapidly change).
### ABNF specification
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
we have the following format:
```abnf
; Packet codes 0 - 127 are reserved for Waku protocol
@@ -129,21 +167,23 @@ p2p-message = 1*waku-envelope
packet-format = "[" packet-code packet-format "]"
required-packet = 0 status /
1 messages /
22 status-update /
1 messages /
22 status-update /
optional-packet = 126 p2p-request / 127 p2p-message
packet = "[" required-packet [ optional-packet ] "]"
```
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
All primitive types are RLP encoded. Note that, per RLP specification,
integers are encoded starting from `0x00`.
### Packet Codes
The message codes reserved for Waku protocol: 0 - 127.
Messages with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
Messages with unknown codes MUST be ignored without generating any error,
for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
@@ -170,17 +210,25 @@ The Status message serves as a Waku handshake and peers MUST exchange this
message upon connection. It MUST be sent after the RLPx handshake and prior to
any other Waku messages.
A Waku node MUST await the Status message from a peer before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status message from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
A Waku node MUST await the Status message from a peer
before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status message from a peer,
before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status message, the node SHOULD validate the message
received and validated the Status message. Note that its peer might not be in
the same state.
When a node is receiving other Waku messages from a peer before a Status
message is received, the node MUST ignore these messages and SHOULD disconnect from that peer. Status messages received after the handshake is completed MUST also be ignored.
message is received,
the node MUST ignore these messages and SHOULD disconnect from that peer.
Status messages received after the handshake is completed MUST also be ignored.
The status message MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
The status message MUST contain an association list containing various options.
All options within this association list are OPTIONAL,
ordering of the key-value pairs is not guaranteed and
therefore MUST NOT be relied on.
Unknown keys in the association list SHOULD be ignored.
#### Messages
@@ -188,87 +236,130 @@ This packet is used for sending the standard Waku envelopes.
#### Status Update
The Status Update message is used to communicate an update of the settings of the node.
The Status Update message is used to communicate an update
of the settings of the node.
The format is the same as the Status message, all fields are optional.
If none of the options are specified the message MUST be ignored and considered a noop.
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
be transmitted.
If none of the options are specified the message MUST be ignored and
considered a noop.
Fields that are omitted are considered unchanged,
fields that haven't changed SHOULD not be transmitted.
**PoW Requirement update**
##### PoW Requirement update
When PoW is updated, peers MUST NOT deliver the sender envelopes with PoW lower than specified in this message.
When PoW is updated, peers MUST NOT deliver the sender envelopes
with PoW lower than specified in this message.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by message size and TTL:
PoW is defined as average number of iterations,
required to find the current BestBit
(the number of leading zero bits in the hash), divided by message size and TTL:
PoW = (2**BestBit) / (size * TTL)
> PoW = (2**BestBit) / (size * TTL)
PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
```rust
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
```
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
where size is the size of the RLP-encoded envelope,
excluding `env_nonce` field (size of `short_rlp(envelope)`).
**Bloom filter update**
##### Bloom filter update
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
The bloom filter is used to identify a number of topics
to a peer without compromising (too much)
privacy over precisely what topics are of interest.
Precise control over the information content (and thus efficiency of the filter)
may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
Blooms are formed by the bitwise OR operation on a number of bloomed topics.
The bloom function takes the topic and projects them onto a 512-bit slice.
At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
The projection function is defined as a mapping from a 4-byte slice S
to a 512-bit slice D; for ease of explanation, S will dereference to bytes,
whereas D will dereference to bits.
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
```python
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
```
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
A full bloom filter (all the bits set to 1)
means that the node is to be considered a `Full Node` and it will accept any topic.
If both Topic Interest and bloom filter are specified, Topic Interest always takes precedence and bloom filter MUST be ignored.
If both Topic Interest and bloom filter are specified,
Topic Interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current Topic Interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
If only bloom filter is specified, the current Topic Interest MUST be discarded and
only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
A bloom filter with all bits set to 0 signals
that the node is not currently interested in receiving any envelope.
**Topic Interest update**
##### Topic Interest update
This packet is used by Waku nodes for sharing their interest in messages with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
This packet is used by Waku nodes for sharing their interest
in messages with specific topics.
It does this in a more bandwidth considerate way,
at the expense of some metadata protection.
Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics.
If you are interested in more topics than that,
this is currently underspecified and likely requires updating it.
The constant is subject to change.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only Topic Interest is specified,
the current bloom filter MUST be discarded and
only the updated Topic Interest MUST be used when forwarding or posting envelopes.
If only Topic Interest is specified, the current bloom filter MUST be discarded and only the updated Topic Interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node
is not currently interested in receiving any envelope.
An empty array signals that the node is not currently interested in receiving any envelope.
**Rate Limits update**
##### Rate Limits update
This packet is used for informing other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
In order to provide basic Denial-of-Service attack protection,
each node SHOULD define its own rate limits.
The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the rate limits packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD broadcast its rate limits to its peers using the rate limits packet.
The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
Each node SHOULD respect rate limits advertised by its peers.
The number of packets SHOULD be throttled in order not to exceed peer's rate limits.
If the limit gets exceeded, the connection MAY be dropped by the peer.
**Message Confirmations update**
##### Message Confirmations update
Message confirmations tell a node that a message originating from it has been received by its peers, allowing a node to know whether a message has or has not been received.
Message confirmations tell a node that a message originating
from it has been received by its peers,
allowing a node to know whether a message has or has not been received.
A node MAY send a message confirmation for any batch of messages received with a packet Messages Code.
A node MAY send a message confirmation for any batch of messages
received with a packet Messages Code.
A message confirmation is sent using Batch Acknowledge packet or Message Response packet. The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data.
A message confirmation is sent using Batch Acknowledge packet or
Message Response packet.
The Batch Acknowledge packet is followed by a keccak256 hash
of the envelopes batch data.
The current `version` of the message response is `1`.
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
we have the following format:
```abnf
; a version of the Message Response
@@ -294,159 +385,268 @@ confirmation = "[" version response "]"
```
The supported codes:
`1`: means time sync error which happens when an envelope is too old or created in the future (the root cause is no time sync between nodes).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcast by one or more peers.
`1`: means time sync error which happens when an envelope is too old or
created in the future (the root cause is no time sync between nodes).
The drawback of sending message confirmations
is that it increases the noise in the network because for each sent message,
a corresponding confirmation is broadcast by one or more peers.
#### P2P Request
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting old messages from the [Waku Mail Server](./mailserver.md).
This packet is used for sending Dapp-level peer-to-peer requests,
e.g. Waku Mail Client requesting old messages from the [Waku Mail Server](./mailserver.md).
#### P2P Message
This packet is used for sending the peer-to-peer messages, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of old (expired) messages, which is otherwise not allowed.
This packet is used for sending the peer-to-peer messages,
which are not supposed to be forwarded any further.
E.g. it might be used by the Waku Mail Server for delivery of old
(expired) messages, which is otherwise not allowed.
### Payload Encryption
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme
with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
### Packet code Rationale
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions.
Packet code `0x02` and `0x03` were previously used in Whisper but
are deprecated as of Waku v0.4
Packet code `0x22` is used to dynamically change the settings of a node.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without P2P messages it would be impossible to deliver the old messages, since they will be recognized as expired, and the peer will be disconnected for violating the Whisper protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client.
Without P2P messages it would be impossible to deliver the old messages,
since they will be recognized as expired,
and the peer will be disconnected for violating the Whisper protocol.
They might be useful for other purposes
when it is not possible to spend time on PoW,
e.g. if a stock exchange will want to provide live feed about the latest trades.
## Additional capabilities
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Waku supports multiple capabilities.
These include light node, rate limiting and bridging of traffic.
Here we list these capabilities, how they are identified,
what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on [specification](mailserver.md).
Additionally there is the capability of a mailserver
which is documented in its on [specification](mailserver.md).
### Light node
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
The rationale for light nodes is to allow for interaction with waku
on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming messages, they MUST only send their own messages. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in messages being dropped between the two.
Light nodes MUST NOT forward any incoming messages,
they MUST only send their own messages.
When light nodes happen to connect to each other,
they SHOULD disconnect.
As this would result in messages being dropped between the two.
Light nodes are identified by the `light_node` value in the status message.
### Accounting for resources (experimental)
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
Nodes MAY implement accounting, keeping track of resource usage.
It is heavily inspired by
Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf),
and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Each node keeps track of resource usage with all other nodes.
Whenever an envelope is received from a node that is expected
(fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
Every epoch (say, every minute or every time an event happens)
statistics SHOULD be aggregated and saved by the client:
| peer | sent | received |
|-------|------|----------|
| peer1 | 0 | 123 |
| peer2 | 10 | 40 |
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
In later versions this will be amended by nodes communication thresholds,
settlements and disconnect logic.
## Upgradability and Compatibility
### General principles and policy
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
These are policies that guide how we make decisions when it comes to upgradability,
compatibility, and extensibility:
1. Waku aims to be compatible with previous and future versions.
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
2. In cases where we want to break this compatibility, we do so gracefully and
as a single decision point.
3. To achieve this,
we employ the following two general strategies:
3. To achieve this, we employ the following two general strategies:
- a) Accretion (including protocol negotiation) over changing data
- b) When we want to change things, we give it a new name (for example, a version number).
- b) When we want to change things, we give it a new name
(for example, a version number).
Examples:
- We enable bridging between `shh/6` and `waku/0` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
- When we we want to provide a new set of messages that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
- We enable bridging between `shh/6` and
`waku/0` until such a time as when we are ready to gracefully drop support
for `shh/6` (1, 2, 3).
- When we add parameter fields, we (currently) do so by accreting them in a list,
so old clients can ignore new fields (dynamic list)
and new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future,
we will likely release a new version that gives better support for open,
growable maps (association lists or native map type) (3)
- When we we want to provide a new set of messages that have different requirements,
we do so under a new protocol version and employ protocol versioning.
This is a form of accretion at a level above -
it ensures a client can support both protocols at once and
drop support for legacy versions gracefully. (1,2,3)
### Backwards Compatibility
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
Waku is a different subprotocol from Whisper so it isn't directly compatible.
However, the data format is the same,
so compatibility can be achieved by the use of a bridging mode as described below.
Any client which does not implement certain packet codes
should gracefully ignore the packets with those codes.
This will ensure the forward compatibility.
### Waku-Whisper bridging
`waku/0` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
`waku/0` and `shh/6` are different DevP2P subprotocols,
however they share the same data format making their envelopes compatible.
This means we can bridge the protocols naively, this works as follows.
**Roles:**
- Waku client A, only Waku capability
- Whisper client B, only Whisper capability
- WakuWhisper bridge C, both Waku and Whisper capability
**Flow:**
1. A posts message; B posts message.
2. C picks up message from A and B and relays them both to Waku and Whisper.
3. A receives message on Waku; B on Whisper.
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a message between C1 and C2. I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
**Note**: This flow means if another bridge C1 is active,
we might get duplicate relaying for a message between C1 and C2.
I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B.
Theoretically this bridging chain can get as long as TTL permits.
### Forward Compatibility
It is desirable to have a strategy for maintaining forward compatibility between `waku/0` and future version of waku. Here we outline some concerns and strategy for this.
It is desirable to have a strategy for maintaining forward compatibility
between `waku/0` and future version of waku.
Here we outline some concerns and strategy for this.
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished in the future is by negotiating the versions of subprotocols, within the `hello` message nodes transmit their capabilities along with a version. As suggested in [EIP-8](https://eips.ethereum.org/EIPS/eip-8), if a node connects that has a higher version number for a specific capability, the node with a lower number SHOULD assume backwards compatibility. The node with the higher version will decide if compatibility can be assured between versions, if this is not the case it MUST disconnect.
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
- **Connecting to nodes with multiple versions:**
The way this SHOULD be accomplished in the future
is by negotiating the versions of subprotocols,
within the `hello` message nodes transmit their capabilities along with a version.
As suggested in [EIP-8](https://eips.ethereum.org/EIPS/eip-8),
if a node connects that has a higher version number for a specific capability,
the node with a lower number SHOULD assume backwards compatibility.
The node with the higher version
will decide if compatibility can be assured between versions,
if this is not the case it MUST disconnect.
- **Adding new packet codes:**
New packet codes can be added easily due to the available packet codes.
Unknown packet codes SHOULD be ignored.
Upgrades that add new packet codes SHOULD implement some fallback mechanism
if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:**
New options can be added to the `status-options` association list
in the `status` and `status-update` packet as options are OPTIONAL and
unknown option keys SHOULD be ignored.
A node SHOULD NOT disconnect from a peer
when receiving `status-options` with unknown option keys.
## Appendix A: Security considerations
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](./mailserver.md#security-considerations) can be found in their respective specifications.
There are several security considerations to take into account when running Waku.
Chief among them are: scalability, DDoS-resistance and privacy.
These also vary depending on what capabilities are used.
The security considerations for extra capabilities such as [mailservers](./mailserver.md#security-considerations)
can be found in their respective specifications.
### Scalability and UX
**Bandwidth usage:**
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
In version 0 of Waku, bandwidth usage is likely to be an issue.
For more investigation into this,
see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
**Gossip-based routing:**
Use of gossip-based routing doesn't necessarily scale. It means each node can see a message multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
Use of gossip-based routing doesn't necessarily scale.
It means each node can see a message multiple times,
and having too many light nodes can cause propagation probability that is too low.
See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/)
for more and a possible Kademlia based alternative.
**Lack of incentives:**
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
Waku currently lacks incentives to run nodes,
which means node operators are more likely to create centralized choke points.
### Privacy
**Light node privacy:**
The main privacy concern with light nodes is that directly connected peers will know that a message originates from them (as it are the only ones it sends). This means nodes can make assumptions about what messages (topics) their peers are interested in.
The main privacy concern with light nodes
is that directly connected peers will know that a message originates from them
(as it are the only ones it sends).
This means nodes can make assumptions about what messages (topics)
their peers are interested in.
**Bloom filter privacy:**
By having a bloom filter where only the topics you are interested in are set, you reveal which messages you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
By having a bloom filter where only the topics you are interested in are set,
you reveal which messages you are interested in.
This is a fundamental tradeoff between bandwidth usage and privacy,
though the tradeoff space is likely suboptimal in terms of the
[Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
**Privacy guarantees not rigorous:**
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
Privacy for Whisper / Waku haven't been studied rigorously for various threat models
like global passive adversary, local active attacker, etc.
This is unlike e.g. Tor and mixnets.
**Topic hygiene:**
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
Similar to bloom filter privacy,
if you use a very specific topic you reveal more information.
See scalability model linked above.
### Spam resistance
**PoW bad for heterogeneous devices:**
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
Proof of work is a poor spam prevention mechanism.
A mobile device can only have a very low PoW
in order not to use too much CPU / burn up its phone battery.
This means someone can spin up a powerful node and overwhelm the network.
### Censorship resistance
**Devp2p TCP port blockable:**
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
By default Devp2p runs on port `30303`,
which is not commonly used for any other service.
This means it is easy to censor, e.g. airport WiFi.
This can be mitigated somewhat by running on e.g. port `80` or `443`,
but there are still outstanding issues.
See libp2p and Tor's Pluggable Transport for how this can be improved.
## Appendix B: Implementation Notes
@@ -461,17 +661,22 @@ By default Devp2p runs on port `30303`, which is not commonly used for any other
Notes useful for implementing Waku mode.
1. Avoid duplicate envelopes
- Avoid duplicate envelopes:
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
To avoid duplicate envelopes, only connect to one Waku node.
Benign duplicate envelopes is an intrinsic property of Whisper
which often leads to a N factor increase in traffic,
where N is the number of peers you are connected to.
2. Topic specific recommendations
- Topic specific recommendations -
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
Consider partition topics based on some usage,
to avoid too much traffic on a single topic.
### Node discovery
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
Resource restricted devices SHOULD use
[EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
Known static nodes MAY also be used.
@@ -487,16 +692,21 @@ Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f2417984
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
- Clarify the preferred way of handling unknown keys
in the `status-options` association list.
- Correct spec/implementation mismatch:
Change RLP keys to be the their int values in order to reflect production behavior
### Version 0.4
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
- Simplify implementation matrix with latest state
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
- Introduces a new required packet code Status Code (`0x22`)
for communicating option changes
- Deprecates the following packet codes:
PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`),
Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
- Increased `topic-interest` capacity from 1000 to 10000
### Version 0.3
@@ -506,7 +716,8 @@ Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954
- Recommend DNS based node discovery over other Discovery methods.
- Mark spec as Draft mode in terms of its lifecycle.
- Simplify Changelog and misc formatting.
- Handshake/Status message not compatible with shh/6 nodes; specifying options as association list.
- Handshake/Status message not compatible with shh/6 nodes;
specifying options as association list.
- Include topic-interest in Status handshake.
- Upgradability policy.
- `topic-interest` packet code.
@@ -522,7 +733,8 @@ Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/wak
- More details on handshake modifications.
- Accounting for resources mode (experimental)
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
- Appendix with implementation notes and
implementation matrix across various clients with breakdown per capability.
- More details on handshake and parameters.
- Describe rate limits in more detail.
- More details on mailserver and mail client API.
@@ -549,7 +761,6 @@ confirmations-enabled and rate-limits
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## Footnotes
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.

7
waku/deprecated/README.md
View File

@@ -1,6 +1,7 @@
# Deprecated RFCs
# Deprecated RFCs
Deprecated specifications are no longer used in Waku products.
This subdirectory is for achrive purpose and
This subdirectory is for achrive purpose and
should not be used in production ready implementations.
Visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.
Visit [Waku RFCs](https://github.com/waku-org/specs)
for new Waku specifications under discussion.

BIN
waku/informational/.DS_Store vendored Normal file
View File

Binary file not shown.

12
waku/informational/22/toy-chat.md
View File

@@ -19,12 +19,14 @@ This protocol is mainly used to:
Currently, all main Waku v2 implementations support the toy chat protocol:
[nim-waku](https://github.com/status-im/nim-waku/blob/master/examples/v2/chat2.nim),
js-waku ([NodeJS](https://github.com/status-im/js-waku/tree/main/examples/cli-chat) and [web](https://github.com/status-im/js-waku/tree/main/examples/web-chat))
js-waku ([NodeJS](https://github.com/status-im/js-waku/tree/main/examples/cli-chat)
and [web](https://github.com/status-im/js-waku/tree/main/examples/web-chat))
and [go-waku](https://github.com/status-im/go-waku/tree/master/examples/chat2).
Note that this is completely separate from the protocol the Status app is using for its chat functionality.
Note that this is completely separate from the protocol the Status app
is using for its chat functionality.
# Design
## Design
The chat protocol enables sending and receiving messages in a chat room.
There is currently only one chat room, which is tied to the content topic.
@@ -32,7 +34,7 @@ The messages SHOULD NOT be encrypted.
The `contentTopic` MUST be set to `/toy-chat/2/huilong/proto`.
# Payloads
## Payloads
```protobuf
syntax = "proto3";
@@ -48,6 +50,6 @@ message Chat2Message {
- `nick`: The nickname of the user sending the message,
- `payload`: The text of the messages, UTF-8 encoded.
# Copyright
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

81
waku/informational/23/topics.md
View File

@@ -16,12 +16,12 @@ In [10/WAKU2 spec](../../standards/core/10/waku2.md) there are two types of topi
- pubsub topics, used for routing
- Content topics, used for content-based filtering
## Pubsub Topics
Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](../../standards/core/11/relay.md)),
and can be named implicitly by Waku sharding (see [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
This document comprises recommendations for explicitly naming pubsub topics (e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
This document comprises recommendations for explicitly naming pubsub topics
(e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
### Pubsub Topic Format
@@ -29,26 +29,32 @@ Pubsub topics SHOULD follow the following structure:
`/waku/2/{topic-name}`
This namespaced structure makes compatibility, discoverability, and automatic handling of new topics easier.
This namespaced structure makes compatibility, discoverability,
and automatic handling of new topics easier.
The first two parts indicate
1) it relates to the Waku protocol domain, and
2) the version is 2.
If applicable, it is RECOMMENDED to structure `{topic-name}` in a hierarchical way as well.
If applicable, it is RECOMMENDED to structure `{topic-name}`
in a hierarchical way as well.
> *Note*: In previous versions of this document, the structure was `/waku/2/{topic-name}/{encoding}`.
The now deprecated `/{encoding}` was always set to `/proto`,
which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition) in pubsub is serialized/encoded as protobuf.
which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition)
in pubsub is serialized/encoded as protobuf.
The inspiration for this format was taken from
[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md) must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md)
must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
which specifies the wire format as protobuf,`/proto` is the only valid encoding.
This makes the `/proto` indication obsolete.
The encoding of the `payload` field of a Waku Message is indicated by the `/{encoding}` part of the content topic name.
The encoding of the `payload` field of a Waku Message
is indicated by the `/{encoding}` part of the content topic name.
Specifying an encoding is only significant for the actual payload/data field.
Waku preserves this option by allowing to specify an encoding for the WakuMessage payload field as part of the content topic name.
Waku preserves this option by allowing to specify an encoding
for the WakuMessage payload field as part of the content topic name.
### Default PubSub Topic
@@ -56,25 +62,27 @@ The Waku v2 default pubsub topic is:
`/waku/2/default-waku/proto`
The `{topic name}` part is `default-waku/proto`, which indicates it is default topic for exchanging WakuMessages;
The `{topic name}` part is `default-waku/proto`,
which indicates it is default topic for exchanging WakuMessages;
`/proto` remains for backwards compatibility.
### Application Specific Names
Larger apps can segregate their pubsub meshes using topics named like:
```
```text
/waku/2/status/
/waku/2/walletconnect/
```
This indicates that these networks carry WakuMessages, but for different domains completely.
This indicates that these networks carry WakuMessages,
but for different domains completely.
### Named Topic Sharding Example
The following is an example of named sharding, as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md).
```
```text
waku/2/waku-9_shard-0/
...
waku/2/waku-9_shard-9/
@@ -86,10 +94,12 @@ This indicates explicitly that the network traffic has been partitioned into 10
The other type of topic that exists in Waku v2 is a content topic.
This is used for content based filtering.
See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md) for where this is specified.
See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md)
for where this is specified.
Note that this doesn't impact routing of messages between relaying nodes,
but it does impact how request/reply protocols such as
[12/WAKU2-FILTER](../../standards/core/12/filter.md) and [13/WAKU2-STORE](../../standards/core/13/store.md) are used.
but it does impact how request/reply protocols such as
[12/WAKU2-FILTER](../../standards/core/12/filter.md) and
[13/WAKU2-STORE](../../standards/core/13/store.md) are used.
This is especially useful for nodes that have limited bandwidth,
and only want to pull down messages that match this given content topic.
@@ -114,16 +124,19 @@ Application names should be unique to avoid conflicting issues with other protoc
Applications should specify their version (if applicable) in the version field.
The `{content-topic-name}` portion of the content topic is up to the application,
and depends on the problem domain.
It can be hierarchical, for instance to separate content, or to indicate different bandwidth and privacy guarantees.
The encoding field indicates the serialization/encoding scheme for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
It can be hierarchical, for instance to separate content, or
to indicate different bandwidth and privacy guarantees.
The encoding field indicates the serialization/encoding scheme
for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
## Differences with Waku v1
In [5/WAKU1](../../deprecated/5/waku0.md) there is no actual routing.
All messages are sent to all other nodes.
This means that we are implicitly using the same pubsub topic that would be something like:
This means that we are implicitly using the same pubsub topic
that would be something like:
```
```text
/waku/1/default-waku/rlp
```
@@ -135,19 +148,21 @@ To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](../../standards/core/15
For mapping Waku v1 topics to Waku v2 content topics,
the following structure for the content topic SHOULD be used:
```
```text
/waku/1/<4bytes-waku-v1-topic>/rfc26
```
The `<4bytes-waku-v1-topic>` SHOULD be the lowercase hex representation of the 4-byte Waku v1 topic.
The `<4bytes-waku-v1-topic>` SHOULD be the lowercase hex representation
of the 4-byte Waku v1 topic.
A `0x` prefix SHOULD be used.
`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](../../standards/application/26/payload.md).
See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description of the bridged fields.
See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description
of the bridged fields.
This creates a direct mapping between the two protocols.
For example:
```
```text
/waku/1/0x007f80ff/rfc26
```
@@ -158,13 +173,13 @@ Copyright and related rights waived via
## References
* [10/WAKU2 spec](../../standards/core/10/waku2.md)
* [11/WAKU2-RELAY](../../standards/core/11/relay.md)
* [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)
* [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
* [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
* [12/WAKU2-FILTER](../../standards/core/12/filter.md)
* [13/WAKU2-STORE](../../standards/core/13/store.md)
* [6/WAKU1](../../deprecated/5/waku0.md)
* [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
* [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)
- [10/WAKU2 spec](../../standards/core/10/waku2.md)
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
- [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)
- [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
- [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
- [12/WAKU2-FILTER](../../standards/core/12/filter.md)
- [13/WAKU2-STORE](../../standards/core/13/store.md)
- [6/WAKU1](../../deprecated/5/waku0.md)
- [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
- [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)

56
waku/informational/27/peers.md
View File

@@ -7,24 +7,31 @@ editor: Hanno Cornelius <hanno@status.im>
contributors:
---
`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and peer management features to be implemented by Waku v2 clients.
`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and
peer management features to be implemented by Waku v2 clients.
In this context, peer _storage_ refers to a client's ability to keep track of discovered or statically-configured peers and their metadata.
In this context, peer _storage_ refers to a client's ability to keep track of discovered
or statically-configured peers and their metadata.
It also deals with matters of peer _persistence_,
or the ability to store peer data on disk to resume state after a client restart.
Peer _management_ is a closely related concept and refers to the set of actions a client MAY choose to perform based on its knowledge of its connected peers,
Peer _management_ is a closely related concept and
refers to the set of actions a client MAY choose to perform
based on its knowledge of its connected peers,
e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
## Peer store
The peer store SHOULD be an in-memory data structure where information about discovered or configured peers are stored.
It SHOULD be considered the main source of truth for peer-related information in a Waku v2 client.
The peer store SHOULD be an in-memory data structure
where information about discovered or configured peers are stored.
It SHOULD be considered the main source of truth
for peer-related information in a Waku v2 client.
Clients MAY choose to persist this store on-disk.
### Tracked peer metadata
It is RECOMMENDED that a Waku v2 client tracks at least the following information about each of its peers in a peer store:
It is RECOMMENDED that a Waku v2 client tracks at least the following information
about each of its peers in a peer store:
| Metadata | Description |
| --- | --- |
@@ -36,13 +43,16 @@ It is RECOMMENDED that a Waku v2 client tracks at least the following informatio
### Peer connectivity
A Waku v2 client SHOULD track _at least_ the following connectivity states for each of its peers:
- **`NotConnected`**: The peer has been discovered or configured on this client,
A Waku v2 client SHOULD track _at least_ the following connectivity states
for each of its peers:
- **`NotConnected`**: The peer has been discovered or configured on this client,
but no attempt has yet been made to connect to this peer.
This is the default state for a new peer.
- **`CannotConnect`**: The client attempted to connect to this peer, but failed.
- **`CanConnect`**: The client was recently connected to this peer and disconnected gracefully.
- **`Connected`**: The client is actively connected to this peer.
- **`CannotConnect`**: The client attempted to connect to this peer, but failed.
- **`CanConnect`**: The client was recently connected to this peer and
disconnected gracefully.
- **`Connected`**: The client is actively connected to this peer.
This list does not preclude clients from tracking more advanced connectivity metadata,
such as a peer's blacklist status (see [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)).
@@ -55,30 +65,40 @@ Peer persistence MAY be used to resume peer connections after a client restart.
## Peer management
Waku v2 clients will have different requirements when it comes to managing the peers tracked in the [**peer store**](#peer-store).
Waku v2 clients will have different requirements
when it comes to managing the peers tracked in the [**peer store**](#peer-store).
It is RECOMMENDED that clients support:
- [automatic reconnection](#reconnecting-peers) to peers under certain conditions
- [connection keep-alive](#connection-keep-alive)
### Reconnecting peers
A Waku v2 client MAY choose to reconnect to previously connected, managed peers under certain conditions.
A Waku v2 client MAY choose to reconnect to previously connected,
managed peers under certain conditions.
Such conditions include, but are not limited to:
- Reconnecting to all `relay`-capable peers after a client restart. This will require [persistent peer storage](#persistence).
- Reconnecting to all `relay`-capable peers after a client restart.
This will require [persistent peer storage](#persistence).
If a client chooses to automatically reconnect to previous peers,
it MUST respect the [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange) specified for GossipSub v1.1 before attempting to reconnect.
it MUST respect the
[backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
specified for GossipSub v1.1 before attempting to reconnect.
This requires keeping track of the [last time each peer was disconnected](#tracked-peer-metadata).
### Connection keep-alive
A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers.
If a client chooses to implement keep-alive on a connection,
it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping) as per `10/WAKU2` [client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients).
The recommended period between pings SHOULD be _at most_ 50% of the shortest idle connection timeout for the specific client and transport.
it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
as per `10/WAKU2` [client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients).
The recommended period between pings SHOULD be _at most_ 50%
of the shortest idle connection timeout for the specific client and transport.
For example, idle TCP connections often times out after 10 to 15 minutes.
> **Implementation note:** the `nim-waku` client currently implements a keep-alive mechanism every `5 minutes`,
> **Implementation note:**
the `nim-waku` client currently implements a keep-alive mechanism every `5 minutes`,
in response to a TCP connection timeout of `10 minutes`.
## Copyright

13
waku/informational/29/config.md
View File

@@ -7,7 +7,8 @@ editor: Hanno Cornelius <hanno@status.im>
contributors:
---
`29/WAKU2-CONFIG` describes the RECOMMENDED values to assign to configurable parameters for Waku v2 clients.
`29/WAKU2-CONFIG` describes the RECOMMENDED values
to assign to configurable parameters for Waku v2 clients.
Since Waku v2 is built on [libp2p](https://github.com/libp2p/specs),
most of the parameters and reasonable defaults are derived from there.
@@ -36,7 +37,8 @@ We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementation
## GossipSub v1.1 parameters
GossipSub v1.1 extended GossipSub v1.0 and introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
We repeat the global parameters here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
We repeat the global parameters here with RECOMMMENDED values
for `11/WAKU2-RELAY` implementations.
| Parameter | Description | RECOMMENDED value |
|----------------|------------------------------------------------------------------------|-------------------|
@@ -46,12 +48,15 @@ We repeat the global parameters here with RECOMMMENDED values for `11/WAKU2-RELA
| `D_score` | Number of peers to retain by score when pruning from oversubscription | `D_low` |
| `D_out` | Number of outbound connections to keep in the mesh. | `D_low` - 1 |
`11/WAKU2-RELAY` clients SHOULD implement a peer scoring mechanism with the parameter constraints as [specified by libp2p](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
`11/WAKU2-RELAY` clients SHOULD implement a peer scoring mechanism
with the parameter constraints as
[specified by libp2p](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
## Other configuration
The following behavioural parameters are not specified by `libp2p`,
but nevertheless describes constraints that `11/WAKU2-RELAY` clients MAY choose to implement.
but nevertheless describes constraints that `11/WAKU2-RELAY` clients
MAY choose to implement.
| Parameter | Description | RECOMMENDED value |
|--------------------|---------------------------------------------------------------------------|-------------------|

37
waku/informational/30/adaptive-nodes.md
View File

@@ -11,7 +11,8 @@ This is an informational spec that show cases the concept of adaptive nodes.
## Node types - a continuum
We can look at node types as a continuum, from more restricted to less restricted, fewer resources to more resources.
We can look at node types as a continuum,
from more restricted to less restricted, fewer resources to more resources.
![Node types - a continuum](./images/adaptive_node_continuum2.png)
@@ -29,11 +30,13 @@ Some examples:
- Desktop: download, leave in background, contribute somewhat
- Cluster: expensive, upkeep, but can contribute a lot
These are also illustrative, so a node in a browser in certain environment might contribute similarly to Desktop.
These are also illustrative,
so a node in a browser in certain environment might contribute similarly to Desktop.
### Adaptive nodes
We call these nodes *adaptive nodes* to highlights different modes of contributing, such as:
We call these nodes *adaptive nodes* to highlights different modes of contributing,
such as:
- Only leeching from the network
- Relaying messages for one or more topics
@@ -55,11 +58,18 @@ Each node can choose which protocols to support, depending on its resources and
![Protocol selection](./images/adaptive_node_protocol_selection2.png)
In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md) etc (12, 13, 19, 21) these correspond to Libp2p protocols.
In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md)
etc (12, 13, 19, 21) these correspond to Libp2p protocols.
However, other protocols like 16/WAKU2-RPC (local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY, Discovery v5 (DevP2P) or interfacing with distributed storage, are running on different network stacks.
However, other protocols like 16/WAKU2-RPC
(local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY,
Discovery v5 (DevP2P) or interfacing with distributed storage,
are running on different network stacks.
This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE, 26/WAKU2-PAYLOAD, or application specific ones. As well as specs that act more as recommendations, such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE,
26/WAKU2-PAYLOAD, or application specific ones.
As well as specs that act more as recommendations,
such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
## Waku network visualization
@@ -67,7 +77,8 @@ We can better visualize the network with some illustrative examples.
### Topology and topics
The first one shows an example topology with different PubSub topics for the relay protocol.
The first one shows an example topology with different PubSub topics
for the relay protocol.
![Waku Network visualization](./images/adaptive_node_network_topology_protocols2.png)
@@ -75,21 +86,25 @@ The first one shows an example topology with different PubSub topics for the rel
![Waku Network visualization legend](./images/adaptive_node_network_topology_protocols_legend.png)
The dotted box shows what content topics (application-specific) a node is interested in.
The dotted box shows what content topics (application-specific)
a node is interested in.
A node that is purely providing a service to the network might not care.
In this example, we see support for toy chat, a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
In this example, we see support for toy chat,
a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
### Auxiliary network
This is a separate component with its own topology.
Behavior and interaction with other protocols specified in Vac RFCs, e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
Behavior and interaction with other protocols specified in Vac RFCs,
e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
### Node Cross Section
This one shows a cross-section of nodes in different dimensions and shows how the connections look different for different protocols.
This one shows a cross-section of nodes in different dimensions and
shows how the connections look different for different protocols.
![Node Cross Section](./images/adaptive_node_cross_section2.png)

BIN
waku/standards/.DS_Store vendored Normal file
View File

Binary file not shown.

BIN
waku/standards/application/.DS_Store vendored Normal file
View File

Binary file not shown.

116
waku/standards/application/20/toy-eth-pm.md
View File

@@ -17,12 +17,16 @@ This specification explains the Toy Ethereum Private Message protocol
which enables a peer to send an encrypted message to another peer
using the Waku v2 network, and the peer's Ethereum address.
The main purpose of this specification is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
The main purpose of this specification
is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
using Ethereum accounts for identity.
This protocol caters for Web3 wallets restrictions, allowing it to be implemented only using standard Web3 API.
In the current state, the protocol has privacy and features [limitations](#limitations), has not been audited
and hence is not fit for production usage.
We hope this can be an inspiration for developers wishing to build on top of Waku v2.
This protocol caters for Web3 wallets restrictions,
allowing it to be implemented only using standard Web3 API.
In the current state,
the protocol has privacy and features [limitations](#limitations),
has not been audited and hence is not fit for production usage.
We hope this can be an inspiration for developers
wishing to build on top of Waku v2.
## Goal
@@ -42,14 +46,16 @@ Here are the variables used in the protocol and their definition:
The proposed protocol MUST adhere to the following design requirements:
1. Alice knows Bob's Ethereum address,
1. Alice knows Bob's Ethereum address,
2. Bob is willing to participate to Eth-PM, and publishes `B'`,
3. Bob's ownership of `B'` MUST be verifiable,
4. Alice wants to send message `M` to Bob,
5. Bob SHOULD be able to get `M` using [10/WAKU2 spec](../../core/10/waku2.md),
6. Participants only have access to their Ethereum Wallet via the Web3 API,
7. Carole MUST NOT be able to read `M`'s content even if she is storing it or relaying it,
8. [Waku Message Version 1](../26/payload.md) Asymmetric Encryption is used for encryption purposes.
7. Carole MUST NOT be able to read `M`'s content
even if she is storing it or relaying it,
8. [Waku Message Version 1](../26/payload.md) Asymmetric Encryption
is used for encryption purposes.
## Limitations
@@ -58,16 +64,20 @@ meaning that there is no programmatic way for Bob to reply to Alice
or verify her identity.
Private messages are sent on the same content topic for all users.
As the recipient data is encrypted, all participants must decrypt all messages which can lead to scalability issues.
As the recipient data is encrypted,
all participants must decrypt all messages which can lead to scalability issues.
This protocol does not guarantee Perfect Forward Secrecy nor Future Secrecy:
If Bob's private key is compromised, past and future messages could be decrypted.
A solution combining regular [X3DH](https://www.signal.org/docs/specifications/x3dh/)
bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) encryption would remove these limitations;
See the [Status secure transport spec](https://specs.status.im/spec/5) for an example of a protocol that achieves this in a peer-to-peer setting.
bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
encryption would remove these limitations;
See the [Status secure transport spec](https://specs.status.im/spec/5)
for an example of a protocol that achieves this in a peer-to-peer setting.
Bob MUST decide to participate in the protocol before Alice can send him a message.
This is discussed in more in details in [Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
This is discussed in more in details in
[Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
## The protocol
@@ -82,7 +92,8 @@ Then Bob can compute the corresponding SECP-256k1 Public Key, `B'`.
For Alice to encrypt messages for Bob,
Bob SHOULD broadcast his Encryption Public Key `B'`.
To prove that the Encryption Public Key `B'` is indeed owned by the owner of Bob's Ethereum Account `B`,
To prove that the Encryption Public Key `B'`
is indeed owned by the owner of Bob's Ethereum Account `B`,
Bob MUST sign `B'` using `B`.
### Sign Encryption Public Key
@@ -91,13 +102,15 @@ To prove ownership of the Encryption Public Key,
Bob must sign it using [EIP-712](https://eips.ethereum.org/EIPS/eip-712) v3,
meaning calling `eth_signTypedData_v3` on his Wallet's API.
Note: While v4 also exists,
it is not available on all wallets and the features brought by v4 is not needed for the current use case.
Note: While v4 also exists, it is not available on all wallets and
the features brought by v4 is not needed for the current use case.
The `TypedData` to be passed to `eth_signTypedData_v3` MUST be as follows, where:
- `encryptionPublicKey` is Bob's Encryption Public Key, `B'`, in hex format, **without** `0x` prefix.
- `bobAddress` is Bob's Ethereum address, corresponding to `B`, in hex format, **with** `0x` prefix.
- `encryptionPublicKey` is Bob's Encryption Public Key, `B'`,
in hex format, **without** `0x` prefix.
- `bobAddress` is Bob's Ethereum address, corresponding to `B`,
in hex format, **with** `0x` prefix.
```js
const typedData = {
@@ -143,8 +156,9 @@ message PublicKeyMessage {
}
```
This MUST be wrapped in a Waku Message version 0, with the Public Key Broadcast content topic.
Finally, Bob SHOULD publish the message on Waku v2.
This MUST be wrapped in a Waku Message version 0,
with the Public Key Broadcast content topic.
Finally, Bob SHOULD publish the message on Waku v2.
## Consideration for a non-interactive/uncoordinated protocol
@@ -157,63 +171,86 @@ and Alice MUST receive it before she can send him a message.
Moreover, nim-waku, the reference implementation of [13/WAKU2-STORE](../../core/13/store.md),
stores messages for a maximum period of 30 days.
This means that Bob would need to broadcast his public key at least every 30 days to be reachable.
This means that Bob would need to broadcast his public key
at least every 30 days to be reachable.
Below we are reviewing possible solutions to mitigate this "sign up" step.
### Retrieve the public key from the blockchain
If Bob has signed at least one transaction with his account then his Public Key can be extracted from the transaction's ECDSA signature.
The challenge with this method is that standard Web3 Wallet API does not allow Alice to specifically retrieve all/any transaction sent by Bob.
If Bob has signed at least one transaction with his account
then his Public Key can be extracted from the transaction's ECDSA signature.
The challenge with this method is that standard Web3 Wallet API
does not allow Alice to specifically retrieve all/any transaction sent by Bob.
Alice would instead need to use the `eth.getBlock` API to retrieve Ethereum blocks one by one.
For each block, she would need to check the `from` value of each transaction until she finds a transaction sent by Bob.
Alice would instead need to use the `eth.getBlock` API
to retrieve Ethereum blocks one by one.
For each block, she would need to check the `from` value of each transaction
until she finds a transaction sent by Bob.
This process is resource intensive and can be slow when using services such as Infura due to rate limits in place,
This process is resource intensive and
can be slow when using services such as Infura due to rate limits in place,
which makes it inappropriate for a browser or mobile phone environment.
An alternative would be to either run a backend that can connect directly to an Ethereum node,
An alternative would be to either run a backend
that can connect directly to an Ethereum node,
use a centralized blockchain explorer
or use a decentralized indexing service such as [The Graph](https://thegraph.com/).
Note that these would resolve a UX issue only if a sender wants to proceed with _air drops_.
Note that these would resolve a UX issue
only if a sender wants to proceed with _air drops_.
Indeed, if Bob does not publish his Public Key in the first place
then it can be an indication that he simply does not participate in this protocol and hence will not receive messages.
then it can be an indication that he simply does not participate in this protocol
and hence will not receive messages.
However, these solutions would be helpful if the sender wants to proceed with an _air drop_ of messages:
Send messages over Waku for users to retrieve later, once they decide to participate in this protocol.
However, these solutions would be helpful
if the sender wants to proceed with an _air drop_ of messages:
Send messages over Waku for users to retrieve later,
once they decide to participate in this protocol.
Bob may not want to participate first but may decide to participate at a later stage
and would like to access previous messages.
This could make sense in an NFT offer scenario:
Users send offers to any NFT owner,
NFT owner may decide at some point to participate in the protocol and retrieve previous offers.
NFT owner may decide at some point to participate in the protocol and
retrieve previous offers.
### Publishing the public in long term storage
Another improvement would be for Bob not having to re-publish his public key every 30 days or less.
Similarly to above, if Bob stops publishing his public key then it may be an indication that he does not participate in the protocol anymore.
Another improvement would be for Bob not having to re-publish his public key
every 30 days or less.
Similarly to above,
if Bob stops publishing his public key
then it may be an indication that he does not participate in the protocol anymore.
In any case, the protocol could be modified to store the Public Key in a more permanent storage, such as a dedicated smart contract on the blockchain.
In any case,
the protocol could be modified to store the Public Key in a more permanent storage,
such as a dedicated smart contract on the blockchain.
## Send Private Message
Alice MAY monitor the Waku v2 to collect Ethereum Address and Encryption Public Key tuples.
Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives are valid as per EIP-712.
Alice MAY monitor the Waku v2 to collect Ethereum Address and
Encryption Public Key tuples.
Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives
are valid as per EIP-712.
She SHOULD drop any message without a signature or with an invalid signature.
Using Bob's Encryption Public Key, retrieved via [10/WAKU2 spec](../../core/10/waku2.md), Alice MAY now send an encrypted message to Bob.
Using Bob's Encryption Public Key,
retrieved via [10/WAKU2 spec](../../core/10/waku2.md),
Alice MAY now send an encrypted message to Bob.
If she wishes to do so, Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
If she wishes to do so,
Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](../26/payload.md/#asymmetric).
Alice SHOULD now publish this message on the Private Message content topic.
# Copyright
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [10/WAKU2 spec](../../core/10/waku2.md)
- [Waku Message Version 1](../26/payload.md)
- [X3DH](https://www.signal.org/docs/specifications/x3dh/)
@@ -222,4 +259,3 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
- [EIP-712](https://eips.ethereum.org/EIPS/eip-712)
- [13/WAKU2-STORE](../../core/13/store.md)
- [The Graph](https://thegraph.com/)

91
waku/standards/application/21/fault-tolerant-store.md
View File

@@ -7,23 +7,45 @@ editor: Sanaz Taheri <sanaz@status.im>
contributors:
---
The reliability of [13/WAKU2-STORE](../../core/13/store.md) protocol heavily relies on the fact that full nodes i.e., those who persist messages have high availability and uptime and do not miss any messages.
If a node goes offline, then it will risk missing all the messages transmitted in the network during that time.
In this specification, we provide a method that makes the store protocol resilient in presence of faulty nodes.
Relying on this method, nodes that have been offline for a time window will be able to fix the gap in their message history when getting back online.
Moreover, nodes with lower availability and uptime can leverage this method to reliably provide the store protocol services as a full node.
The reliability of [13/WAKU2-STORE](../../core/13/store.md)
protocol heavily relies on the fact that full nodes i.e.,
those who persist messages have high availability and
uptime and do not miss any messages.
If a node goes offline,
then it will risk missing all the messages transmitted
in the network during that time.
In this specification,
we provide a method that makes the store protocol resilient
in presence of faulty nodes.
Relying on this method,
nodes that have been offline for a time window will be able to fix the gap
in their message history when getting back online.
Moreover, nodes with lower availability and
uptime can leverage this method to reliably provide the store protocol services
as a full node.
## Method description
As the first step towards making the [13/WAKU2-STORE](../../core/13/store.md) protocol fault-tolerant, we introduce a new type of time-based query through which nodes fetch message history from each other based on their desired time window.
This method operates based on the assumption that the querying node knows some other nodes in the store protocol which have been online for that targeted time window.
## Method description
As the first step
towards making the [13/WAKU2-STORE](../../core/13/store.md) protocol fault-tolerant,
we introduce a new type of time-based query through which nodes fetch message history
from each other based on their desired time window.
This method operates based on the assumption that the querying node
knows some other nodes in the store protocol
which have been online for that targeted time window.
## Security Consideration
The main security consideration to take into account while using this method is that a querying node has to reveal its offline time to the queried node.
This will gradually result in the extraction of the node's activity pattern which can lead to inference attacks.
The main security consideration to take into account
while using this method is that a querying node
has to reveal its offline time to the queried node.
This will gradually result in the extraction of the node's activity pattern
which can lead to inference attacks.
## Wire Specification
We extend the [HistoryQuery](../../core/13/store.md/#payloads) protobuf message with two fields of `start_time` and `end_time` to signify the time range to be queried.
We extend the [HistoryQuery](../../core/13/store.md/#payloads) protobuf message
with two fields of `start_time` and `end_time` to signify the time range to be queried.
### Payloads
@@ -44,24 +66,43 @@ message HistoryQuery {
### HistoryQuery
RPC call to query historical messages.
- `start_time`: this field MAY be filled out to signify the starting point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding [`HistoryResponse`](../../core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../core/14/message.md/#Payloads) is larger than or equal to the `start_time`.
- `end_time` this field MAY be filled out to signify the ending point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding [`HistoryResponse`](../../core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../core/14/message.md/#Payloads) is less than or equal to the `end_time`.
A time-based query is considered valid if its `end_time` is larger than or equal to the `start_time`.
Queries that do not adhere to this condition will not get through e.g. an open-end time query in which the `start_time` is given but no `end_time` is supplied is not valid.
If both `start_time` and `end_time` are omitted then no time-window filter takes place.
- `start_time`:
this field MAY be filled out to signify the starting point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding
[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
MUST contain historical waku messages whose
[`timestamp`](../../core/14/message.md/#Payloads)
is larger than or equal to the `start_time`.
- `end_time`:
this field MAY be filled out to signify the ending point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding
[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
MUST contain historical waku messages whose
[`timestamp`](../../core/14/message.md/#Payloads) is less than or equal to the `end_time`.
A time-based query is considered valid if
its `end_time` is larger than or equal to the `start_time`.
Queries that do not adhere to this condition will not get through e.g.
an open-end time query in which the `start_time` is given but
no `end_time` is supplied is not valid.
If both `start_time` and
`end_time` are omitted then no time-window filter takes place.
In order to account for nodes asynchrony, and
assuming that nodes may be out of sync for at most 20 seconds
(i.e., 20000000000 nanoseconds),
the querying nodes SHOULD add an offset of 20 seconds to their offline time window.
That is if the original window is [`l`,`r`]
then the history query SHOULD be made for `[start_time: l - 20s, end_time: r + 20s]`.
In order to account for nodes asynchrony, and assuming that nodes may be out of sync for at most 20 seconds (i.e., 20000000000 nanoseconds), the querying nodes SHOULD add an offset of 20 seconds to their offline time window.
That is if the original window is [`l`,`r`] then the history query SHOULD be made for `[start_time: l - 20s, end_time: r + 20s]`.
Note that `HistoryQuery` preserves `AND` operation among the queried attributes.
As such, The `messages` field of the corresponding [`HistoryResponse`](../../core/13/store.md/#HistoryResponse) MUST contain historical waku messages that satisfy the indicated `pubsubtopic` AND `contentFilters` AND the time range [`start_time`, `end_time`].
Note that `HistoryQuery` preserves `AND` operation among the queried attributes.
As such, the `messages` field of the corresponding
[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
MUST contain historical waku messages that satisfy the indicated `pubsubtopic` AND
`contentFilters` AND the time range [`start_time`, `end_time`].
## Copyright

126
waku/standards/application/26/payload.md
View File

@@ -7,23 +7,40 @@ editor: Oskar Thoren <oskarth@titanproxy.com>
contributors:
---
This specification describes how Waku provides confidentiality, authenticity, and integrity, as well as some form of unlinkability.
Specifically, it describes how encryption, decryption and signing works in [6/WAKU1](../../legacy/6/waku1.md) and in [10/WAKU2 spec](../../core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1).
This specification describes how Waku provides confidentiality, authenticity, and
integrity, as well as some form of unlinkability.
Specifically, it describes how encryption, decryption and
signing works in [6/WAKU1](../../legacy/6/waku1.md) and
in [10/WAKU2 spec](../../core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1).
This specification effectively replaces [7/WAKU-DATA](../../legacy/7/data.md) as well as [6/WAKU1 Payload encryption](../../legacy/6/waku1.md/#payload-encryption) but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
This specification effectively replaces [7/WAKU-DATA](../../legacy/7/data.md)
as well as [6/WAKU1 Payload encryption](../../legacy/6/waku1.md/#payload-encryption)
but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
Large sections of the specification originate from [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) with some modifications.
Large sections of the specification originate from
[EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from
[RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
with some modifications.
## Design requirements
- *Confidentiality*: The adversary should not be able to learn what data is being sent from one Waku node to one or several other Waku nodes.
- *Authenticity*: The adversary should not be able to cause Waku endpoint to accept data from any third party as though it came from the other endpoint.
- *Integrity*: The adversary should not be able to cause a Waku endpoint to accept data that has been tampered with.
- *Confidentiality*:
The adversary should not be able to learn what data is being sent from one Waku node
to one or several other Waku nodes.
- *Authenticity*:
The adversary should not be able to cause Waku endpoint
to accept data from any third party as though it came from the other endpoint.
- *Integrity*:
The adversary should not be able to cause a Waku endpoint to
accept data that has been tampered with.
Notable, *forward secrecy* is not provided for at this layer.
If this property is desired, a more fully featured secure communication protocol can be used on top, such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
If this property is desired,
a more fully featured secure communication protocol can be used on top,
such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
It also provides some form of *unlinkability* since:
- only participants who are able to decrypt a message can see its signature
- payload are padded to a fixed length
@@ -35,6 +52,7 @@ It also provides some form of *unlinkability* since:
- KECCAK-256
ECIES is using the following cryptosystem:
- Curve: secp256k1
- KDF: NIST SP 800-56 Concatenation Key Derivation Function, with SHA-256 option
- MAC: HMAC with SHA-256
@@ -42,20 +60,27 @@ ECIES is using the following cryptosystem:
## Specification
For [6/WAKU1](../../legacy/6/waku1.md), the `data` field is used in the `waku envelope`, and the field MAY contain the encrypted payload.
For [6/WAKU1](../../legacy/6/waku1.md),
the `data` field is used in the `waku envelope`,
and the field MAY contain the encrypted payload.
For [10/WAKU2 spec](../../core/10/waku2.md), the `payload` field is used in `WakuMessage` and MAY contain the encrypted payload.
For [10/WAKU2 spec](../../core/10/waku2.md),
the `payload` field is used in `WakuMessage` and
MAY contain the encrypted payload.
The fields that are concatenated and encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
- flags
- payload-length
- payload
- padding
- signature
The fields that are concatenated and
encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
- flags
- payload-length
- payload
- padding
- signature
### ABNF
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
we have the following format:
```abnf
; 1 byte; first two bits contain the size of payload-length field,
@@ -83,29 +108,43 @@ payload = data
### Signature
Those unable to decrypt the payload/data are also unable to access the signature.
The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity.
The signature is serialized as the concatenation of the `r`, `s` and `v` parameters of the SECP-256k1 ECDSA signature, in that order.
The signature, if provided,
is the ECDSA signature of the Keccak-256 hash of the unencrypted data
using the secret key of the originator identity.
The signature is serialized as the concatenation of the `r`, `s` and `v` parameters
of the SECP-256k1 ECDSA signature, in that order.
`r` and `s` MUST be big-endian encoded, fixed-width 256-bit unsigned.
`v` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
`v` MUST be an 8-bit big-endian encoded,
non-normalized and should be either 27 or 28.
See [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf) for more information on signature generation, parameters and public key recovery.
See [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
for more information on signature generation, parameters and public key recovery.
### Encryption
#### Symmetric
Symmetric encryption uses AES-256-GCM for [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption).
The output of encryption is of the form (`ciphertext`, `tag`, `iv`) where `ciphertext` is the encrypted message, `tag` is a 16 byte message authentication tag and `iv` is a 12 byte initialization vector (nonce).
The message authentication `tag` and initialization vector `iv` field MUST be appended to the resulting `ciphertext`, in that order.
Note that previous specifications and some implementations might refer to `iv` as `nonce` or `salt`.
Symmetric encryption uses AES-256-GCM for
[authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption).
The output of encryption is of the form (`ciphertext`, `tag`, `iv`)
where `ciphertext` is the encrypted message,
`tag` is a 16 byte message authentication tag and
`iv` is a 12 byte initialization vector (nonce).
The message authentication `tag` and
initialization vector `iv` field MUST be appended to the resulting `ciphertext`,
in that order.
Note that previous specifications and
some implementations might refer to `iv` as `nonce` or `salt`.
#### Asymmetric
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme (ECIES) with SECP-256k1 public key.
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme
(ECIES) with SECP-256k1 public key.
#### ECIES
This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) spec with minor modifications.
This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
spec with minor modifications.
The cryptosystem used is:
@@ -116,24 +155,39 @@ The cryptosystem used is:
Special notation used: `X || Y` denotes concatenation of `X` and `Y`.
Alice wants to send an encrypted message that can be decrypted by Bob's static private key `kB`. Alice knows about Bobs static public key `KB`.
Alice wants to send an encrypted message that can be decrypted by
Bob's static private key `kB`.
Alice knows about Bobs static public key `KB`.
To encrypt the message `m`, Alice generates a random number `r` and corresponding elliptic curve public key `R = r * G` and computes the shared secret `S = Px` where `(Px, Py) = r * KB`.
She derives key material for encryption and authentication as `kE || kM = KDF(S, 32)` as well as a random initialization vector `iv`.
Alice sends the encrypted message `R || iv || c || d` where `c = AES(kE, iv , m)` and `d = MAC(sha256(kM), iv || c)` to Bob.
To encrypt the message `m`, Alice generates a random number `r` and
corresponding elliptic curve public key `R = r * G` and
computes the shared secret `S = Px` where `(Px, Py) = r * KB`.
She derives key material for encryption and
authentication as `kE || kM = KDF(S, 32)`
as well as a random initialization vector `iv`.
Alice sends the encrypted message `R || iv || c || d` where `c = AES(kE, iv , m)`
and `d = MAC(sha256(kM), iv || c)` to Bob.
For Bob to decrypt the message `R || iv || c || d`, he derives the shared secret `S = Px` where `(Px, Py) = kB * R` as well as the encryption and authentication keys `kE || kM = KDF(S, 32)`.
Bob verifies the authenticity of the message by checking whether `d == MAC(sha256(kM), iv || c)` then obtains the plaintext as `m = AES(kE, iv || c)`.
For Bob to decrypt the message `R || iv || c || d`,
he derives the shared secret `S = Px` where `(Px, Py) = kB * R`
as well as the encryption and authentication keys `kE || kM = KDF(S, 32)`.
Bob verifies the authenticity of the message
by checking whether `d == MAC(sha256(kM), iv || c)`
then obtains the plaintext as `m = AES(kE, iv || c)`.
### Padding
The padding field is used to align data size, since data size alone might reveal important metainformation.
The padding field is used to align data size,
since data size alone might reveal important metainformation.
Padding can be arbitrary size.
However, it is recommended that the size of Data Field (excluding the IV and tag) before encryption (i.e. plain text) SHOULD be a multiple of 256 bytes.
However, it is recommended that the size of Data Field
(excluding the IV and tag) before encryption (i.e. plain text)
SHOULD be a multiple of 256 bytes.
### Decoding a message
In order to decode a message, a node SHOULD try to apply both symmetric and asymmetric decryption operations.
In order to decode a message, a node SHOULD try to apply both symmetric and
asymmetric decryption operations.
This is because the type of encryption is not included in the message.
## Copyright

167
waku/standards/application/53/x3dh.md
View File

@@ -16,75 +16,102 @@ contributors:
## Abstract
This document describes a method that can be used to provide a secure channel between two peers, and thus provide confidentiality, integrity, authenticity and forward secrecy.
This document describes a method that can be used to provide a secure channel
between two peers, and thus provide confidentiality, integrity,
authenticity and forward secrecy.
It is transport-agnostic and works over asynchronous networks.
It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications, with some adaptations to operate in a decentralized environment.
It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/)
and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications,
with some adaptations to operate in a decentralized environment.
## Motivation
Nodes on a network may want to communicate with each other in a secure manner, without other nodes network being able to read their messages.
Nodes on a network may want to communicate with each other in a secure manner,
without other nodes network being able to read their messages.
## Specification
### Definitions
- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols which provide assurances that session keys will not be compromised even if the private keys of the participants are compromised.
Specifically, past messages cannot be decrypted by a third-party who manages to get a hold of a private key.
- **Secret channel** describes a communication channel where a Double Ratchet algorithm is in use.
- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols
which provide assurances that session keys will not be compromised
even if the private keys of the participants are compromised.
Specifically, past messages cannot be decrypted by a third-party
who manages to get a hold of a private key.
- **Secret channel** describes a communication channel
where a Double Ratchet algorithm is in use.
### Design Requirements
- **Confidentiality**: The adversary should not be able to learn what data is being exchanged between two Status clients.
- **Authenticity**: The adversary should not be able to cause either endpoint to accept data from any third party as though it came from the other endpoint.
- **Forward Secrecy**: The adversary should not be able to learn what data was exchanged between two clients if, at some later time, the adversary compromises one or both of the endpoints.
- **Integrity**: The adversary should not be able to cause either endpoint to accept data that has been tampered with.
- **Confidentiality**:
The adversary should not be able to learn what data is being exchanged
between two Status clients.
- **Authenticity**:
The adversary should not be able to cause either endpoint
to accept data from any third party as though it came from the other endpoint.
- **Forward Secrecy**:
The adversary should not be able to learn what data was exchanged
between two clients if, at some later time,
the adversary compromises one or both of the endpoints.
- **Integrity**:
The adversary should not be able to cause either endpoint
to accept data that has been tampered with.
All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
### Conventions
Types used in this specification are defined using the [Protobuf](https://developers.google.com/protocol-buffers/) wire format.
## Specification
Types used in this specification are defined using the
[Protobuf](https://developers.google.com/protocol-buffers/) wire format.
### End-to-End Encryption
End-to-end encryption (E2EE) takes place between two clients.
The main cryptographic protocol is a Double Ratchet protocol, which is derived from the [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html), using a different ratchet.
[The Waku v2 protocol](../../core/10/waku2.md) subsequently encrypts the message payload, using symmetric key encryption.
Furthermore, the concept of prekeys (through the use of [X3DH](https://signal.org/docs/specifications/x3dh/)) is used to allow the protocol to operate in an asynchronous environment.
It is not necessary for two parties to be online at the same time to initiate an encrypted conversation.
End-to-end encryption (E2EE) takes place between two clients.
The main cryptographic protocol is a Double Ratchet protocol,
which is derived from the
[Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html),
using a different ratchet.
[The Waku v2 protocol](../../core/10/waku2.md)
subsequently encrypts the message payload, using symmetric key encryption.
Furthermore, the concept of prekeys
(through the use of [X3DH](https://signal.org/docs/specifications/x3dh/))
is used to allow the protocol to operate in an asynchronous environment.
It is not necessary for two parties to be online at the same time
to initiate an encrypted conversation.
### Cryptographic Protocols
This protocol uses the following cryptographic primitives:
- X3DH
- Elliptic curve Diffie-Hellman key exchange (secp256k1)
- KECCAK-256
- ECDSA
- ECIES
- Elliptic curve Diffie-Hellman key exchange (secp256k1)
- KECCAK-256
- ECDSA
- ECIES
- Double Ratchet
- HMAC-SHA-256 as MAC
- Elliptic curve Diffie-Hellman key exchange (Curve25519)
- AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
- HMAC-SHA-256 as MAC
- Elliptic curve Diffie-Hellman key exchange (Curve25519)
- AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
The node achieves key derivation using [HKDF](https://www.rfc-editor.org/rfc/rfc5869).
### Pre-keys
Every client SHOULD initially generate some key material which is stored locally:
- Identity keypair based on secp256k1 - `IK`
- A signed prekey based on secp256k1 - `SPK`
- A prekey signature - `Sig(IK, Encode(SPK))`
More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles).
Prekey bundles MAY be extracted from any peer's messages, or found via searching for their specific topic, `{IK}-contact-code`.
Prekey bundles MAY be extracted from any peer's messages,
or found via searching for their specific topic, `{IK}-contact-code`.
The following methods can be used to retrieve prekey bundles from a peer's messages:
- contact codes;
- public and one-to-one chats;
- QR codes;
@@ -94,28 +121,41 @@ The following methods can be used to retrieve prekey bundles from a peer's messa
Waku SHOULD be used for retrieving prekey bundles.
Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys, the bundle MAY be rotated every 24 hours, and distributed via Waku.
Since bundles stored in QR codes or
ENS records cannot be updated to delete already used keys,
the bundle MAY be rotated every 24 hours, and distributed via Waku.
### Flow
The key exchange can be summarized as follows:
1. Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles, and derive a shared secret.
1. Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles,
and derive a shared secret.
2. Double Ratchet: The two parties use the shared secret to derive a new encryption key for each message they send.
2. Double Ratchet:
The two parties use the shared secret to derive a new encryption key
for each message they send.
3. Chain key update: The two parties update their chain keys. The chain key is used to derive new encryption keys for future messages.
3. Chain key update: The two parties update their chain keys.
The chain key is used to derive new encryption keys for future messages.
4. Message key derivation: The two parties derive a new message key from their chain key, and use it to encrypt a message.
4. Message key derivation:
The two parties derive a new message key from their chain key, and
use it to encrypt a message.
#### 1. Initial key exchange flow (X3DH)
[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message) describes the initial key exchange flow, with some additional context:
- The peers' identity keys `IK_A` and `IK_B` correspond to their public keys;
- Since it is not possible to guarantee that a prekey will be used only once in a decentralized world, the one-time prekey `OPK_B` is not used in this scenario;
- Nodes SHOULD not send Bundles to a centralized server, but instead provide them in a decentralized way as described in the [Pre-keys section](#pre-keys).
[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message)
describes the initial key exchange flow, with some additional context:
Alice retrieves Bob's prekey bundle, however it is not specific to Alice. It contains:
- The peers' identity keys `IK_A` and `IK_B` correspond to their public keys;
- Since it is not possible to guarantee that a prekey will be used only once
in a decentralized world, the one-time prekey `OPK_B` is not used in this scenario;
- Nodes SHOULD not send Bundles to a centralized server,
but instead provide them in a decentralized way as described in the [Pre-keys section](#pre-keys).
Alice retrieves Bob's prekey bundle, however it is not specific to Alice.
It contains:
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12))
@@ -144,17 +184,23 @@ message SignedPreKey {
}
```
The `signature` is generated by sorting `installation-id` in lexicographical order, and concatenating the `signed-pre-key` and `version`:
The `signature` is generated by sorting `installation-id` in lexicographical order,
and concatenating the `signed-pre-key` and `version`:
`installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2`
#### 2. Double Ratchet
Having established the initial shared secret `SK` through X3DH, it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
Having established the initial shared secret `SK` through X3DH,
it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
Refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/) for more details.
Refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/)
for more details.
The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage` ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65)) containing a map of `DirectMessageProtocol` indexed by `installation-id` ([reference wire format](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage`
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65))
containing a map of `DirectMessageProtocol` indexed by `installation-id`
([reference wire format](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
``` protobuf
message ProtocolMessage {
@@ -180,21 +226,26 @@ message EncryptedMessageProtocol {
bytes payload = 3;
}
```
Where:
- `X3DH_header`: the `X3DHHeader` field in `DirectMessageProtocol` contains:
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
``` protobuf
```protobuf
message X3DHHeader {
// Alice's ephemeral key `EK_A`
bytes key = 1;
// Bob's bundle signed prekey
bytes id = 4;
}
```
```
- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)). Used when Bob's public bundle is available:
``` protobuf
- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)).
Used when Bob's public bundle is available:
``` protobuf
message DRHeader {
// Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet))
bytes key = 1;
@@ -205,38 +256,46 @@ Where:
// Bob's bundle ID
bytes id = 4;
}
```
```
- `DH_header`: Diffie-Hellman header (used when Bob's bundle is not available):
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
``` protobuf
``` protobuf
message DHHeader {
// Alice's compressed ephemeral public key.
bytes key = 1;
}
```
```
#### 3. Chain key update
The chain key MUST be updated according to the `DR_Header` received in the `EncryptedMessageProtocol` message, described in [2.Double Ratchet](#2-double-ratchet).
The chain key MUST be updated according to the `DR_Header`
received in the `EncryptedMessageProtocol` message,
described in [2.Double Ratchet](#2-double-ratchet).
#### 4. Message key derivation
The message key MUST be derived from a single ratchet step in the symmetric-key ratchet as described in [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
The message key MUST be derived from a single ratchet step in the symmetric-key ratchet
as described in [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
The message key MUST be used to encrypt the next message to be sent.
## Security Considerations
1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations)
and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
2. Inherits the security considerations of the [Waku v2 protocol](../../core/10/waku2.md).
3. The protocol is designed to be used in a decentralized manner, however, it is possible to use a centralized server to serve prekey bundles. In this case, the server is trusted.
3. The protocol is designed to be used in a decentralized manner, however,
it is possible to use a centralized server to serve prekey bundles.
In this case, the server is trusted.
## Privacy Considerations
1. This protocol does not provide message unlinkability. It is possible to link messages signed by the same keypair.
1. This protocol does not provide message unlinkability.
It is possible to link messages signed by the same keypair.
## Copyright
@@ -254,5 +313,3 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
- [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles)
- [reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12)
- [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
-
-

108
waku/standards/application/54/x3dh-sessions.md
View File

@@ -17,29 +17,37 @@ contributors:
## Abstract
This document specifies how to manage sessions based on an X3DH key exchange.
This includes how to establish new sessions, how to re-establish them, how to maintain them, and how to close them.
This includes how to establish new sessions,
how to re-establish them, how to maintain them, and how to close them.
[53/WAKU2-X3DH](../53/x3dh.md) specifies the Waku `X3DH` protocol for end-to-end encryption.
[53/WAKU2-X3DH](../53/x3dh.md) specifies the Waku `X3DH` protocol
for end-to-end encryption.
Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
## Session Establishment
A node identifies a peer by their `installation-id` which MAY be interpreted as a device identifier.
A node identifies a peer by their `installation-id`
which MAY be interpreted as a device identifier.
### Discovery of pre-key bundles
The node's pre-key bundle MUST be broadcast on a content topic derived from the node's public key, so that the first message may be PFS-encrypted.
Each peer MUST publish their pre-key bundle periodically to this topic, otherwise they risk not being able to perform key-exchanges with other peers.
Each peer MAY publish to this topic when their metadata changes, so that the other peer can update their local record.
The node's pre-key bundle MUST be broadcast on a content topic
derived from the node's public key, so that the first message may be PFS-encrypted.
Each peer MUST publish their pre-key bundle periodically to this topic,
otherwise they risk not being able to perform key-exchanges with other peers.
Each peer MAY publish to this topic when their metadata changes,
so that the other peer can update their local record.
If peer A wants to send a message to peer B, it MUST derive the topic from peer B's public key, which has been shared out of band.
Partitioned topics have been used to balance privacy and efficiency of broadcasting pre-key bundles.
If peer A wants to send a message to peer B,
it MUST derive the topic from peer B's public key, which has been shared out of band.
Partitioned topics have been used to balance privacy and
efficiency of broadcasting pre-key bundles.
The number of partitions that MUST be used is 5000.
The topic MUST be derived as follows:
```
```js
var partitionsNum *big.Int = big.NewInt(5000)
var partition *big.Int = big.NewInt(0).Mod(peerBPublicKey, partitionsNum)
@@ -59,15 +67,18 @@ for i = 0; i < topicLen; i++ {
```
### Initialization
A node initializes a new session once a successful X3DH exchange has taken place.
A node initializes a new session once a successful X3DH exchange has taken place.
Subsequent messages will use the established session until re-keying is necessary.
### Negotiated topic to be used for the session
After the peers have performed the initial key exchange, they MUST derive a topic from their shared secret to send messages on.
To obtain this value, take the first four bytes of the keccak256 hash of the shared secret encoded in hexadecimal format.
After the peers have performed the initial key exchange,
they MUST derive a topic from their shared secret to send messages on.
To obtain this value, take the first four bytes of the keccak256 hash
of the shared secret encoded in hexadecimal format.
```
```js
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
ecies.ImportECDSAPublic(theirPublicKey),
16,
@@ -90,59 +101,85 @@ for i = 0; i < topicLen; i++ {
}
```
To summarize, following is the process for peer B to establish a session with peer A:
1. Listen to peer B's Contact Code Topic to retrieve their bundle information, including a list of active devices
To summarize,
following is the process for peer B to establish a session with peer A:
1. Listen to peer B's Contact Code Topic to retrieve their bundle information,
including a list of active devices
2. Peer A sends their pre-key bundle on peer B's partitioned topic
3. Peer A and peer B perform the key-exchange using the shared pre-key bundles
3. The negotiated topic is derived from the shared secret
4. Peers A & B exchange messages on the negotiated topic
4. The negotiated topic is derived from the shared secret
5. Peers A & B exchange messages on the negotiated topic
### Concurrent sessions
If a node creates two sessions concurrently between two peers, the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
If a node creates two sessions concurrently between two peers,
the one with the symmetric key first in byte order SHOULD be used,
this marks that the other has expired.
### Re-keying
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
On receiving a bundle from a given peer with a higher version,
the old bundle SHOULD be marked as expired and
a new session SHOULD be established on the next message sent.
### Multi-device support
Multi-device support is quite challenging as there is not a central place where information on which and how many devices (identified by their respective `installation-id`) a peer has, is stored.
Multi-device support is quite challenging
as there is not a central place where information on which and how many devices
(identified by their respective `installation-id`) a peer has, is stored.
Furthermore, account recovery always needs to be taken into consideration, where a user wipes clean the whole device and the node loses all the information about any previous sessions.
Taking these considerations into account, the way the network propagates multi-device information using X3DH bundles, which will contain information about paired devices as well as information about the sending device.
This means that every time a new device is paired, the bundle needs to be updated and propagated with the new information, the user has the responsibility to make sure the pairing is successful.
Furthermore, account recovery always needs to be taken into consideration,
where a user wipes clean the whole device and
the node loses all the information about any previous sessions.
Taking these considerations into account,
the way the network propagates multi-device information using X3DH bundles,
which will contain information about paired devices
as well as information about the sending device.
This means that every time a new device is paired,
the bundle needs to be updated and propagated with the new information,
the user has the responsibility to make sure the pairing is successful.
The method is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/).
### Pairing
A new `installation-id` MUST be generated on a per-device basis.
The device should be paired as soon as possible if other devices are present.
A new `installation-id` MUST be generated on a per-device basis.
The device should be paired as soon as possible if other devices are present.
If a bundle is received, which has the same `IK` as the keypair present on the device, the devices MAY be paired.
Once a user enables a new device, a new bundle MUST be generated which includes pairing information.
If a bundle is received, which has the same `IK` as the keypair present on the device,
the devices MAY be paired.
Once a user enables a new device,
a new bundle MUST be generated which includes pairing information.
The bundle MUST be propagated to contacts through the usual channels.
Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
Removal of paired devices is a manual step that needs to be applied on each device,
and consist simply in disabling the device,
at which point pairing information will not be propagated anymore.
### Sending messages to a paired group
When sending a message, the peer SHOULD send a message to other `installation-id` that they have seen.
The node caps the number of devices to `n`, ordered by last activity.
When sending a message,
the peer SHOULD send a message to other `installation-id` that they have seen.
The node caps the number of devices to `n`, ordered by last activity.
The node sends messages using pairwise encryption, including their own devices.
Where `n` is the maximum number of devices that can be paired.
### Account recovery
Account recovery is the same as adding a new device, and it MUST be handled the same way.
Account recovery is the same as adding a new device,
and it MUST be handled the same way.
### Partitioned devices
In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own `installation-id`.
In this case an empty message containing bundle information MUST be sent back, which will notify the receiving end not to include the device in any further communication.
In some cases
(i.e. account recovery when no other pairing device is available, device not paired),
it is possible that a device will receive a message
that is not targeted to its own `installation-id`.
In this case an empty message containing bundle information MUST be sent back,
which will notify the receiving end not to include the device in any further communication.
## Security Considerations
@@ -151,7 +188,9 @@ In this case an empty message containing bundle information MUST be sent back, w
### Recommendations
1. The value of `n` SHOULD be configured by the app-protocol.
- The default value SHOULD be 3, since a larger number of devices will result in a larger bundle size, which may not be desirable in a peer-to-peer network.
- The default value SHOULD be 3,
since a larger number of devices will result in a larger bundle size,
which may not be desirable in a peer-to-peer network.
## Copyright
@@ -161,4 +200,3 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
1. [53/WAKU2-X3DH](../53/x3dh.md)
2. [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)

BIN
waku/standards/core/.DS_Store vendored Normal file
View File

Binary file not shown.

363
waku/standards/core/10/waku2.md
View File

@@ -15,61 +15,75 @@ contributors:
## Abstract
Waku v2 is family of modular peer-to-peer protocols for secure communication.
The protocols are designed to be secure, privacy-preserving, censorship-resistant and being able to run in resource restricted environments.
At a high level, it implements Pub/Sub over [libp2p](https://github.com/libp2p/specs) and adds a set of capabilities to it.
The protocols are designed to be secure, privacy-preserving, censorship-resistant
and being able to run in resource restricted environments.
At a high level, it implements Pub/Sub over [libp2p](https://github.com/libp2p/specs)
and adds a set of capabilities to it.
These capabilities are things such as:
(i) retrieving historical messages for mostly-offline devices
(ii) adaptive nodes, allowing for heterogeneous nodes to contribute to the network
(iii) preserving bandwidth usage for resource-restriced devices
This makes Waku ideal for running a p2p protocol on mobile and in similarly restricted environments.
This makes Waku ideal for running a p2p protocol on mobile and
in similarly restricted environments.
Historically, it has its roots in [6/WAKU1](../../legacy/6/waku1.md),
which stems from [Whisper](https://eips.ethereum.org/EIPS/eip-627), originally part of the Ethereum stack.
which stems from [Whisper](https://eips.ethereum.org/EIPS/eip-627),
originally part of the Ethereum stack.
However, Waku v2 acts more as a thin wrapper for PubSub and has a different API.
It is implemented in an iterative manner where initial focus is on porting essential functionality to libp2p.
It is implemented in an iterative manner where initial focus
is on porting essential functionality to libp2p.
See [rough road map (2020)](https://vac.dev/waku-v2-plan) for more historical context.
## Motivation and goals
Waku as a family of protocols is designed to have a set of properties that are useful for many applications:
Waku as a family of protocols is designed to have a set of properties
that are useful for many applications:
1. **Useful for generalized messaging.**
1.**Useful for generalized messaging.**
Many applications require some form of messaging protocol to communicate between different subsystems or different nodes.
Many applications require some form of messaging protocol to communicate
between different subsystems or different nodes.
This messaging can be human-to-human or machine-to-machine or a mix.
Waku is designed to work for all these scenarios.
2. **Peer-to-peer.**
2.**Peer-to-peer.**
Applications sometimes have requirements that make them suitable for peer-to-peer solutions:
- Censorship-resistant with no single point of failure
Applications sometimes have requirements that make them suitable
for peer-to-peer solutions:
- Censorship-resistant with no single point of failure
- Adaptive and scalable network
- Shared infrastructure
3. **Runs anywhere.**
3.**Runs anywhere.**
Applications often run in restricted environments, where resources or the environment is restricted in some fashion.
Applications often run in restricted environments,
where resources or the environment is restricted in some fashion.
For example:
- Limited bandwidth, CPU, memory, disk, battery, etc
- Not being publicly connectable
- Only being intermittently connected; mostly-offline
4. **Privacy-preserving.**
4.**Privacy-preserving.**
Applications often have a desire for some privacy guarantees, such as:
- Pseudonymity and not being tied to any personally identifiable information (PII)
- Metadata protection in transit
- Various forms of unlinkability, etc
5. **Modular design.**
- Pseudonymity and not being tied to any personally identifiable information (PII)
- Metadata protection in transit
- Various forms of unlinkability, etc
Applications often have different trade-offs when it comes to what properties they and their users value.
Waku is designed in a modular fashion where an application protocol or node can choose what protocols they run.
5.**Modular design.**
Applications often have different trade-offs when it comes to what properties they
and their users value.
Waku is designed in a modular fashion where an application protocol or
node can choose what protocols they run.
We call this concept *adaptive nodes*.
For example:
- Resource usage vs metadata protection
- Providing useful services to the network vs mostly using it
- Stronger guarantees for spam protection vs economic registration cost
@@ -79,7 +93,8 @@ please see the [30/ADAPTIVE-NODES](../../../informational/30/adaptive-nodes.md)
## Network interaction domains
While Waku is best thought of as a single cohesive thing, there are three network interaction domains:
While Waku is best thought of as a single cohesive thing,
there are three network interaction domains:
(a) gossip domain
(b) discovery domain
@@ -88,19 +103,24 @@ While Waku is best thought of as a single cohesive thing, there are three networ
### Protocols and identifiers
Since Waku v2 is built on top of libp2p, many protocols have a libp2p protocol identifier.
The current main [protocol identifiers](https://docs.libp2p.io/concepts/protocols/) are:
The current main [protocol identifiers](https://docs.libp2p.io/concepts/protocols/)
are:
1. `/vac/waku/relay/2.0.0`
2. `/vac/waku/store/2.0.0-beta4`
3. `/vac/waku/filter/2.0.0-beta1`
4. `/vac/waku/lightpush/2.0.0-beta1`
This is in addition to protocols that specify messages, payloads, and recommended usages.
Since these aren't negotiated libp2p protocols, they are referred to by their RFC ID.
This is in addition to protocols that specify messages, payloads, and
recommended usages.
Since these aren't negotiated libp2p protocols,
they are referred to by their RFC ID.
For example:
- [14/WAKU2-MESSAGE](../14/message.md) and [26/WAKU-PAYLOAD](../../application/26/payload.md) for message payloads
- [23/WAKU2-TOPICS](../../../informational/23/topics.md) and [27/WAKU2-PEERS](../../../informational/27/peers.md) for recommendations around usage
- [14/WAKU2-MESSAGE](../14/message.md) and
[26/WAKU-PAYLOAD](../../application/26/payload.md) for message payloads
- [23/WAKU2-TOPICS](../../../informational/23/topics.md) and
[27/WAKU2-PEERS](../../../informational/27/peers.md) for recommendations around usage
There are also more experimental libp2p protocols such as:
@@ -109,12 +129,14 @@ There are also more experimental libp2p protocols such as:
These protocols and their semantics are elaborated on in their own specs.
### Use of libp2p and protobuf
Unless otherwise specified, all protocols are implemented over libp2p and use Protobuf by default.
Unless otherwise specified,
all protocols are implemented over libp2p and use Protobuf by default.
Since messages are exchanged over a [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/),
as a convention, libp2p protocols prefix binary message payloads with the length of the message in bytes.
as a convention,
libp2p protocols prefix binary message payloads with
the length of the message in bytes.
This length integer is encoded as a [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints).
### Gossip domain
@@ -127,62 +149,74 @@ See [11/WAKU2-RELAY](../11/relay.md) spec for more details.
For an experimental privacy-preserving economic spam protection mechanism, see [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
See [23/WAKU2-TOPICS](../../../informational/23/topics.md) for more information about recommended topic usage.
See [23/WAKU2-TOPICS](../../../informational/23/topics.md)
for more information about recommended topic usage.
### Direct use of libp2p protocols
In addition to `/vac/waku/*` protocols, Waku v2 MAY directly use the following libp2p protocols:
In addition to `/vac/waku/*` protocols,
Waku v2 MAY directly use the following libp2p protocols:
* [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping) with protocol id
- [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping)
with protocol id
```
```text
/ipfs/ping/1.0.0
```
for liveness checks between peers, or to keep peer-to-peer connections alive.
* [libp2p identity and identity/push](https://docs.libp2p.io/concepts/protocols/#identify) with protocol IDs
- [libp2p identity and identity/push](https://docs.libp2p.io/concepts/protocols/#identify)
with protocol IDs
```
```text
/ipfs/id/1.0.0
```
and
```
```text
/ipfs/id/push/1.0.0
```
respectively, as basic means for capability discovery.
These protocols are anyway used by the libp2p connection establishment layer Waku v2 is built on.
We plan to introduce a new Vac capability discovery protocol with better anonymity properties and more functionality.
These protocols are anyway used by the libp2p connection
establishment layer Waku v2 is built on.
We plan to introduce a new Vac capability discovery protocol
with better anonymity properties and more functionality.
# Transports
#### Transports
Waku v2 is built in top of libp2p, and like libp2p it strives to be transport agnostic.
We define a set of recommended transports in order to achieve a baseline of interoperability between clients.
We define a set of recommended transports in order to achieve a baseline of
interoperability between clients.
This section describes these recommended transports.
Waku client implementations SHOULD support the TCP transport.
Where TCP is supported it MUST be enabled for both dialing and listening, even if other transports are available.
Where TCP is supported it MUST be enabled for both dialing and listening,
even if other transports are available.
Waku v2 nodes where the environment do not allow to use TCP directly, MAY use other transports.
Waku v2 nodes where the environment do not allow to use TCP directly,
MAY use other transports.
A Waku v2 node SHOULD support secure websockets for bidirectional communication streams, for example in a web browser context.
A node MAY support unsecure websockets if required by the application or running environment.
A Waku v2 node SHOULD support secure websockets for bidirectional communication streams,
for example in a web browser context.
A node MAY support unsecure websockets if required by the application or
running environment.
### Discovery domain
#### Discovery methods
Waku v2 can retrieve a list of nodes to connect to using DNS-based discovery as per [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459).
Waku v2 can retrieve a list of nodes to connect to using DNS-based discovery
as per [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459).
While this is a useful way of bootstrapping connection to a set of peers,
it MAY be used in conjunction with an [ambient peer discovery](https://docs.libp2p.io/concepts/publish-subscribe/#discovery) procedure to find still other nodes to connect to,
it MAY be used in conjunction with an [ambient peer discovery](https://docs.libp2p.io/concepts/publish-subscribe/#discovery)
procedure to find still other nodes to connect to,
such as [Node Discovery v5](https://github.com/ethereum/devp2p/blob/8fd5f7e1c1ec496a9d8dc1640a8548b8a8b5986b/discv5/discv5.md).
More ambient peer discovery methods are being tested for Waku v2,
and will be specified for wider adoption.
@@ -190,13 +224,15 @@ It is possible to bypass the discovery domain by specifying static nodes.
#### Use of ENR
[31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md) describes the usage of [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778) for Waku v2 discovery purposes.
It introduces two new ENR fields, `multiaddrs` and `waku2`, that a Waku v2 node MAY use for discovery purposes.
[31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
describes the usage of [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778)
for Waku v2 discovery purposes.
It introduces two new ENR fields, `multiaddrs` and
`waku2`, that a Waku v2 node MAY use for discovery purposes.
These fields MUST be used under certain conditions, as set out in the spec.
Both EIP-1459 DNS-based discovery and Node Discovery v5 operates on ENR,
and it's reasonable to expect even wider utility for ENR in Waku v2 networks in future.
### Request/Reply domain
In addition to the Gossip domain,
@@ -211,7 +247,8 @@ such as low bandwidth or being mostly offline.
This is used to fetch historical messages for mostly offline devices.
See [13/WAKU2-STORE spec](../13/store.md) spec for more details.
There is also an experimental fault-tolerant addition to the store protocol that relaxes the high availability requirement.
There is also an experimental fault-tolerant addition to the store protocol
that relaxes the high availability requirement.
See [21/WAKU2-FT-STORE](../../application/21/fault-tolerant-store.md)
#### Content filtering
@@ -225,13 +262,15 @@ See [12/WAKU2-FILTER](../12/filter.md) spec for more details.
**Protocol identifier***: `/vac/waku/lightpush/2.0.0-beta1`
This is used for nodes with short connection windows and limited bandwidth to publish messages into the Waku network.
This is used for nodes with short connection windows and
limited bandwidth to publish messages into the Waku network.
See [19/WAKU2-LIGHTPUSH](../19/lightpush.md) spec for more details.
#### Other protocols
The above is a non-exhaustive list,
and due to the modular design of Waku there may be other protocols here that provide a useful service to the Waku network.
and due to the modular design of Waku,
there may be other protocols here that provide a useful service to the Waku network.
### Overview of protocol interaction
@@ -241,20 +280,29 @@ See the sequence diagram below for an overview of how different protocols intera
0. We have six nodes, A-F.
The protocols initially mounted are indicated as such.
The PubSub topics `pubtopic1` and `pubtopic2` is used for routing and indicates that it is subscribed to messages on that topic for relay, see [11/WAKU2-RELAY](../11/relay.md) for details.
Ditto for [13/WAKU2-STORE](../13/store.md) where it indicates that these messages are persisted on that node.
The PubSub topics `pubtopic1` and
`pubtopic2` is used for routing and
indicates that it is subscribed to messages on that topic for relay,
see [11/WAKU2-RELAY](../11/relay.md) for details.
Ditto for [13/WAKU2-STORE](../13/store.md)
where it indicates that these messages are persisted on that node.
1. Node A creates a WakuMessage `msg1` with a ContentTopic `contentTopic1`.
See [14/WAKU2-MESSAGE](../14/message.md) for more details.
If WakuMessage version is set to 1, we use the [6/WAKU1](../../legacy/6/waku1.md) compatible `data` field with encryption.
If WakuMessage version is set to 1,
we use the [6/WAKU1](../../legacy/6/waku1.md) compatible `data` field with encryption.
See [7/WAKU-DATA](../../legacy/7/data.md) for more details.
2. Node F requests to get messages filtered by PubSub topic `pubtopic1` and ContentTopic `contentTopic1`.
Node D subscribes F to this filter and will in the future forward messages that match that filter.
2. Node F requests to get messages filtered by PubSub topic `pubtopic1` and
ContentTopic `contentTopic1`.
Node D subscribes F to this filter and
will in the future forward messages that match that filter.
See [12/WAKU2-FILTER](../12/filter.md) for more details.
3. Node A publishes `msg1` on `pubtopic1` and subscribes to that relay topic pick it up.
It then gets relayed further from B to D, but not C since it doesn't subscribe to that topic.
3. Node A publishes `msg1` on `pubtopic1` and
subscribes to that relay topic pick it up.
It then gets relayed further from B to D, but
not C since it doesn't subscribe to that topic.
See [11/WAKU2-RELAY](../11/relay.md).
4. Node D saves `msg1` for possible later retrieval by other nodes.
@@ -272,81 +320,135 @@ Node D responds with messages meeting this (and possibly other) criteria. See [1
### Compatibility with Waku v1
Waku v1 and Waku v2 are different protocols all together.
They use a different transport protocol underneath; Waku v1 is devp2p RLPx based while Waku v2 uses libp2p.
They use a different transport protocol underneath;
Waku v1 is devp2p RLPx based while Waku v2 uses libp2p.
The protocols themselves also differ as does their data format.
Compatibility can be achieved only by using a bridge that not only talks both devp2p RLPx and libp2p, but that also transfers (partially) the content of a packet from one version to the other.
Compatibility can be achieved only by using a bridge
that not only talks both devp2p RLPx and libp2p,
but that also transfers (partially) the content of a packet from one version
to the other.
See [15/WAKU-BRIDGE](../15/bridge.md) for details on a bidirectional bridge mode.
# Appendix B: Security
## Appendix B: Security
Each protocol layer of Waku v2 provides a distinct service and is associated with a separate set of security features and concerns.
Therefore, the overall security of Waku v2 depends on how the different layers are utilized.
In this section, we overview the security properties of Waku v2 protocols against a static adversarial model which is described below.
Note that a more detailed security analysis of each Waku protocol is supplied in its respective specification as well.
Each protocol layer of Waku v2 provides a distinct service and
is associated with a separate set of security features and concerns.
Therefore, the overall security of Waku v2
depends on how the different layers are utilized.
In this section,
we overview the security properties of Waku v2 protocols
against a static adversarial model which is described below.
Note that a more detailed security analysis of each Waku protocol
is supplied in its respective specification as well.
## Primary Adversarial Model
In the primary adversarial model, we consider adversary as a passive entity that attempts to collect information from others to conduct an attack,
In the primary adversarial model,
we consider adversary as a passive entity that attempts to collect information
from others to conduct an attack,
but it does so without violating protocol definitions and instructions.
The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers
(unless the adversary is one end of the communication).
Specifically, the communication channels are assumed to be secure.
The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links
between arbitrary pairs of peers
(unless the adversary is one end of the communication).
Specifically, the communication channels are assumed to be secure.
## Security Features
### Pseudonymity
### Pseudonymity
Waku v2 by default guarantees pseudonymity for all of the protocol layers since parties do not have to disclose their true identity
Waku v2 by default guarantees pseudonymity for all of the protocol layers
since parties do not have to disclose their true identity
and instead they utilize libp2p `PeerID` as their identifiers.
While pseudonymity is an appealing security feature, it does not guarantee full anonymity since the actions taken under the same pseudonym
i.e., `PeerID` can be linked together and potentially result in the re-identification of the true actor.
While pseudonymity is an appealing security feature,
it does not guarantee full anonymity since the actions taken under the same pseudonym
i.e., `PeerID` can be linked together and
potentially result in the re-identification of the true actor.
### Anonymity / Unlinkability
At a high level, anonymity is the inability of an adversary in linking an actor to its data/performed action (the actor and action are context-dependent).
To be precise about linkability, we use the term Personally Identifiable Information (PII) to refer to any piece of data that could potentially be used to uniquely identify a party.
For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
Notice that users' actions can be traced through their PIIs (e.g., signatures) and hence result in their re-identification risk.
As such, we seek anonymity by avoiding linkability between actions and the actors / actors' PII. Concerning anonymity, Waku v2 provides the following features:
At a high level,
anonymity is the inability of an adversary in linking an actor
to its data/performed action (the actor and action are context-dependent).
To be precise about linkability,
we use the term Personally Identifiable Information (PII)
to refer to any piece of data that could potentially
be used to uniquely identify a party.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and
hence count as PII.
Notice that users' actions can be traced through their PIIs
(e.g., signatures) and hence result in their re-identification risk.
As such, we seek anonymity by avoiding linkability between actions and
the actors / actors' PII. Concerning anonymity, Waku v2 provides the following features:
**Publisher-Message Unlinkability**:
This feature signifies the unlinkability of a publisher to its published messages in the 11/WAKU2-RELAY protocol.
The [Publisher-Message Unlinkability](../11/relay.md/#security-analysis) is enforced through the `StrictNoSign` policy due to which the data fields of pubsub messages that count as PII for the publisher must be left unspecified.
This feature signifies the unlinkability of a publisher
to its published messages in the 11/WAKU2-RELAY protocol.
The [Publisher-Message Unlinkability](../11/relay.md/#security-analysis)
is enforced through the `StrictNoSign` policy due to which the data fields
of pubsub messages that count as PII for the publisher must be left unspecified.
**Subscriber-Topic Unlinkability**:
This feature stands for the unlinkability of the subscriber to its subscribed topics in the 11/WAKU2-RELAY protocol.
The [Subscriber-Topic Unlinkability](../11/relay.md/#security-analysis) is achieved through the utilization of a single PubSub topic.
As such, subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of subscribers).
Note that there is no hard limit on the number of the pubsub topics, however, the use of one topic is recommended for the sake of anonymity.
This feature stands for the unlinkability of the subscriber
to its subscribed topics in the 11/WAKU2-RELAY protocol.
The [Subscriber-Topic Unlinkability](../11/relay.md/#security-analysis)
is achieved through the utilization of a single PubSub topic.
As such, subscribers are not re-identifiable from their subscribed topic IDs
as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
where k is proportional to the system size (number of subscribers).
Note that there is no hard limit on the number of the pubsub topics, however,
the use of one topic is recommended for the sake of anonymity.
### Spam protection
This property indicates that no adversary can flood the system (i.e., publishing a large number of messages in a short amount of time), either accidentally or deliberately, with any kind of message i.e. even if the message content is valid or useful.
Spam protection is partly provided in `11/WAKU2-RELAY` through the [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures) provided for by GossipSub v1.1.
At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
This property indicates that no adversary can flood the system
(i.e., publishing a large number of messages in a short amount of time),
either accidentally or deliberately, with any kind of message
i.e. even if the message content is valid or useful.
Spam protection is partly provided in `11/WAKU2-RELAY`
through the [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures)
rovided for by GossipSub v1.1.
At a high level,
peers utilize a scoring function to locally score the behavior
of their connections and remove peers with a low score.
### Data confidentiality, Integrity, and Authenticity
Confidentiality can be addressed through data encryption whereas integrity and authenticity are achievable through digital signatures.
These features are provided for in [14/WAKU2-MESSAGE (version 1)](../14/message.md/#version-1)` through payload encryption as well as encrypted signatures.
Confidentiality can be addressed through data encryption whereas integrity and
authenticity are achievable through digital signatures.
These features are provided for in [14/WAKU2-MESSAGE (version 1)](../14/message.md/#version-1)`
through payload encryption as well as encrypted signatures.
## Security Considerations
**Lack of anonymity/unlinkability in the protocols involving direct connections including `13/WAKU2-STORE` and `12/WAKU2-FILTER` protocols**:
The anonymity/unlinkability is not guaranteed in the protocols like `13/WAKU2-STORE` and `12/WAKU2-FILTER` where peers need to have direct connections to benefit from the designated service.
This is because during the direct connections peers utilize `PeerID` to identify each other,
therefore the service obtained in the protocol is linkable to the beneficiary's `PeerID` (which counts as PII).
For `13/WAKU2-STORE`, the queried node would be able to link the querying node's `PeerID` to its queried topics.
Likewise, in the `12/WAKU2-FILTER`, a full node can link the light node's `PeerID`s to its content filter.
Lack of anonymity/unlinkability in the protocols involving direct connections
including `13/WAKU2-STORE` and `12/WAKU2-FILTER` protocols:
<!-- TODO: to inspect the nim-libp2p codebase and figure out the exact use of PeerIDs in direct communication, it might be the case that the requester does not have to disclose its PeerID-->
The anonymity/unlinkability is not guaranteed in the protocols like `13/WAKU2-STORE`
and `12/WAKU2-FILTER` where peers need to have direct connections
to benefit from the designated service.
This is because during the direct connections peers utilize `PeerID`
to identify each other,
therefore the service obtained in the protocol is linkable
to the beneficiary's `PeerID` (which counts as PII).
For `13/WAKU2-STORE`,
the queried node would be able to link the querying node's `PeerID`
to its queried topics.
Likewise, in the `12/WAKU2-FILTER`,
a full node can link the light node's `PeerID`s to its content filter.
<!--TODO: might be good to add a figure visualizing the Waku protocol stack and the security features of each layer-->
<!-- TODO: to inspect the nim-libp2p codebase and
figure out the exact use of PeerIDs in direct communication,
it might be the case that the requester does not have to disclose its PeerID-->
<!--TODO: might be good to add a figure visualizing the Waku protocol stack and
the security features of each layer-->
## Appendix C: Implementation Notes
@@ -358,34 +460,35 @@ There are multiple implementations of Waku v2 and its protocols:
- [go-waku (Go)](https://github.com/status-im/go-waku/)
- [js-waku (NodeJS and Browser)](https://github.com/status-im/js-waku/)
Below you can find an overview of the specs that they implement as they relate to Waku v2.
Below you can find an overview of the specs that they implement
as they relate to Waku v2.
This includes Waku v1 specs, as they are used for bridging between the two networks.
| Spec | nim-waku (Nim) | go-waku (Go) | js-waku (Node JS) | js-waku (Browser JS) |
| ---- | -------------- | ------------ | ----------------- | -------------------- |
|[6/WAKU1](../../legacy/6/waku1.md)|✔|||
|[7/WAKU-DATA](../../legacy/7/data.md)|✔|✔||
|[8/WAKU-MAIL](../../legacy/8/mail.md)|✔|||
|[9/WAKU-RPC](../../legacy/9/rpc.md)|✔|||
|[6/WAKU1](../../legacy/6/waku1.md)|✔||||
|[7/WAKU-DATA](../../legacy/7/data.md)|✔|✔|||
|[8/WAKU-MAIL](../../legacy/8/mail.md)|✔||||
|[9/WAKU-RPC](../../legacy/9/rpc.md)|✔||||
|[10/WAKU2](../10/waku2.md)|✔|🚧|🚧|🚧|
|[11/WAKU2-RELAY](../11/relay.md)|✔|✔|✔|✔|
|[12/WAKU2-FILTER](../12/filter.md)|✔|✔||
|[12/WAKU2-FILTER](../12/filter.md)|✔|✔|||
|[13/WAKU2-STORE](../13/store.md)|✔|✔|✔\*|✔\*|
|[14/WAKU2-MESSAGE](../14/message.md))|✔|✔|✔|✔|
|[15/WAKU2-BRIDGE](../15/bridge.md)|✔|||
|[16/WAKU2-RPC](../16/rpc.md)|✔|||
|[17/WAKU2-RLN-RELAY](../17/rln-relay.md)|🚧|||
|[18/WAKU2-SWAP](../../application/18/swap.md)|🚧|||
|[15/WAKU2-BRIDGE](../15/bridge.md)|✔||||
|[16/WAKU2-RPC](../16/rpc.md)|✔||||
|[17/WAKU2-RLN-RELAY](../17/rln-relay.md)|🚧||||
|[18/WAKU2-SWAP](../../application/18/swap.md)|🚧||||
|[19/WAKU2-LIGHTPUSH](../19/lightpush.md)|✔|✔|✔\**|✔\**|
|[21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)|✔|✔||
|[21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)|✔|✔|||
*js-waku implements [13/WAKU2-STORE](../13/store.md) as a querying node only.
**js-waku only implements [19/WAKU2-LIGHTPUSH](../19/lightpush.md) requests.
### Recommendations for clients
To implement a minimal Waku v2 client, we recommend implementing the following subset in the following order:
To implement a minimal Waku v2 client,
we recommend implementing the following subset in the following order:
- [10/WAKU2](../10/waku2.md) - this spec
- [11/WAKU2-RELAY](../11/relay.md) - for basic operation
@@ -404,23 +507,32 @@ with periodic pings to connected peers
## Appendix D: Future work
The following features are currently experimental and under research and initial implementation:
The following features are currently experimental and under research and
initial implementation:
**Economic Spam resistance**:
We aim to enable an incentivized spam protection technique to enhance `11/WAKU2-RELAY` by using rate limiting nullifiers.
We aim to enable an incentivized spam protection technique
to enhance `11/WAKU2-RELAY` by using rate limiting nullifiers.
More details on this can be found in [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
In this advanced method, peers are limited to a certain rate of messaging per epoch and an immediate financial penalty is enforced for spammers who break this rate.
In this advanced method,
peers are limited to a certain rate of messaging per epoch and
an immediate financial penalty is enforced for spammers who break this rate.
**Prevention of Denial of Service (DoS) and Node Incentivization**:
Denial of service signifies the case where an adversarial node exhausts another node's service capacity (e.g., by making a large number of requests) and makes it unavailable to the rest of the system.
Denial of service signifies the case where an adversarial node
exhausts another node's service capacity (e.g., by making a large number of requests)
and makes it unavailable to the rest of the system.
DoS attack is to be mitigated through the accounting model as described in [18/WAKU2-SWAP](../../application/18/swap.md).
In a nutshell, peers have to pay for the service they obtain from each other.
In addition to incentivizing the service provider, accounting also makes DoS attacks costly for malicious peers.
The accounting model can be used in `13/WAKU2-STORE` and `12/WAKU2-FILTER` to protect against DoS attacks.
Additionally, this gives node operators who provide a useful service to the network an incentive to perform that service.
See [18/WAKU2-SWAP](../../application/18/swap.md) for more details on this piece of work.
In addition to incentivizing the service provider,
accounting also makes DoS attacks costly for malicious peers.
The accounting model can be used in `13/WAKU2-STORE` and
`12/WAKU2-FILTER` to protect against DoS attacks.
Additionally, this gives node operators who provide a useful service to the network
an incentive to perform that service.
See [18/WAKU2-SWAP](../../application/18/swap.md)
for more details on this piece of work.
## Copyright
@@ -448,7 +560,7 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
10. [27/WAKU2-PEERS](../../../informational/27/peers.md)
11. [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/)
11. [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/)
12. [Protobuf varint encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints)
@@ -497,4 +609,3 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
34. [18/WAKU2-SWAP spec](../../application/18/swap.md)
35. [21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)

199
waku/standards/core/11/relay.md
View File

@@ -10,53 +10,84 @@ contributors:
- Sanaz Taheri <sanaz@status.im>
---
`11/WAKU2-RELAY` specifies a [Publish/Subscribe approach](https://docs.libp2p.io/concepts/publish-subscribe/) to peer-to-peer messaging with a strong focus on privacy, censorship-resistance, security and scalability.
Its current implementation is a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) and prescribes gossip-based dissemination.
As such the scope is limited to defining a separate [`protocol id`](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation) for `11/WAKU2-RELAY`, establishing privacy and security requirements, and defining how the underlying GossipSub is to be interpreted and implemented within the Waku and cryptoeconomic domain.
`11/WAKU2-RELAY` specifies a [Publish/Subscribe approach](https://docs.libp2p.io/concepts/publish-subscribe/)
to peer-to-peer messaging with a strong focus on privacy,
censorship-resistance, security and scalability.
Its current implementation is a minor extension of the
[libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
and prescribes gossip-based dissemination.
As such the scope is limited to defining a separate
[`protocol id`](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation)
for `11/WAKU2-RELAY`, establishing privacy and security requirements,
and defining how the underlying GossipSub is to be interpreted and
implemented within the Waku and cryptoeconomic domain.
`11/WAKU2-RELAY` should not be confused with [libp2p circuit relay](https://github.com/libp2p/specs/tree/master/relay).
**Protocol identifier**: `/vac/waku/relay/2.0.0`
## Security Requirements
The `11/WAKU2-RELAY` protocol is designed to provide the following security properties under a static [Adversarial Model](#adversarial-model).
Note that data confidentiality, integrity, and authenticity are currently considered out of scope for `11/WAKU2-RELAY` and must be handled by higher layer protocols such as [`14/WAKU2-MESSAGE`](../14/message.md).
The `11/WAKU2-RELAY` protocol is designed to provide the following security properties
under a static [Adversarial Model](#adversarial-model).
Note that data confidentiality, integrity, and
authenticity are currently considered out of scope for `11/WAKU2-RELAY` and
must be handled by higher layer protocols such as [`14/WAKU2-MESSAGE`](../14/message.md).
<!-- May add the definition of the unsupported feature:
Confidentiality indicates that an adversary should not be able to learn the data carried by the `WakuRelay` protocol.
Integrity indicates that the data transferred by the `WakuRelay` protocol can not be tampered with by an adversarial entity without being detected.
Authenticity no adversary can forge data on behalf of a targeted publisher and make it accepted by other subscribers as if the origin is the target. -->
Confidentiality indicates that an adversary
should not be able to learn the data carried by the `WakuRelay` protocol.
Integrity indicates that the data transferred by the `WakuRelay` protocol
can not be tampered with by an adversarial entity without being detected.
Authenticity no adversary can forge data on behalf of a targeted publisher and
make it accepted by other subscribers as if the origin is the target. -->
- **Publisher-Message Unlinkability**:
This property indicates that no adversarial entity can link a published `Message` to its publisher.
This feature also implies the unlinkability of the publisher to its published topic ID as the `Message` embodies the topic IDs.
This property indicates that no adversarial entity can link a published `Message`
to its publisher.
This feature also implies the unlinkability of the publisher
to its published topic ID as the `Message` embodies the topic IDs.
- **Subscriber-Topic Unlinkability**:
This feature stands for the inability of any adversarial entity from linking a subscriber to its subscribed topic IDs.
This feature stands for the inability of any adversarial entity
from linking a subscriber to its subscribed topic IDs.
<!-- TODO: more requirements can be added, but that needs further and deeper investigation-->
<!-- TODO: more requirements can be added,
but that needs further and deeper investigation-->
### Terminology
_Personally identifiable information_ (PII) refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
_Personally identifiable information_ (PII)
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key,
and the hash of one's static IP address are unique for each user and
hence count as PII.
## Adversarial Model
- Any entity running the `11/WAKU2-RELAY` protocol is considered an adversary.
This includes publishers, subscribers, and all the peers' direct connections.
Furthermore, we consider the adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions.
For example, under the passive adversarial model, no malicious subscriber hides the messages it receives from other subscribers as it is against the description of `11/WAKU2-RELAY`.
However, a malicious subscriber may learn which topics are subscribed to by which peers.
- The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
- Any entity running the `11/WAKU2-RELAY` protocol is considered an adversary.
This includes publishers, subscribers, and all the peers' direct connections.
Furthermore,
we consider the adversary as a passive entity that attempts to collect information
from others to conduct an attack but
it does so without violating protocol definitions and instructions.
For example, under the passive adversarial model,
no malicious subscriber hides the messages it receives from other subscribers
as it is against the description of `11/WAKU2-RELAY`.
However,
a malicious subscriber may learn which topics are subscribed to by which peers.
- The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs
of peers (unless the adversary is one end of the communication).
In other words, the communication channels are assumed to be secure.
## Wire Specification
The [PubSub interface specification](https://github.com/libp2p/specs/blob/master/pubsub/README.md) defines the protobuf RPC messages exchanged between peers participating in a GossipSub network.
We republish these messages here for ease of reference and define how `11/WAKU2-RELAY` uses and interprets each field.
The [PubSub interface specification](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
defines the protobuf RPC messages
exchanged between peers participating in a GossipSub network.
We republish these messages here for ease of reference and
define how `11/WAKU2-RELAY` uses and interprets each field.
### Protobuf definitions
@@ -86,10 +117,11 @@ message RPC {
```
> **_NOTE:_**
The various [control messages](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#control-messages) defined for GossipSub are used as specified there.
The various [control messages](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#control-messages)
defined for GossipSub are used as specified there.
> **_NOTE:_**
The [`TopicDescriptor`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) is not currently used by `11/WAKU2-RELAY`.
The [`TopicDescriptor`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
is not currently used by `11/WAKU2-RELAY`.
### Message fields
@@ -103,19 +135,25 @@ See [`14/WAKU2-MESSAGE`](../14/message.md) for more details.
- The `seqno` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
- The `topicIDs` field MUST contain the content-topics that a message is being published on.
- The `topicIDs` field MUST contain the content-topics
that a message is being published on.
- The `signature` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
- The `signature` field MUST NOT be used,
following the [`StrictNoSign` signature policy](#signature-policy).
- The `key` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-solicy).
- The `key` field MUST NOT be used,
following the [`StrictNoSign` signature policy](#signature-policy).
### SubOpts fields
The `SubOpts` protobuf defines the format in which subscription options are relayed between peers.
A `11/WAKU2-RELAY` node MAY decide to subscribe or unsubscribe from topics by sending updates using `SubOpts`.
The `SubOpts` protobuf defines the format
in which subscription options are relayed between peers.
A `11/WAKU2-RELAY` node MAY decide to subscribe or
unsubscribe from topics by sending updates using `SubOpts`.
The following usage requirements apply:
- The `subscribe` field MUST contain a boolean, where `true` indicates subscribe and `false` indicates unsubscribe to a topic.
- The `subscribe` field MUST contain a boolean,
where `true` indicates subscribe and `false` indicates unsubscribe to a topic.
- The `topicid` field MUST contain the pubsub topic.
@@ -124,44 +162,93 @@ The following usage requirements apply:
### Signature Policy
The [`StrictNoSign` option](https://github.com/libp2p/specs/blob/master/pubsub/README.md#signature-policy-options) MUST be used, to ensure that messages are built without the `signature`, `key`, `from` and `seqno` fields.
Note that this does not merely imply that these fields be empty, but that they MUST be _absent_ from the marshalled message.
The [`StrictNoSign` option](https://github.com/libp2p/specs/blob/master/pubsub/README.md#signature-policy-options)
MUST be used, to ensure that messages are built without the `signature`,
`key`, `from` and `seqno` fields.
Note that this does not merely imply that these fields be empty, but
that they MUST be _absent_ from the marshalled message.
## Security Analysis
<!-- TODO: realized that the prime security objective of the `WakuRelay` protocol is to provide peers unlinkability as such this feature is prioritized over other features e.g., unlinkability is preferred over authenticity and integrity. It might be good to motivate unlinkability and its impact on the relay protocol or other protocols invoking relay protocol.-->
<!-- TODO: realized that the prime security objective of the `WakuRelay`
protocol is to provide peers unlinkability
as such this feature is prioritized over other features
e.g., unlinkability is preferred over authenticity and integrity.
It might be good to motivate unlinkability and
its impact on the relay protocol or other protocols invoking relay protocol.-->
- **Publisher-Message Unlinkability**:
To address publisher-message unlinkability, one should remove any PII from the published message.
As such, `11/WAKU2-RELAY` follows the `StrictNoSign` policy as described in [libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing).
As the result of the `StrictNoSign` policy, `Message`s should be built without the `from`, `signature` and `key` fields since each of these three fields individually counts as PII for the author of the message (one can link the creation of the message with libp2p peerId and thus indirectly with the IP address of the publisher).
Note that removing identifiable information from messages cannot lead to perfect unlinkability.
The direct connections of a publisher might be able to figure out which `Message`s belong to that publisher by analyzing its traffic.
The possibility of such inference may get higher when the `data` field is also not encrypted by the upper-level protocols. <!-- TODO: more investigation on traffic analysis attacks and their success probability-->
To address publisher-message unlinkability,
one should remove any PII from the published message.
As such, `11/WAKU2-RELAY` follows the `StrictNoSign` policy as described in
[libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing).
As the result of the `StrictNoSign` policy,
`Message`s should be built without the `from`,
`signature` and `key` fields since each of these three fields individually
counts as PII for the author of the message
(one can link the creation of the message with libp2p peerId and
thus indirectly with the IP address of the publisher).
Note that removing identifiable information from messages
cannot lead to perfect unlinkability.
The direct connections of a publisher
might be able to figure out which `Message`s belong to that publisher
by analyzing its traffic.
The possibility of such inference may get higher
when the `data` field is also not encrypted by the upper-level protocols.
<!-- TODO: more investigation on traffic analysis attacks and their success probability-->
- **Subscriber-Topic Unlinkability:**
To preserve subscriber-topic unlinkability, it is recommended by [`10/WAKU2`](../10/waku2.md) to use a single PubSub topic in the `11/WAKU2-RELAY` protocol.
This allows an immediate subscriber-topic unlinkability where subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of participants of Waku relay protocol).
To preserve subscriber-topic unlinkability,
it is recommended by [`10/WAKU2`](../10/waku2.md) to use a single PubSub topic
in the `11/WAKU2-RELAY` protocol.
This allows an immediate subscriber-topic unlinkability
where subscribers are not re-identifiable from their subscribed topic IDs
as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity
is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
where k is proportional to the system size
(number of participants of Waku relay protocol).
However, note that `11/WAKU2-RELAY` supports the use of more than one topic.
In case that more than one topic id is utilized, preserving unlinkability is the responsibility of the upper-level protocols which MAY adopt [partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic) to achieve K-anonymity for the subscribed peers.
In case that more than one topic id is utilized,
preserving unlinkability is the responsibility of the upper-level protocols
which MAY adopt
[partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic)
to achieve K-anonymity for the subscribed peers.
## Future work
- **Economic spam resistance**:
In the spam-protected `11/WAKU2-RELAY` protocol, no adversary can flood the system with spam messages (i.e., publishing a large number of messages in a short amount of time).
In the spam-protected `11/WAKU2-RELAY` protocol,
no adversary can flood the system with spam messages
(i.e., publishing a large number of messages in a short amount of time).
Spam protection is partly provided by GossipSub v1.1 through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures).
At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
`11/WAKU2-RELAY` aims at enabling an advanced spam protection mechanism with economic disincentives by utilizing Rate Limiting Nullifiers.
In a nutshell, peers must conform to a certain message publishing rate per a system-defined epoch, otherwise, they get financially penalized for exceeding the rate.
More details on this new technique can be found in [`17/WAKU2-RLN-RELAY`](../17/rln-relay.md).
<!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1 are taken in the nim-libp2p as well, may need to audit the code -->
At a high level,
peers utilize a scoring function to locally score the behavior of their connections
and remove peers with a low score.
`11/WAKU2-RELAY` aims at enabling an advanced spam protection mechanism
with economic disincentives by utilizing Rate Limiting Nullifiers.
In a nutshell,
peers must conform to a certain message publishing rate per a system-defined epoch,
otherwise, they get financially penalized for exceeding the rate.
More details on this new technique can be found in [`17/WAKU2-RLN-RELAY`](../17/rln-relay.md).
<!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1
are taken in the nim-libp2p as well, may need to audit the code -->
- Providing **Unlinkability**, **Integrity** and **Authenticity** simultaneously:
Integrity and authenticity are typically addressed through digital signatures and Message Authentication Code (MAC) schemes, however, the usage of digital signatures (where each signature is bound to a particular peer) contradicts with the unlinkability requirement (messages signed under a certain signature key are verifiable by a verification key that is bound to a particular publisher).
As such, integrity and authenticity are missing features in `11/WAKU2-RELAY` in the interest of unlinkability.
In future work, advanced signature schemes like group signatures can be utilized to enable authenticity, integrity, and unlinkability simultaneously.
In a group signature scheme, a member of a group can anonymously sign a message on behalf of the group as such the true signer is indistinguishable from other group members. <!-- TODO: shall I add a reference for group signatures?-->
Integrity and authenticity are typically addressed through digital signatures and
Message Authentication Code (MAC) schemes, however,
the usage of digital signatures (where each signature is bound to a particular peer)
contradicts with the unlinkability requirement
(messages signed under a certain signature key are verifiable by a verification key
that is bound to a particular publisher).
As such, integrity and authenticity are missing features in `11/WAKU2-RELAY`
in the interest of unlinkability.
In future work, advanced signature schemes like group signatures
can be utilized to enable authenticity, integrity, and unlinkability simultaneously.
In a group signature scheme, a member of a group can anonymously sign a message
on behalf of the group as such the true signer
is indistinguishable from other group members.
<!-- TODO: shall I add a reference for group signatures?-->
## Copyright

BIN
waku/standards/core/12/.DS_Store vendored Normal file
View File

Binary file not shown.

171
waku/standards/core/12/filter.md
View File

@@ -17,11 +17,16 @@ previous versions: [00](./previous-versions00)
---
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives.
This is a more lightweight version of `WakuRelay`
specifically designed for bandwidth restricted devices.
This is due to the fact that light nodes subscribe to full-nodes and
only receive the messages they desire.
## Content filtering
**Protocol identifiers**:
- _filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
- _filter-push_: `/vac/waku/filter-push/2.0.0-beta1`
@@ -46,24 +51,49 @@ It is worth noting that a light node could get by with only using the `store`
protocol to query for a recent time window, provided it is acceptable to do
frequent polling.
## Design Requirements
The effectiveness and reliability of the content filtering service enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
The effectiveness and reliability of the content filtering service enabled by
`WakuFilter` protocol rely on the _high availability_ of the full nodes
as the service providers.
To this end, full nodes must feature _high uptime_
(to persistently listen and capture the network messages)
as well as _high Bandwidth_ (to provide timely message delivery to the light nodes).
## Security Consideration
Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section.
Note that while using `WakuFilter` allows light nodes to save bandwidth,
it comes with a privacy cost in the sense that they need to
disclose their liking topics to the full nodes to retrieve the relevant messages.
Currently, anonymous subscription is not supported by the `WakuFilter`, however,
potential solutions in this regard are sketched
below in [Future Work](#future-work) section.
### Terminology
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
The term Personally identifiable information (PII)
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and hence count as PII.
## Adversarial Model
Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
Any node running the `WakuFilter` protocol
i.e., both the subscriber node and the queried node are considered as an adversary.
Furthermore, we consider the adversary as a passive entity
that attempts to collect information from other nodes to conduct an attack but
it does so without violating protocol definitions and instructions.
For example, under the passive adversarial model,
no malicious node intentionally hides the messages
matching to one's subscribed content filter
as it is against the description of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links
between arbitrary pairs of nodes (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
### Protobuf
@@ -106,9 +136,12 @@ message MessagePush {
### Filter-Subscribe
A filter service node MUST support the _filter-subscribe_ protocol
to allow filter clients to subscribe, modify, refresh and unsubscribe a desired set of filter criteria.
The combination of different filter criteria for a specific filter client node is termed a "subscription".
A filter client is interested in receiving messages matching the filter criteria in its registered subscriptions.
to allow filter clients to subscribe, modify, refresh and
unsubscribe a desired set of filter criteria.
The combination of different filter criteria
for a specific filter client node is termed a "subscription".
A filter client is interested in receiving messages matching the filter criteria
in its registered subscriptions.
Since a filter service node is consuming resources to provide this service,
it MAY account for usage and adapt its service provision to certain clients.
@@ -124,21 +157,29 @@ Each request MUST include a `filter_subscribe_type`, indicating the type of requ
#### Filter Subscribe Response
In return to any `FilterSubscribeRequest`,
a filter service node SHOULD respond with a `FilterSubscribeResponse` with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful or not.
a filter service node SHOULD respond with a `FilterSubscribeResponse`
with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful
or not.
Successful status codes are in the `2xx` range.
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
In addition, the filter service node MAY choose to provide a more detailed status description in the `status_desc` field.
Client nodes SHOULD consider all other status codes as error codes and
assume that the requested operation had failed.
In addition,
the filter service node MAY choose to provide a more detailed status description
in the `status_desc` field.
#### Filter matching
In the description of each request type below,
the term "filter criteria" refers to the combination of `pubsub_topic` and a set of `content_topics`.
The request MAY include filter criteria, conditional to the selected `filter_subscribe_type`.
the term "filter criteria" refers to the combination of `pubsub_topic` and
a set of `content_topics`.
The request MAY include filter criteria,
conditional to the selected `filter_subscribe_type`.
If the request contains filter criteria,
it MUST contain a `pubsub_topic`
and the `content_topics` set MUST NOT be empty.
A `WakuMessage` matches filter criteria when its `content_topic` is in the `content_topics` set
A `WakuMessage` matches filter criteria
when its `content_topic` is in the `content_topics` set
and it was published on a matching `pubsub_topic`.
#### Filter Subscribe Types
@@ -147,23 +188,28 @@ The following filter subscribe types are defined:
##### SUBSCRIBER_PING
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBER_PING`
requests that the service node SHOULD indicate if it has any active subscriptions for this client.
A filter client that sends a `FilterSubscribeRequest` with
`filter_subscribe_type` set to `SUBSCRIBER_PING`
requests that the service node SHOULD indicate if it has any active subscriptions
for this client.
The filter client SHOULD exclude any filter criteria from the request.
The filter service node SHOULD respond with a success code if it has any active subscriptions for this client
The filter service node SHOULD respond with a success code
if it has any active subscriptions for this client
or an error code if not.
The filter service node SHOULD ignore any filter criteria in the request.
##### SUBSCRIBE
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBE`
A filter client that sends a `FilterSubscribeRequest` with
`filter_subscribe_type` set to `SUBSCRIBE`
requests that the service node SHOULD push messages matching this filter to the client.
The filter client MUST include the desired filter criteria in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _additional_ filter criteria in a new request.
A client MAY use this request type to _refresh_ an existing subscription
by providing _the same_ filter criteria in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
The filter service node SHOULD respond with a success code
if it successfully honored this request
or an error code if not.
The filter service node SHOULD respond with an error code and discard the request
if the subscribe request does not contain valid filter criteria,
@@ -171,12 +217,17 @@ i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
##### UNSUBSCRIBE
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE`
requests that the service node SHOULD _stop_ pushing messages matching this filter to the client.
The filter client MUST include the filter criteria it desires to unsubscribe from in the request.
A filter client that sends a `FilterSubscribeRequest` with
`filter_subscribe_type` set to `UNSUBSCRIBE`
requests that the service node SHOULD _stop_ pushing messages
matching this filter to the client.
The filter client MUST include the filter criteria
it desires to unsubscribe from in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _a subset of_ the original filter criteria to unsubscribe from in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
by providing _a subset of_ the original filter criteria
to unsubscribe from in a new request.
The filter service node SHOULD respond with a success code
if it successfully honored this request
or an error code if not.
The filter service node SHOULD respond with an error code and discard the request
if the unsubscribe request does not contain valid filter criteria,
@@ -184,8 +235,10 @@ i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
##### UNSUBSCRIBE_ALL
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
requests that the service node SHOULD _stop_ pushing messages matching _any_ filter to the client.
A filter client that sends a `FilterSubscribeRequest` with
`filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
requests that the service node SHOULD _stop_ pushing messages
matching _any_ filter to the client.
The filter client SHOULD exclude any filter criteria from the request.
The filter service node SHOULD remove any existing subscriptions for this client.
It SHOULD respond with a success code if it successfully honored this request
@@ -194,7 +247,8 @@ or an error code if not.
### Filter-Push
A filter client node MUST support the _filter-push_ protocol
to allow filter service nodes to push messages matching registered subscriptions to this client.
to allow filter service nodes to push messages
matching registered subscriptions to this client.
A filter service node SHOULD push all messages
matching the filter criteria in a registered subscription
@@ -205,8 +259,10 @@ This is up to the consumer of the protocol.
If a message push fails,
the filter service node MAY consider the client node to be unreachable.
If a specific filter client node is not reachable from the service node for a period of time,
the filter service node MAY choose to stop pushing messages to the client and remove its subscription.
If a specific filter client node is not reachable from the service node
for a period of time,
the filter service node MAY choose to stop pushing messages to the client and
remove its subscription.
This period is up to the service node implementation.
We consider `1 minute` to be a reasonable default.
@@ -224,25 +280,58 @@ A filter client SHOULD verify that each `MessagePush` it receives
originated from a service node where the client has an active subscription
and that it matches filter criteria belonging to that subscription.
---
---
## Future Work
<!-- Alternative title: Filter-subscriber unlinkability -->
**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->.
Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
**Anonymous filter subscription**:
This feature guarantees that nodes can anonymously subscribe for a message filter
(i.e., without revealing their exact content filter).
As such, no adversary in the `WakuFilter` protocol
would be able to link nodes to their subscribed content filers.
The current version of the `WakuFilter` protocol does not provide anonymity
as the subscribing node has a direct connection to the full node and
explicitly submits its content filter to be notified about the matching messages.
However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication.
That is the subscribing node shall hide all its PII in its filter request
e.g., its IP address.
This can happen by the utilization of a proxy server or by using Tor
<!-- TODO: if nodes have to disclose their PeerIDs
(e.g., for authentication purposes)
when connecting to other nodes in the WakuFilter protocol,
then Tor does not preserve anonymity since it only helps in hiding the IP.
So, the PeerId usage in switches must be investigated further.
Depending on how PeerId is used,
one may be able to link between a subscriber and
its content filter despite hiding the IP address-->.
Note that the current structure of filter requests
i.e., `FilterRPC` does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which
the subscribing node obtains the messages matching a content filter
whereas the full node learns nothing about the content filter as well as
the messages pushed to the subscribing node.
Examples of such 2PC protocols are
[Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.)
and one-way Private Set Intersections (PSI).
## Changelog
### Next
- Added initial threat model and security analysis.
### 2.0.0-beta2
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
- Fix: Ensure contentFilter is a repeated field, on implementation
- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
- Change: Add ability to unsubscribe from filters.
Make `subscribe` an explicit boolean indication.
Edit protobuf field order to be consistent with libp2p.
### 2.0.0-beta1

108
waku/standards/core/12/previous-versions00/filter.md
View File

@@ -14,7 +14,11 @@ contributors:
version: 00
---
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives.
This is a more lightweight version of `WakuRelay`
specifically designed for bandwidth restricted devices.
This is due to the fact that light nodes subscribe to full-nodes and
only receive the messages they desire.
## Content filtering
@@ -41,24 +45,49 @@ It is worth noting that a light node could get by with only using the `store`
protocol to query for a recent time window, provided it is acceptable to do
frequent polling.
## Design Requirements
The effectiveness and reliability of the content filtering service enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
The effectiveness and reliability of the content filtering service
enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes
as the service providers.
To this end, full nodes must feature *high uptime*
(to persistently listen and capture the network messages)
as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
## Security Consideration
Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section.
Note that while using `WakuFilter` allows light nodes to save bandwidth,
it comes with a privacy cost in the sense that they need to disclose their liking
topics to the full nodes to retrieve the relevant messages.
Currently, anonymous subscription is not supported by the `WakuFilter`, however,
potential solutions in this regard are sketched below in [Future Work](#future-work)
section.
### Terminology
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
The term Personally identifiable information (PII)
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and hence count as PII.
## Adversarial Model
Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
Any node running the `WakuFilter` protocol i.e.,
both the subscriber node and the queried node are considered as an adversary.
Furthermore, we consider the adversary as a passive entity
that attempts to collect information from other nodes to conduct an attack but
it does so without violating protocol definitions and instructions.
For example, under the passive adversarial model,
no malicious node intentionally hides the messages matching
to one's subscribed content filter as it is against the description
of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links
between arbitrary pairs of nodes (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
### Protobuf
@@ -86,13 +115,14 @@ message FilterRPC {
#### FilterRPC
A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
Filter protocol is not a request response based protocol but instead a push based system.
A node MUST send all Filter messages (`FilterRequest`, `MessagePush`)
wrapped inside a `FilterRPC` this allows the node handler
to determine how to handle a message as the Waku Filter protocol
is not a request response based protocol but instead a push based system.
The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
matched the message causing it to be pushed.
the `requestId` MUST match the `requestId` of the subscribing `FilterRequest`
whose filters matched the message causing it to be pushed.
#### FilterRequest
@@ -100,7 +130,7 @@ A `FilterRequest` contains an optional topic, zero or more content filters and
a boolean signifying whether to subscribe or unsubscribe to the given filters.
True signifies 'subscribe' and false signifies 'unsubscribe'.
A node that sends the RPC with a filter request and `subscribe` set to 'true'
A node that sends the RPC with a filter request and `subscribe` set to 'true'
requests that the filter node SHOULD notify the light requesting node of messages
matching this filter.
@@ -123,7 +153,8 @@ mechanism is currently planned but underspecified.
#### MessagePush
A filter node that has received a filter request SHOULD push all messages that
match this filter to a light node. These [`WakuMessage`'s](../14/message.md) are likely to come from the
match this filter to a light node. These [`WakuMessage`'s](../14/message.md)
are likely to come from the
`relay` protocol and be kept at the Node, but there MAY be other sources or
protocols where this comes from. This is up to the consumer of the protocol.
@@ -135,25 +166,54 @@ period of time (e.g. a TTL), then the filter node MAY choose to not push these
messages to the node. This period is up to the consumer of the protocol and node
implementation, though a reasonable default is one minute.
---
# Future Work
---
## Future Work
<!-- Alternative title: Filter-subscriber unlinkability -->
**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->.
Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
**Anonymous filter subscription**:
This feature guarantees that nodes can anonymously subscribe for a message filter
(i.e., without revealing their exact content filter).
As such, no adversary in the `WakuFilter` protocol would be able to link nodes
to their subscribed content filers.
The current version of the `WakuFilter` protocol does not provide anonymity
as the subscribing node has a direct connection to the full node and
explicitly submits its content filter to be notified about the matching messages.
However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication.
That is the subscribing node shall hide all its PII in its filter request e.g.,
its IP address.
This can happen by the utilization of a proxy server or by using Tor
<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes)
when connecting to other nodes in the WakuFilter protocol,
then Tor does not preserve anonymity since it only helps in hiding the IP.
So, the PeerId usage in switches must be investigated further.
Depending on how PeerId is used, one may be able to link between a subscriber and
its content filter despite hiding the IP address-->.
Note that the current structure of filter requests i.e.,
`FilterRPC` does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which the subscribing node obtains
the messages matching a content filter whereas the full node learns nothing
about the content filter as well as the messages pushed to the subscribing node.
Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.)
and one-way Private Set Intersections (PSI).
## Changelog
### Next
- Added initial threat model and security analysis.
### 2.0.0-beta2
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
- Fix: Ensure contentFilter is a repeated field, on implementation
- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
- Change: Add ability to unsubscribe from filters.
Make `subscribe` an explicit boolean indication.
Edit protobuf field order to be consistent with libp2p.
### 2.0.0-beta1

317
waku/standards/core/13/store.md
View File

@@ -14,55 +14,72 @@ contributors:
---
## Abstract
This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
stored by other nodes.
It also supports pagination for more efficient querying of historical messages.
This specification explains the `13/WAKU2-STORE` protocol
which enables querying of messages received through the relay protocol and
stored by other nodes.
It also supports pagination for more efficient querying of historical messages.
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
## Terminology
The term PII, Personally Identifiable Information,
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
The term PII, Personally Identifiable Information,
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and hence count as PII.
## Design Requirements
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
“OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
SHOULD provide a complete and full view of message history.
As such, they are required to be *highly available* and
specifically have a *high uptime* to consistently receive and store network messages.
The high uptime requirement makes sure that no message is missed out hence a complete and
intact view of the message history is delivered to the querying nodes.
Nevertheless, in case storage provider nodes cannot afford high availability,
the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
As such, they are required to be *highly available* and
specifically have a *high uptime* to consistently receive and store network messages.
The high uptime requirement makes sure that no message is missed out
hence a complete and intact view of the message history
is delivered to the querying nodes.
Nevertheless, in case storage provider nodes cannot afford high availability,
the querying nodes may retrieve the historical messages from multiple sources
to achieve a full and intact view of the past.
The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](../14/message.md).
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
The concept of `ephemeral` messages introduced in
[`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in
[14/WAKU2-MESSAGE](../14/message.md).
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages
with the `ephemeral` flag set to `true`.
## Adversarial Model
Any peer running the `13/WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary.
Furthermore,
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
it does so without violating protocol definitions and instructions.
As we evolve the protocol,
Any peer running the `13/WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary.
Furthermore,
we currently consider the adversary as a passive entity
that attempts to collect information from other peers to conduct an attack but
it does so without violating protocol definitions and instructions.
As we evolve the protocol,
further adversarial models will be considered.
For example, under the passive adversarial model,
no malicious node hides or
lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol.
For example, under the passive adversarial model,
no malicious node hides or
lies about the history of messages
as it is against the description of the `13/WAKU2-STORE` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
- An adversary that can eavesdrop on communication links
between arbitrary pairs of peers (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
## Wire Specification
Peers communicate with each other using a request / response API.
The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
Peers communicate with each other using a request / response API.
The messages sent are Protobuf RPC messages which are implemented using
[protocol buffers v3](https://developers.google.com/protocol-buffers/).
The following are the specifications of the Protobuf messages.
### Payloads
@@ -118,24 +135,35 @@ message HistoryRPC {
#### Index
To perform pagination,
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts.
To perform pagination,
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol
is associated with a unique `Index` that encapsulates the following parts.
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node.
- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender.
The hash is computed over the concatenation of `contentTopic`
and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
- `receiverTime`: the UNIX time in nanoseconds
at which the `WakuMessage` is received by the receiving node.
- `senderTime`: the UNIX time in nanoseconds
at which the `WakuMessage` is generated by its sender.
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
#### PagingInfo
`PagingInfo` holds the information required for pagination. It consists of the following components.
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery`
`PagingInfo` holds the information required for pagination.
It consists of the following components.
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s
in a `HistoryQuery`
(or retrieved `WakuMessage`s in a `HistoryResponse`).
- `cursor`: holds the `Index` of a `WakuMessage`.
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
- `direction`: indicates the direction of paging
which can be either `FORWARD` or `BACKWARD`.
#### ContentFilter
`ContentFilter` carries the information required for filtering historical messages.
`ContentFilter` carries the information required for filtering historical messages.
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
@@ -143,111 +171,179 @@ each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is ass
RPC call to query historical messages.
- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved.
This field denotes the pubsub topic on which `WakuMessage`s are published.
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
Leaving this field empty means no filter on the pubsub topic of message history is requested.
This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
Leaving this field empty means no filter on the content topic of message history is required.
This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
- The `pubsubTopic` field MUST indicate the pubsub topic
of the historical messages to be retrieved.
This field denotes the pubsub topic on which `WakuMessage`s are published.
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
Leaving this field empty means no filter on the pubsub topic
of message history is requested.
This field SHOULD be left empty in order to retrieve the historical `WakuMessage`
regardless of the pubsub topics on which they are published.
- The `contentFilters` field MUST indicate the list of content filters
based on which the historical messages are to be retrieved.
Leaving this field empty means no filter on the content topic
of message history is required.
This field SHOULD be left empty in order
to retrieve historical `WakuMessage` regardless of their content topics.
- `PagingInfo` holds the information required for pagination.
Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`.
It is RECOMMENDED that the queried node defines a maximum page size internally.
If the querying node leaves the `pageSize` unspecified,
or if the `pageSize` exceeds the maximum page size,
the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
This allows mitigation of long response time for `HistoryQuery`.
In the forward pagination request,
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
(and vise versa for the backward pagination).
Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
Its `pageSize` field indicates the number of `WakuMessage`s
to be included in the corresponding `HistoryResponse`.
It is RECOMMENDED that the queried node defines a maximum page size internally.
If the querying node leaves the `pageSize` unspecified,
or if the `pageSize` exceeds the maximum page size,
the queried node SHOULD auto-paginate the `HistoryResponse`
to no more than the configured maximum page size.
This allows mitigation of long response time for `HistoryQuery`.
In the forward pagination request,
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
the `pageSize` amount of `WakuMessage` whose `Index`
values are larger than the given `cursor`
(and vise versa for the backward pagination).
Note that the `cursor` of a `HistoryQuery` MAY be empty
(e.g., for the initial query), as such, and
depending on whether the `direction` is `BACKWARD` or
`FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned,
respectively.
#### Sorting Messages
The queried node MUST sort the `WakuMessage` based on their `Index`,
where the `senderTime` constitutes the most significant part and the `digest` comes next, and
then perform pagination on the sorted result.
As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one.
Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process.
However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and
The queried node MUST sort the `WakuMessage` based on their `Index`,
where the `senderTime` constitutes the most significant part and
the `digest` comes next, and
then perform pagination on the sorted result.
As such, the retrieved page contains an ordered list of `WakuMessage`
from the oldest messages to the most recent one.
Alternatively, the `receiverTime` (instead of `senderTime`)
MAY be used to sort messages during the paging process.
However, it is RECOMMENDED the use of the `senderTime`
for sorting as it is invariant and
consistent across all the nodes.
This has the benefit of `cursor` reusability i.e.,
a `cursor` obtained from one node can be consistently used to query from another node.
However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and
This has the benefit of `cursor` reusability i.e.,
a `cursor` obtained from one node can be consistently used
to query from another node.
However, this `cursor` reusability does not hold when the `receiverTime` is utilized
as the receiver time is affected by the network delay and
nodes' clock asynchrony.
#### HistoryResponse
RPC call to respond to a HistoryQuery call.
- The `messages` field MUST contain the messages found,
these are [14/WAKU2-MESSAGE](../14/message.md) types.
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries.
The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`).
The `direction` is the same direction as in the corresponding `HistoryQuery`.
In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging).
Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent.
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).
The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different.
- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`.
`NONE` stands for no error.
This is also the default value.
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
- `PagingInfo` holds the paging information based
on which the querying node can resume its further history queries.
The `pageSize` indicates the number of returned Waku messages
(i.e., the number of messages included in the `messages` field of `HistoryResponse`).
The `direction` is the same direction as in the corresponding `HistoryQuery`.
In the forward pagination, the `cursor` holds the `Index` of the last message
in the `HistoryResponse` `messages` (and the first message in the backward paging).
Regardless of the paging direction,
the retrieved `messages` are always sorted in ascending order
based on their timestamp as explained in the [sorting messages](#sorting-messages)section,
that is, from the oldest to the most recent.
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery`
to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).
The `cursor` obtained from one node SHOULD NOT be used in a request to another node
because the result may be different.
- The `error` field contains information about any error that has occurred
while processing the corresponding `HistoryQuery`.
`NONE` stands for no error.
This is also the default value.
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery`
does not match with the `Index` of any of the `WakuMessage`
persisted by the queried node.
## Security Consideration
The main security consideration to take into account while using this protocol is that a querying node have to reveal their content filters of interest to the queried node, hence potentially compromising their privacy.
The main security consideration to take into account
while using this protocol is that a querying node
have to reveal their content filters of interest to the queried node,
hence potentially compromising their privacy.
## Future Work
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md) they are interested in.
As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](../14/message.md).
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries,
as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways:
- **Anonymous query**: This feature guarantees that nodes
can anonymously query historical messages from other nodes i.e.,
without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md)
they are interested in.
As such, no adversary in the `13/WAKU2-STORE` protocol
would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](../14/message.md).
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity
for historical queries,
as the querying node needs to directly connect to another node
in the `13/WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest
to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the request i.e., anonymous communication.
That is the querying node shall hide all its PII in its history request e.g., its IP address.
This can happen by the utilization of a proxy server or by using Tor.
Note that the current structure of historical requests does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve query anonymity.
<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address-->
- By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic,
the queried node learns nothing about the query.
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
<!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included -->
<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).-->
That is the querying node shall hide all its PII in its history request
e.g., its IP address.
This can happen by the utilization of a proxy server or by using Tor.
Note that the current structure of historical requests
does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve query anonymity.
<!-- TODO: if nodes have to disclose their PeerIDs
(e.g., for authentication purposes) when connecting to other nodes
in the store protocol,
then Tor does not preserve anonymity since it only helps in hiding the IP.
So, the PeerId usage in switches must be investigated further.
Depending on how PeerId is used, one may be able to link between a querying node
and its queried topics despite hiding the IP address-->
- By deploying secure 2-party computations in which the querying node
obtains the historical messages of a certain topic,
the queried node learns nothing about the query.
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
<!-- TODO: add a reference for PSIs? -->
<!-- TODO: more techniques to be included -->
<!-- TODO: Censorship resistant:
this is about a node that hides the historical messages from other nodes.
This attack is not included in the specs
since it does not fit the passive adversarial model
(the attacker needs to deviate from the store protocol).-->
- **Robust and verifiable timestamps**: Messages timestamp is a way to show that the message existed prior to some point in time.
- **Robust and verifiable timestamps**:
Messages timestamp is a way to show that the message existed
prior to some point in time.
However, the lack of timestamp verifiability can create room for a range of attacks,
including injecting messages with invalid timestamps pointing to the far future.
including injecting messages with invalid timestamps pointing to the far future.
To better understand the attack,
consider a store node whose current clock shows `2021-01-01 00:00:30` (and assume all the other nodes have a synchronized clocks +-20seconds).
consider a store node whose current clock shows `2021-01-01 00:00:30`
(and assume all the other nodes have a synchronized clocks +-20seconds).
The store node already has a list of messages,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
that are sorted based on their timestamp.
An attacker sends a message with an arbitrary large timestamp e.g.,
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
The store node places `m'` at the end of the list,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30)`.
```text
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),(m',2021-01-01 10:00:30).
```
Now another message arrives with a valid timestamp e.g.,
`(m11, 2021-01-01 00:00:45)`.
However, since its timestamp precedes the malicious message `m'`,
it gets placed before `m'` in the list i.e.,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
```text
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30).
```
In fact, for the next 10 hours,
`m'` will always be considered as the most recent message and
served as the last message to the querying nodes irrespective of how many other messages arrive afterward.
served as the last message to the querying nodes irrespective
of how many other messages arrive afterward.
A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp.
A robust and verifiable timestamp allows the receiver of a message
to verify that a message has been generated prior to the claimed timestamp.
One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
block height in Blockchain-based timestamps.
That is, messages contain the most recent block height perceived by their senders at the time of message generation.
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
seconds (e.g., in Ethereum 2.0) from the time of origination.
block height in Blockchain-based timestamps.
That is, messages contain the most recent block height
perceived by their senders at the time of message generation.
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
seconds (e.g., in Ethereum 2.0) from the time of origination.
## Copyright
@@ -255,7 +351,8 @@ Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [14/WAKU2-MESSAGE](../14/message.md)
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
3. [11/WAKU2-RELAY](../11/relay.md)
4. [Open timestamps](https://opentimestamps.org/)
4. [Open timestamps](https://opentimestamps.org/)

124
waku/standards/core/14/message.md
View File

@@ -17,11 +17,14 @@ contributors:
## Abstract
Waku v2 is a family of modular peer-to-peer protocols for secure communication.
These protocols are designed to be secure, privacy-preserving, and censorship-resistant and can run in resource-restricted environments.
At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and adds capabilities.
The present document specifies the Waku v2 message format, a way to encapsulate the messages sent with specific information security goals, and Whisper/Waku v1 backward compatibility.
These protocols are designed to be secure, privacy-preserving,
and censorship-resistant and can run in resource-restricted environments.
At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and
adds capabilities.
The present document specifies the Waku v2 message format,
a way to encapsulate the messages sent with specific information security goals,
and Whisper/Waku v1 backward compatibility.
## Motivation
@@ -35,38 +38,48 @@ When sending messages over Waku, there are multiple requirements:
This specification attempts to provide for these various requirements.
## Semantics
### Waku Message
A Waku message is constituted by the combination of data payload and attributes that, for example, a *publisher* sends to a *topic* and is eventually delivered to *subscribers*.
A Waku message is constituted by the combination of data payload and
attributes that, for example, a *publisher* sends to a *topic* and
is eventually delivered to *subscribers*.
Waku message attributes are key-value pairs of metadata associated with a message.
And the message data payload is the part of the transmitted Waku message that is the actual message information.
Waku message attributes are key-value pairs of metadata associated with a message.
And the message data payload is the part of the transmitted Waku message
that is the actual message information.
The data payload is also treated as a Waku message attribute for convenience.
### Message Attributes
* The `payload` attribute MUST contain the message data payload to be sent.
- The `payload` attribute MUST contain the message data payload to be sent.
* The `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering,
- The `content_topic` attribute MUST specify a string identifier
that can be used for content-based filtering,
as described in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
* The `meta` attribute, if present, contains an arbitrary application-specific variable-length byte array with a maximum length limit of 64 bytes.
This attribute can be utilized to convey supplementary details to various Waku protocols, thereby enabling customized processing based on its contents.
- The `meta` attribute, if present,
contains an arbitrary application-specific variable-length byte array
with a maximum length limit of 64 bytes.
This attribute can be utilized to convey supplementary details
to various Waku protocols,
thereby enabling customized processing based on its contents.
* The `version` attribute, if present, contains a version number to discriminate different types of payload encryption.
- The `version` attribute, if present,
contains a version number to discriminate different types of payload encryption.
If omitted, the value SHOULD be interpreted as version 0.
* The `timestamp` attribute, if present, signifies the time at which the message was generated by its sender.
- The `timestamp` attribute, if present,
signifies the time at which the message was generated by its sender.
This attribute MAY contain the Unix epoch time in nanoseconds.
If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
* The `ephemeral` attribute, if present, signifies the transient nature of the message.
- The `ephemeral` attribute, if present, signifies the transient nature of the message.
For example, an ephemeral message SHOULD not be persisted by the Waku network.
If this attribute is set to `true`, the message SHOULD be interpreted as ephemeral.
If, instead, the attribute is omitted or set to `false`, the message SHOULD be interpreted as non-ephemeral.
If, instead, the attribute is omitted or set to `false`,
the message SHOULD be interpreted as non-ephemeral.
## Wire Format
@@ -87,14 +100,16 @@ message WakuMessage {
An example proto file following this specification can be found [here (vacp2p/waku)](https://github.com/vacp2p/waku/blob/main/waku/message/v1/message.proto).
## Payload encryption
The Waku message payload MAY be encrypted.
The message `version` attribute indicates the schema used to encrypt the payload data.
The message `version` attribute indicates
the schema used to encrypt the payload data.
- **Version 0:**
The payload SHOULD be interpreted as unencrypted; additionally, it CAN indicate that the message payload has been encrypted at the application layer.
The payload SHOULD be interpreted as unencrypted; additionally,
it CAN indicate that the message payload has been encrypted
at the application layer.
- **Version 1:**
The payload SHOULD be encrypted using Waku v1 payload encryption specified in [26/WAKU-PAYLOAD](../../application/26/payload.md).
@@ -109,36 +124,49 @@ Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
Any `version` value not included in this list is reserved for future specification.
And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
## Whisper/Waku v1 envelope compatibility
Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
* Whisper/Waku v1 `topic` field SHOULD be mapped to Waku v2 message's `content_topic` attribute.
* Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
- Whisper/Waku v1 `topic` field
SHOULD be mapped to Waku v2 message's `content_topic` attribute.
- Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
Waku v2 implements a pub/sub messaging pattern over libp2p.
This makes redundant some Whisper/Waku v1 envelope fields (e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
This makes redundant some Whisper/Waku v1 envelope fields
(e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
## Deterministic message hashing
In Protocol Buffers v3, the deterministic serialization is not canonical across the different implementations and languages.
It is also unstable across different builds with schema changes due to unknown fields.
In Protocol Buffers v3,
the deterministic serialization is not canonical across the different implementations
and languages.
It is also unstable across different builds with schema changes due to unknown fields.
To overcome this interoperability limitation, a Waku v2 message's hash MUST be computed following this schema:
To overcome this interoperability limitation,
a Waku v2 message's hash MUST be computed following this schema:
```
```js
message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp))
```
If an optional attribute, such as `meta`, is absent, the concatenation of attributes SHOULD exclude it. This recommendation is made to ensure that the concatenation process proceeds smoothly when certain attributes are missing and to maintain backward compatibility.
If an optional attribute, such as `meta`, is absent,
the concatenation of attributes SHOULD exclude it.
This recommendation is made to ensure that the concatenation process proceeds smoothly
when certain attributes are missing and to maintain backward compatibility.
This hashing schema is deemed appropriate for use cases where a cross-implementation deterministic hash is needed, such as message deduplication and integrity validation. The collision probability offered by this hashing schema can be considered negligible. This is due to the deterministic concatenation order of the message attributes, coupled with using a SHA-2 (256-bit) hashing algorithm.
This hashing schema is deemed appropriate for use cases
where a cross-implementation deterministic hash is needed,
such as message deduplication and integrity validation.
The collision probability offered by this hashing schema can be considered negligible.
This is due to the deterministic concatenation order of the message attributes,
coupled with using a SHA-2 (256-bit) hashing algorithm.
### Test vectors
Waku message hash computation (`meta` size of 12 bytes):
```
```js
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
@@ -150,7 +178,7 @@ message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe0
Waku message hash computation (`meta` size of 64 bytes):
```
```js
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
@@ -161,7 +189,8 @@ message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b2
```
Waku message hash computation (`meta` attribute not present):
```
```js
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
@@ -172,7 +201,8 @@ message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd
```
Waku message hash computation (`payload` length 0):
```
```js
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = []
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
@@ -186,29 +216,39 @@ message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de
### Confidentiality, integrity, and authenticity
The level of confidentiality, integrity, and authenticity of the Waku message payload is discretionary.
Accordingly, the application layer shall utilize the encryption and signature schemes supported by Waku v2 to meet the application-specific privacy needs.
The level of confidentiality, integrity, and
authenticity of the Waku message payload is discretionary.
Accordingly, the application layer shall utilize the encryption and
signature schemes supported by Waku v2 to meet the application-specific privacy needs.
### Reliability of the `timestamp` attribute
The Waku message `timestamp` attribute is set by the sender.
Therefore, because message timestamps aren’t independently verified, this attribute is prone to exploitation and misuse.
Therefore, because message timestamps aren’t independently verified,
this attribute is prone to exploitation and misuse.
It should not solely be relied upon for operations such as message ordering.
For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message to a high value so that it always shows up as the most recent message in a chat application.
Applications using Waku messages’ `timestamp` attribute are recommended to use additional methods for more robust message ordering.
An example of how to deal with message ordering against adversarial message timestamps can be found in the Status protocol, see [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering).
For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message
to a high value so that it always shows up as the most recent message
in a chat application.
Applications using Waku messages’ `timestamp` attribute
are recommended to use additional methods for more robust message ordering.
An example of how to deal with message ordering against adversarial message timestamps
can be found in the Status protocol,
see [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering).
### Reliability of the `ephemeral` attribute
The Waku message `ephemeral` attribute is set by the sender.
Since there is currently no incentive mechanism for network participants to behave correctly, this attribute is inherently insecure.
A malicious actor can tamper with the value of a Waku message’s `ephemeral` attribute, and the receiver would not be able to verify the integrity of the message.
Since there is currently no incentive mechanism
for network participants to behave correctly,
this attribute is inherently insecure.
A malicious actor can tamper with the value of a Waku message’s `ephemeral` attribute,
and the receiver would not be able to verify the integrity of the message.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [6/WAKU1](../../legacy/6/waku1.md)

1
waku/standards/core/15/bridge.md
View File

@@ -45,6 +45,7 @@ fields. Next, before posting on the network, the bridge MUST set a new expiry
and ttl and do the PoW nonce calculation.
### Security Considerations
As mentioned above, a bridge will be posting new Waku v1 envelopes, which
requires doing the PoW nonce calculation.

288
waku/standards/core/17/rln-relay.md
View File

@@ -13,65 +13,87 @@ contributors:
---
## Abstract
This specification describes the `17/WAKU2-RLN-RELAY` protocol,
which is an extension of [`11/WAKU2-RELAY`](../11/relay.md) to provide spam protection using [Rate Limiting Nullifiers (RLN)](../../../../vac/32/rln-v1.md).
The security objective is to contain spam activity in the (64/WAKU-NETWORK)[] by enforcing a global messaging rate to all the peers.
Peers that violate the messaging rate are considered spammers and
This specification describes the `17/WAKU2-RLN-RELAY` protocol,
which is an extension of [`11/WAKU2-RELAY`](../11/relay.md)
to provide spam protection using [Rate Limiting Nullifiers (RLN)](../../../../vac/32/rln-v1.md).
The security objective is to contain spam activity in the (64/WAKU-NETWORK)[]
by enforcing a global messaging rate to all the peers.
Peers that violate the messaging rate are considered spammers and
their message is considered spam.
Spammers are also financially punished and removed from the system.
Spammers are also financially punished and removed from the system.
## Motivation
In open and anonymous p2p messaging networks,
one big problem is spam resistance.
In open and anonymous p2p messaging networks,
one big problem is spam resistance.
Existing solutions, such as Whisper’s proof of work,
are computationally expensive hence not suitable for resource-limited nodes.
Other reputation-based approaches might not be desirable,
are computationally expensive hence not suitable for resource-limited nodes.
Other reputation-based approaches might not be desirable,
due to issues around arbitrary exclusion and privacy.
We augment the [`11/WAKU2-RELAY`](../11/relay.md) protocol with a novel construct of [RLN](../../../../vac/32/rln-v1.md) to enable an efficient economic spam prevention mechanism that can be run in resource-constrained environments.
We augment the [`11/WAKU2-RELAY`](../11/relay.md) protocol
with a novel construct of [RLN](../../../../vac/32/rln-v1.md)
to enable an efficient economic spam prevention mechanism
that can be run in resource-constrained environments.
## Specification
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”,
“SHOULD NOT”, “RECOMMENDED”, “MAY”, and
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
### Flow
The messaging rate is defined by the `period` which indicates how many messages can be sent in a given period.
We define an `epoch` as $\lceil$ `unix_time` / `period` $\rceil$.
For example, if `unix_time` is `1644810116` and we set `period` to `30`,
The messaging rate is defined by the `period`
which indicates how many messages can be sent in a given period.
We define an `epoch` as $\lceil$ `unix_time` / `period` $\rceil$.
For example, if `unix_time` is `1644810116` and we set `period` to `30`,
then `epoch` is $\lceil$ `(unix_time/period)` $\rceil$ `= 54827003`.
> **NOTE:** The `epoch` refers to the epoch in RLN and not Unix epoch.
This means a message can only be sent every period, where the `period` is up to the application.
This means a message can only be sent every period,
where the `period` is up to the application.
See section [Recommended System Parameters](#recommended-system-parameters) for the RECOMMENDED method to set a sensible `period` value depending on the application.
Peers subscribed to a spam-protected `pubsubTopic` are only allowed to send one message per `epoch`.
The higher-level layers adopting `17/WAKU2-RLN-RELAY` MAY choose to enforce the messaging rate for `WakuMessages` with a specific `contentTopic` published on a `pubsubTopic`.
See section [Recommended System Parameters](#recommended-system-parameters)
for the RECOMMENDED method to set a sensible `period` value depending on the application.
Peers subscribed to a spam-protected `pubsubTopic`
are only allowed to send one message per `epoch`.
The higher-level layers adopting `17/WAKU2-RLN-RELAY`
MAY choose to enforce the messaging rate for `WakuMessages`
with a specific `contentTopic` published on a `pubsubTopic`.
#### Setup and Registration
A `pubsubTopic` that is spam-protected requires subscribed peers to form a [RLN group](../../../../vac/32/rln-v1.md).
- Peers MUST be registered to the RLN group to be able to publish messages.
- Registration MAY be moderated through a smart contract deployed on the Ethereum blockchain.
- Registration MAY be moderated through a smart contract
deployed on the Ethereum blockchain.
Each peer has an [RLN key pair](../../../../vac/32/rln-v1.md) denoted by `sk`
and `pk`.
Each peer has an [RLN key pair](../../../../vac/32/rln-v1.md) denoted by `sk` and `pk`.
- The secret key `sk` is secret data and MUST be persisted securely by the peer.
- The state of the membership contract SHOULD contain a list of all registered members' public identity keys i.e.,
`pk`s.
- The state of the membership contract
SHOULD contain a list of all registered members' public identity keys i.e., `pk`s.
For registration, a peer MUST create a transaction to invoke the registration function on the contract,
which registers its `pk` in the RLN group.
- The transaction MUST transfer additional tokens to the contract to be staked.
For registration,
a peer MUST create a transaction to invoke the registration function on the contract,
which registers its `pk` in the RLN group.
- The transaction MUST transfer additional tokens to the contract to be staked.
This amount is denoted by `staked_fund` and is a system parameter.
The peer who has the secret key `sk` associated with a registered `pk` would be able to withdraw a portion `reward_portion` of the staked fund by providing valid proof.
The peer who has the secret key `sk` associated with a registered `pk`
would be able to withdraw a portion `reward_portion`
of the staked fund by providing valid proof.
`reward_portion` is also a system parameter.
> **NOTE:** Initially `sk` is only known to its owning peer however,
it may get exposed to other peers in case the owner attempts spamming the system i.e.,
sending more than one message per `epoch`.
it may get exposed to other peers in case the owner attempts spamming the system
i.e., sending more than one message per `epoch`.
An overview of registration is illustrated in Figure 1.
@@ -79,28 +101,33 @@ An overview of registration is illustrated in Figure 1.
#### Publishing
To publish at a given `epoch`,
the publishing peer proceeds based on the regular [`11/WAKU2-RELAY`](../11/relay.md) protocol.
However, to protect against spamming, each `WakuMessage`
(which is wrapped inside the `data` field of a PubSub message)
MUST carry a [`RateLimitProof`](##RateLimitProof) with the following fields.
Section [Payload](#payloads) covers the details about the type and encoding of these fields.
To publish at a given `epoch`, the publishing peer proceeds
based on the regular [`11/WAKU2-RELAY`](../11/relay.md) protocol.
However, to protect against spamming, each `WakuMessage`
(which is wrapped inside the `data` field of a PubSub message)
MUST carry a [`RateLimitProof`](#ratelimitproof) with the following fields.
Section [Payload](#payloads) covers the details about the type and
encoding of these fields.
- The `merkle_root` contains the root of the Merkle tree.
- The `epoch` represents the current epoch.
- The `nullifier` is an internal nullifier acting as a fingerprint that allows specifying whether two messages are published by the same peer during the same `epoch`.
- The `nullifier` is an internal nullifier acting as a fingerprint
that allows specifying whether two messages are published by the same peer
during the same `epoch`.
- The `nullifier` is a deterministic value derived from `sk` and
`epoch` therefore any two messages issued by the same peer
(i.e., using the same `sk`) for the same `epoch` are guaranteed to have identical `nullifier`s.
- The `share_x` and `share_y` can be seen as partial disclosure of peer's `sk` for the intended `epoch`.
(i.e., using the same `sk`)
for the same `epoch` are guaranteed to have identical `nullifier`s.
- The `share_x` and
`share_y` can be seen as partial disclosure of peer's `sk` for the intended `epoch`.
They are derived deterministically from peer's `sk` and
current `epoch` using [Shamir secret sharing scheme](../../../../vac/32/rln-v1.md).
current `epoch` using [Shamir secret sharing scheme](../../../../vac/32/rln-v1.md).
If a peer discloses more than one such pair (`share_x`, `share_y`) for the same `epoch`,
it would allow full disclosure of its `sk` and
If a peer discloses more than one such pair (`share_x`, `share_y`) for the same `epoch`,
it would allow full disclosure of its `sk` and
hence get access to its staked fund in the membership contract.
- The `proof` field is a zero-knowledge proof signifying that:
- The `proof` field is a zero-knowledge proof signifying that:
1. The message owner is the current member of the group i.e.,
the peer's identity commitment key, `pk`,
@@ -108,73 +135,108 @@ is part of the membership group Merkle tree with the root `merkle_root`.
2. `share_x` and `share_y` are correctly computed.
3. The `nullifier` is constructed correctly.
For more details about the proof generation check [RLN](../../../../vac/32/rln-v1.md)
The proof generation relies on the knowledge of two pieces of private information i.e., `sk` and `authPath`.
The `authPath` is a subset of Merkle tree nodes by which a peer can prove the inclusion of its `pk` in the group. <!-- TODO refer to RLN RFC for authPath def -->
The proof generation relies on the knowledge of two pieces of private information
i.e., `sk` and `authPath`.
The `authPath` is a subset of Merkle tree nodes
by which a peer can prove the inclusion of its `pk` in the group.
<!-- TODO refer to RLN RFC for authPath def -->
The proof generation also requires a set of public inputs which are:
the Merkle tree root `merkle_root`, the current `epoch`, and
the message for which the proof is going to be generated.
In `17/WAKU2-RLN-RELAY`, the message is the concatenation of `WakuMessage`'s `payload` filed and
its `contentTopic` i.e., `payload||contentTopic`.
the message for which the proof is going to be generated.
In `17/WAKU2-RLN-RELAY`,
the message is the concatenation of `WakuMessage`'s `payload` filed and
its `contentTopic` i.e., `payload||contentTopic`.
#### Group Synchronization
Proof generation relies on the knowledge of Merkle tree root `merkle_root` and `authPath` which both require access to the membership Merkle tree.
Getting access to the Merkle tree can be done in various ways:
Proof generation relies on the knowledge of Merkle tree root `merkle_root` and
`authPath` which both require access to the membership Merkle tree.
Getting access to the Merkle tree can be done in various ways:
1. Peers construct the tree locally.
This can be done by listening to the registration and
This can be done by listening to the registration and
deletion events emitted by the membership contract.
Peers MUST update the local Merkle tree on a per-block basis.
This is discussed further in the [Merkle Root Validation](#merkle-root-validation) section.
This is discussed further
in the [Merkle Root Validation](#merkle-root-validation) section.
2. For synchronizing the state of slashed `pk`s,
disseminate such information through a `pubsubTopic` to which all peers are subscribed.
disseminate such information through a `pubsubTopic` to which all peers are subscribed.
A deletion transaction SHOULD occur on the membership contract.
The benefit of an off-chain slashing is that it allows real-time removal of spammers as opposed to on-chain slashing in which peers get informed with a delay,
The benefit of an off-chain slashing
is that it allows real-time removal of spammers as opposed to on-chain slashing
in which peers get informed with a delay,
where the delay is due to mining the slashing transaction.
For the group synchronization,
one important security consideration is that peers MUST make sure they always use the most recent Merkle tree root in their proof generation.
The reason is that using an old root can allow inference about the index of the user's `pk` in the membership tree hence compromising user privacy and breaking message unlinkability.
For the group synchronization,
one important security consideration is that peers MUST make sure they always use
the most recent Merkle tree root in their proof generation.
The reason is that using an old root can allow inference
about the index of the user's `pk` in the membership tree
hence compromising user privacy and breaking message unlinkability.
#### Routing
Upon the receipt of a PubSub message via [`11/WAKU2-RELAY`](../11/relay.md) protocol,
the routing peer parses the `data` field as a `WakuMessage` and gets access to the `RateLimitProof` field.
Upon the receipt of a PubSub message via [`11/WAKU2-RELAY`](../11/relay.md) protocol,
the routing peer parses the `data` field as a `WakuMessage` and
gets access to the `RateLimitProof` field.
The peer then validates the `RateLimitProof` as explained next.
##### Epoch Validation
If the `epoch` attached to the `WakuMessage` is more than `max_epoch_gap`,
apart from the routing peer's current `epoch`,
then the `WakuMessage` MUST be discarded and considered invalid.
This is to prevent a newly registered peer from spamming the system by messaging for all the past epochs.
`max_epoch_gap` is a system parameter for which we provide some recommendations in section [Recommended System Parameters](#recommended-system-parameters).
This is to prevent a newly registered peer from spamming the system
by messaging for all the past epochs.
`max_epoch_gap` is a system parameter
for which we provide some recommendations in section [Recommended System Parameters](#recommended-system-parameters).
##### Merkle Root Validation
The routing peers MUST check whether the provided Merkle root in the `RateLimitProof` is valid.
It can do so by maintaining a local set of valid Merkle roots,
The routing peers MUST check whether the provided Merkle root
in the `RateLimitProof` is valid.
It can do so by maintaining a local set of valid Merkle roots,
which consist of `acceptable_root_window_size` past roots.
These roots refer to the final state of the Merkle tree after a whole block consisting of group changes is processed.
These roots refer to the final state of the Merkle tree
after a whole block consisting of group changes is processed.
The Merkle roots are updated on a per-block basis instead of a per-event basis.
This is done because if Merkle roots are updated on a per-event basis, some peers could send messages with a root that refers to a Merkle tree state that might get invalidated while the message is still propagating in the network, due to many registrations happening during this time frame.
By updating roots on a per-block basis instead, we will have only one root update per-block processed, regardless on how many registrations happened in a block, and peers will be able to successfully propagate messages in a time frame corresponding to roughly the size of the roots window times the block mining time.
This is done because if Merkle roots are updated on a per-event basis,
some peers could send messages with a root that refers to a Merkle tree state
that might get invalidated while the message is still propagating in the network,
due to many registrations happening during this time frame.
By updating roots on a per-block basis instead,
we will have only one root update per-block processed,
regardless on how many registrations happened in a block, and
peers will be able to successfully propagate messages in a time frame
corresponding to roughly the size of the roots window times the block mining time.
Atomic processing of the blocks are necessary so that even if the peer is unable to process one event, the previous roots remain valid, and can be used to generate valid RateLimitProof's.
Atomic processing of the blocks are necessary
so that even if the peer is unable to process one event,
the previous roots remain valid, and can be used to generate valid RateLimitProof's.
This also allows peers which are not well connected to the network to be able to send messages, accounting for network delay.
This network delay is related to the nature of asynchronous network conditions, which means that peers see membership changes asynchronously, and therefore may have differing local Merkle trees.
See [Recommended System Parameters](#recommended-system-parameters) on choosing an appropriate `acceptable_root_window_size`.
This also allows peers which are not well connected to the network
to be able to send messages, accounting for network delay.
This network delay is related to the nature of asynchronous network conditions,
which means that peers see membership changes asynchronously, and
therefore may have differing local Merkle trees.
See [Recommended System Parameters](#recommended-system-parameters)
on choosing an appropriate `acceptable_root_window_size`.
##### Proof Verification
The routing peers MUST check whether the zero-knowledge proof `proof` is valid.
It does so by running the zk verification algorithm as explained in [RLN](../../../../vac/32/rln-v1.md).
If `proof` is invalid then the message MUST be discarded.
It does so by running the zk verification algorithm as explained in [RLN](../../../../vac/32/rln-v1.md).
If `proof` is invalid then the message MUST be discarded.
##### Spam detection
To enable local spam detection and slashing,
routing peers MUST record the `nullifier`, `share_x`, and `share_y`
of incoming messages which are not discarded i.e., not found spam or with invalid proof or epoch.
To spot spam messages, the peer checks whether a message with an identical `nullifier` has already been relayed.
To enable local spam detection and slashing,
routing peers MUST record the `nullifier`, `share_x`, and `share_y`
of incoming messages which are not discarded i.e.,
not found spam or with invalid proof or epoch.
To spot spam messages, the peer checks whether a message
with an identical `nullifier` has already been relayed.
1. If such a message exists and its `share_x` and `share_y`
components are different from the incoming message, then slashing takes place.
@@ -183,9 +245,11 @@ of the new message and the `share'_x` and `share'_y`
of the old record to reconstruct the `sk` of the message owner.
The `sk` then MAY be used to delete the spammer from the group and
withdraw a portion `reward_portion` of its staked funds.
3. If the `share_x` and `share_y` fields of the previously relayed message are identical to the incoming message,
2. If the `share_x` and
`share_y` fields of the previously relayed message are identical
to the incoming message,
then the message is a duplicate and MUST be discarded.
4. If none is found, then the message gets relayed.
3. If none is found, then the message gets relayed.
An overview of the routing procedure and slashing is provided in Figure 2.
@@ -196,9 +260,10 @@ An overview of the routing procedure and slashing is provided in Figure 2.
### Payloads
Payloads are protobuf messages implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
Nodes MAY extend the [14/WAKU2-MESSAGE](../14/message.md) with a `rate_limit_proof` field to indicate that their message is not spam.
Nodes MAY extend the [14/WAKU2-MESSAGE](../14/message.md)
with a `rate_limit_proof` field to indicate that their message is not spam.
```diff
```diff
syntax = "proto3";
@@ -221,22 +286,27 @@ message WakuMessage {
}
```
#### WakuMessage
`rate_limit_proof` holds the information required to prove that the message owner has not exceeded the message rate limit.
`rate_limit_proof` holds the information required to prove that the message owner
has not exceeded the message rate limit.
#### RateLimitProof
Below is the description of the fields of `RateLimitProof` and their types.
| Parameter | Type | Description |
| Parameter | Type | Description |
| ----: | ----------- | ----------- |
| `proof` | array of 256 bytes uncompressed or 128 bytes compressed | the zkSNARK proof as explained in the [Publishing process](##Publishing) |
| `proof` | array of 256 bytes uncompressed or 128 bytes compressed | the zkSNARK proof as explained in the [Publishing process](#publishing) |
| `merkle_root` | array of 32 bytes in little-endian order | the root of membership group Merkle tree at the time of publishing the message |
| `share_x` and `share_y`| array of 32 bytes each | Shamir secret shares of the user's secret identity key `sk` . `share_x` is the Poseidon hash of the `WakuMessage`'s `payload` concatenated with its `contentTopic` . `share_y` is calculated using [Shamir secret sharing scheme](../../../../vac/32/rln-v1.md) |
| `nullifier` | array of 32 bytes | internal nullifier derived from `epoch` and peer's `sk` as explained in [RLN construct](../../../../vac/32/rln-v1.md)|
### Recommended System Parameters
The system parameters are summarized in the following table, and the RECOMMENDED values for a subset of them are presented next.
The system parameters are summarized in the following table,
and the RECOMMENDED values for a subset of them are presented next.
| Parameter | Description |
| ----: |----------- |
@@ -247,35 +317,55 @@ The system parameters are summarized in the following table, and the RECOMMENDED
| `acceptable_root_window_size` | The maximum number of past Merkle roots to store |
#### Epoch Length
A sensible value for the `period` depends on the application for which the spam protection is going to be used.
For example, while the `period` of `1` second i.e.,
messaging rate of `1` per second, might be acceptable for a chat application,
A sensible value for the `period` depends on the application
for which the spam protection is going to be used.
For example, while the `period` of `1` second i.e.,
messaging rate of `1` per second, might be acceptable for a chat application,
might be too low for communication among Ethereum network validators.
One should look at the desired throughput of the application to decide on a proper `period` value.
One should look at the desired throughput of the application
to decide on a proper `period` value.
#### Maximum Epoch Gap
We discussed in the [Routing](#routing) section that the gap between the epoch observed by the routing peer and
the one attached to the incoming message should not exceed a threshold denoted by `max_epoch_gap`.
We discussed in the [Routing](#routing) section that the gap between the epoch
observed by the routing peer and
the one attached to the incoming message
should not exceed a threshold denoted by `max_epoch_gap`.
The value of `max_epoch_gap` can be measured based on the following factors.
- Network transmission delay `Network_Delay`: the maximum time that it takes for a message to be fully disseminated in the GossipSub network.
- Clock asynchrony `Clock_Asynchrony`: The maximum difference between the Unix epoch clocks perceived by network peers which can be due to clock drifts.
- Network transmission delay `Network_Delay`:
the maximum time that it takes for a message to be fully disseminated
in the GossipSub network.
- Clock asynchrony `Clock_Asynchrony`:
The maximum difference between the Unix epoch clocks perceived
by network peers which can be due to clock drifts.
With a reasonable approximation of the preceding values, one can set `max_epoch_gap` as
`max_epoch_gap` $= \lceil \frac{\text{Network Delay} + \text{Clock Asynchrony}}{\text{Epoch Length}}\rceil$ where `period` is the length of the `epoch` in seconds.
`Network_Delay` and `Clock_Asynchrony` MUST have the same resolution as `period` .
By this formulation, `max_epoch_gap` indeed measures the maximum number of `epoch`s that can elapse since a message gets routed from its origin to all the other peers in the network.
With a reasonable approximation of the preceding values,
one can set `max_epoch_gap` as
`acceptable_root_window_size` depends upon the underlying chain's average blocktime, `block_time`
`max_epoch_gap`
$= \lceil \frac{\text{Network Delay} + \text{Clock Asynchrony}}{\text{Epoch Length}}\rceil$
where `period` is the length of the `epoch` in seconds.
`Network_Delay` and `Clock_Asynchrony` MUST have the same resolution as `period`.
By this formulation, `max_epoch_gap` indeed measures the maximum number of `epoch`s
that can elapse since a message gets routed from its origin
to all the other peers in the network.
`acceptable_root_window_size` depends upon the underlying chain's average blocktime,
`block_time`
The lower bound for the `acceptable_root_window_size` SHOULD be set as $acceptable_root_window_size=(Network_Delay)/block_time$
`Network_Delay` MUST have the same resolution as `block_time`.
By this formulation, `acceptable_root_window_size` will provide a lower bound of how many roots can be acceptable by a routing peer.
By this formulation,
`acceptable_root_window_size` will provide a lower bound
of how many roots can be acceptable by a routing peer.
The `acceptable_root_window_size` should indicate how many blocks may have been mined during the time it takes for a peer to receive a message.
This formula represents a lower bound of the number of acceptable roots.
The `acceptable_root_window_size` should indicate how many blocks may have been mined
during the time it takes for a peer to receive a message.
This formula represents a lower bound of the number of acceptable roots.
## Copyright

20
waku/standards/core/19/lightpush.md
View File

@@ -13,8 +13,11 @@ contributors:
## Motivation and Goals
Light nodes with short connection windows and limited bandwidth wish to publish messages into the Waku network.
Additionally, there is sometimes a need for confirmation that a message has been received "by the network"
Light nodes with short connection windows and
limited bandwidth wish to publish messages into the Waku network.
Additionally,
there is sometimes a need for confirmation
that a message has been received "by the network"
(here, at least one node).
`19/WAKU2-LIGHTPUSH` is a request/response protocol for this.
@@ -45,13 +48,18 @@ message PushRPC {
### Message Relaying
Nodes that respond to `PushRequests` MUST either
relay the encapsulated message via [11/WAKU2-RELAY](../11/relay.md) protocol on the specified `pubsub_topic`,
or forward the `PushRequest` via 19/LIGHTPUSH on a [WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md) stem.
If they are unable to do so for some reason, they SHOULD return an error code in `PushResponse`.
relay the encapsulated message via [11/WAKU2-RELAY](../11/relay.md) protocol
on the specified `pubsub_topic`,
or forward the `PushRequest` via 19/LIGHTPUSH on a [WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md)
stem.
If they are unable to do so for some reason,
they SHOULD return an error code in `PushResponse`.
## Security Considerations
Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions.
Since this can introduce an amplification factor,
it is RECOMMENDED for the node relaying to the rest of the network
to take extra precautions.
This can be done by rate limiting via [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
Note that the above is currently not fully implemented.

153
waku/standards/core/33/discv5.md
View File

@@ -9,87 +9,125 @@ contributors:
## Abstract
`33/WAKU2-DISCV5` specifies a modified version of [Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) as a means for ambient node discovery.
[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network for establishing a decentralized network of interconnected Waku2 nodes.
In its current version, the `33/WAKU2-DISCV5` discovery network is isolated from the Ethereum Discovery v5 network.
Isolation improves discovery efficiency, which is especially significant with a low number of Waku nodes compared to the total number of Ethereum nodes.
`33/WAKU2-DISCV5` specifies a modified version of
[Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
as a means for ambient node discovery.
[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network
for establishing a decentralized network of interconnected Waku2 nodes.
In its current version,
the `33/WAKU2-DISCV5` discovery network
is isolated from the Ethereum Discovery v5 network.
Isolation improves discovery efficiency,
which is especially significant with a low number of Waku nodes
compared to the total number of Ethereum nodes.
## Disclaimer
This version of `33/WAKU2-DISCV5` has a focus on timely deployment of an efficient discovery method for [10/WAKU2](../10/waku2.md).
This version of `33/WAKU2-DISCV5` has a focus on timely deployment
of an efficient discovery method for [10/WAKU2](../10/waku2.md).
Establishing a separate discovery network is in line with this focus.
However, we are aware of potential resilience problems (see section on security considerations) and are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
However, we are aware of potential resilience problems
(see section on security considerations) and
are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
and researching hybrid approaches.
## Background and Rationale
[11/WAKU2-RELAY](../11/relay.md) assumes the existence of a network of Waku2 nodes.
For establishing and growing this network, new nodes trying to join the Waku2 network need a means of discovering nodes within the network.
[10/WAKU2](../10/waku2.md) supports the following discovery methods in order of increasing decentralization
For establishing and growing this network,
new nodes trying to join the Waku2 network need a means of discovering nodes
within the network.
[10/WAKU2](../10/waku2.md) supports the following discovery methods
in order of increasing decentralization
* hard coded bootstrap nodes
* [`DNS discovery`](https://rfc.vac.dev/spec/10/#discovery-domain) (based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459))
* `peer-exchange` (work in progress)
* `33/WAKU2-DISCV5` (specified in this document)
The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md) is discovering Waku2 nodes in a decentralized way.
The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network, which allows avoiding hotspots and allows merging the network after a split.
While the other methods provide either a fixed or local set of nodes, `33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.
The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md)
is discovering Waku2 nodes in a decentralized way.
The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network,
which allows avoiding hotspots and allows merging the network after a split.
While the other methods provide either a fixed or local set of nodes,
`33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
Future iterations of this document will add the possibility
of efficiently discovering Waku2 nodes that have certain capabilities,
e.g. holding messages of a certain time frame
during which the querying node was offline.
### Separate Discovery Network
#### w.r.t. Waku2 Relay Network
`33/WAKU2-DISCV5` spans an overlay network separate from the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) network [11/WAKU2-RELAY](../11/relay.md) builds on.
`33/WAKU2-DISCV5` spans an overlay network separate from the
[GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
network [11/WAKU2-RELAY](../11/relay.md) builds on.
Because it is a P2P network on its own, it also depends on bootstrap nodes.
Having a separate discovery network reduces load on the bootstrap nodes, because the actual work is done by randomly discovered nodes.
Having a separate discovery network reduces load on the bootstrap nodes,
because the actual work is done by randomly discovered nodes.
This also increases decentralization.
#### w.r.t. Ethereum Discovery v5
`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.
`33/WAKU2-DISCV5` spans a discovery network
isolated from the Ethereum Discovery v5 network.
Another simple solution would be taking part in the Ethereum Discovery network, and filtering Waku nodes based on whether they support [WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md).
Another simple solution would be taking part in the Ethereum Discovery network,
and filtering Waku nodes based on whether they support [WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md).
This solution is more resilient towards eclipse attacks.
However, this discovery method is very inefficient for small percentages of Waku nodes (see [estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)).
However, this discovery method is very inefficient
for small percentages of Waku nodes
(see [estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)).
It boils down to random walk discovery and does not offer a O(log(n)) hop bound.
The rarer the requested property (in this case Waku), the longer a random walk will take until finding an appropriate node, which leads to a needle-in-the-haystack problem.
Using a dedicated Waku2 discovery network, nodes can query this discovery network for a random set of nodes
and all (well-behaving) returned nodes can serve as bootstrap nodes for other Waku2 protocols.
The rarer the requested property (in this case Waku),
the longer a random walk will take until finding an appropriate node,
which leads to a needle-in-the-haystack problem.
Using a dedicated Waku2 discovery network,
nodes can query this discovery network for a random set of nodes
and all (well-behaving)
returned nodes can serve as bootstrap nodes for other Waku2 protocols.
A more sophisticated solution would be using [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement).
However, in its current state it also has efficiency problems for small percentages of Waku nodes and is still in the design phase ([see here](https://github.com/ethereum/devp2p/issues/199)).
However, in its current state it also has efficiency problems for small percentages
of Waku nodes and is still in the design phase
([see here](https://github.com/ethereum/devp2p/issues/199)).
Currently, the Ethereum discv5 network is very efficient in finding other discv5 nodes,
but it is not so efficient for finding discv5 nodes that have a specific property or offer specific services, e.g. Waku.
Currently,
the Ethereum discv5 network is very efficient in finding other discv5 nodes,
but it is not so efficient for finding discv5 nodes
that have a specific property or
offer specific services, e.g. Waku.
As part of our [discv5 roadmap](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121), we consider two ideas for future versions of `33/WAKU2-DISCV5`
As part of our [discv5 roadmap](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121),
we consider two ideas for future versions of `33/WAKU2-DISCV5`
* [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement) with adjustments (ideally upstream)
* a hybrid solution that uses both a separate discv5 network and a Waku-ENR-filtered Ethereum discv5 network
* [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
with adjustments (ideally upstream)
* a hybrid solution that uses both a separate discv5 network and
a Waku-ENR-filtered Ethereum discv5 network
## Semantics
`33/WAKU2-DISCV5` fully inherits the [discv5 semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
Before announcing their address via Waku2 discv5, nodes SHOULD check if this address is publicly reachable.
Nodes MAY use the [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md) to perform that check.
Before announcing their address via Waku2 discv5,
nodes SHOULD check if this address is publicly reachable.
Nodes MAY use the [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)
to perform that check.
Nodes SHOULD only announce publicly reachable addresses via Waku2 discv5,
to avoid cluttering peer lists with nodes that are not reachable.
## Wire Format Specification
`33/WAKU2-DISCV5` inherits the [discv5 wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md) except for the following differences
`33/WAKU2-DISCV5` inherits the [discv5 wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
except for the following differences
## WAKU2-Specific `protocol-id`
Ethereum discv5:
<pre>
<code>
```text
header = static-header || authdata
static-header = protocol-id || version || flag || nonce || authdata-size
protocol-id = <b>"discv5"</b>
@@ -97,12 +135,12 @@ version = 0x0001
authdata-size = uint16 -- byte length of authdata
flag = uint8 -- packet type identifier
nonce = uint96 -- nonce of message
</code>
</pre>
```
`33/WAKU2-DISCV5`:
<pre>
```text
kcode>
header = static-header || authdata
static-header = protocol-id || version || flag || nonce || authdata-size
@@ -111,56 +149,67 @@ version = 0x0001
authdata-size = uint16 -- byte length of authdata
flag = uint8 -- packet type identifier
nonce = uint96 -- nonce of message
</code>
</pre>
```
## Suggestions for Implementations
Existing discv5 implementations
* can be augmented to make the `protocol-id` selectable using a compile-time flag as in [this feature branch](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim#L34) of nim-eth/discv5.
* can be augmented to make the `protocol-id` selectable using a compile-time flag
as in [this feature branch](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim#L34)
of nim-eth/discv5.
* can be forked followed by changing the `protocol-id` string as in [go-waku](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go#L135-L137).
## Security Considerations
### Sybil attack
Implementations should limit the number of bucket entries that have the same network parameters (IP address / port) to mitigate Sybil attacks.
Implementations should limit the number of bucket entries
that have the same network parameters (IP address / port) to mitigate Sybil attacks.
### Eclipse attack
Eclipse attacks aim to eclipse certain regions in a DHT.
Malicious nodes provide false routing information for certain target regions.
The larger the desired eclipsed region, the more resources (i.e. controlled nodes) the attacker needs.
The larger the desired eclipsed region,
the more resources (i.e. controlled nodes) the attacker needs.
This introduces an efficiency versus resilience tradeoff.
Discovery is more efficient if information about target objects (e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address.
If nodes providing specific information are closer to each other, they cover a smaller range in the DHT and are easier to eclipse.
Discovery is more efficient if information about target objects
(e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address.
If nodes providing specific information are closer to each other,
they cover a smaller range in the DHT and are easier to eclipse.
Sybil attacks greatly increase the power of eclipse attacks, because they significantly reduce resources necessary to mount a successful eclipse attack.
Sybil attacks greatly increase the power of eclipse attacks,
because they significantly reduce resources necessary
to mount a successful eclipse attack.
## Security Implications of a Separate Discovery Network
A dedicated Waku discovery network is more likely to be subject to successful eclipse attacks (and to DoS attacks in general).
A dedicated Waku discovery network is more likely to be subject
to successful eclipse attacks (and to DoS attacks in general).
This is because eclipsing in a smaller network requires less resources for the attacker.
DoS attacks render the whole network unusable if the percentage of attacker nodes is sufficient.
DoS attacks render the whole network unusable
if the percentage of attacker nodes is sufficient.
Using random walk discovery would mitigate eclipse attacks targeted at specific capabilities, e.g. Waku.
However, this is because eclipse attacks aim at the DHT overlay structure, which is not used by random walks.
Using random walk discovery would mitigate eclipse attacks
targeted at specific capabilities, e.g. Waku.
However, this is because eclipse attacks aim at the DHT overlay structure,
which is not used by random walks.
So, this mitigation would come at the cost of giving up overlay routing efficiency.
The efficiency loss is especially severe with a relatively small number of Waku nodes.
Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.
Properly protecting against eclipse attacks is challenging and
raises research questions that we will address in future stages of our discv5 roadmap.
## References
1. [10/WAKU2](../10/waku2.md)
1. [11/WAKU2-RELAY](../11/relay.md)
1. [`WAKU2-ENR`](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
1. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
1. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
1. [`discv5` semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
1. [`discv5` wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
1. [`discv5` wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
1. [`discv5` topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
1. [Waku DNS discovery](https://rfc.vac.dev/spec/10/#discovery-domain)
1. [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)

984
waku/standards/core/36/bindings-api.md
View File

File diff suppressed because it is too large Load Diff

204
waku/standards/core/64/network.md
View File

@@ -11,7 +11,8 @@ contributors:
## Abstract
This specification describes an opinionated deployment of [10/WAKU2](../10/waku2.md) protocols to form a coherent and
This specification describes an opinionated deployment of [10/WAKU2](../10/waku2.md)
protocols to form a coherent and
shared decentralized messaging network that is open-access,
useful for generalized messaging, privacy-preserving, scalable and
accessible even to resource-restricted devices.
@@ -24,35 +25,47 @@ All The Waku Network configuration parameters are listed [here](https://github.c
### Routing protocol
The Waku Network is built on the [17/WAKU2-RLN-RELAY](../17/rln-relay.md) routing protocol,
which in turn is an extension of [11/WAKU2-RELAY](../11/relay.md) with spam protection measures.
The Waku Network is built on the
[17/WAKU2-RLN-RELAY](../17/rln-relay.md) routing protocol,
which in turn is an extension of
[11/WAKU2-RELAY](../11/relay.md) with spam protection measures.
### Network shards
Traffic in the Waku Network is sharded into eight [17/WAKU2-RLN-RELAY](../17/rln-relay.md) pubsub topics.
Traffic in the Waku Network is sharded into eight
[17/WAKU2-RLN-RELAY](../17/rln-relay.md) pubsub topics.
Each pubsub topic is named according to the static shard naming format
defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)
with:
* `<cluster_id>` set to `1`
* `<shard_number>` occupying the range `0` to `7`.
In other words, the Waku Network is a [17/WAKU2-RLN-RELAY](../17/rln-relay.md) network
routed on the combination of the eight pubsub topics:
```
```text
/waku/2/rs/1/0
/waku/2/rs/1/1
...
/waku/2/rs/1/7
```
A node MUST use [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md) protocol to identify the `<cluster_id>` that every
inbound/outbound peer that attempts to connect supports. In any of the following cases, the node MUST trigger a disconnection:
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md) dial fails.
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md) reports an empty `<cluster_id>`.
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md) reports a `<cluster_id>` different than `1`.
A node MUST use [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
protocol to identify the `<cluster_id>` that every
inbound/outbound peer that attempts to connect supports.
In any of the following cases, the node MUST trigger a disconnection:
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
dial fails.
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
reports an empty `<cluster_id>`.
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
reports a `<cluster_id>` different than `1`.
## Roles
There are two distinct roles evident in the network, those of:
1) nodes, and
2) applications.
@@ -61,7 +74,7 @@ There are two distinct roles evident in the network, those of:
Nodes are the individual software units
using [10/WAKU2](../10/waku2.md) protocols to form a p2p messaging network.
Nodes, in turn, can participate in a shard as full relayers, i.e. _relay nodes_,
or by running a combination of protocols suitable for resource-restricted environments,
or by running a combination of protocols suitable for resource-restricted environments,
i.e. _non-relay nodes_.
Nodes can also provide various services to the network,
such as storing historical messages or protecting the network against spam.
@@ -77,92 +90,131 @@ but MUST be subscribed to at least one defined shard.
Each relay node SHOULD be subscribed to as many shards as it has resources to support.
If a relay node supports an encapsulating application,
it SHOULD be subscribed to all the shards servicing that application.
If resource restrictions prevent a relay node from servicing all shards used by the encapsulating application,
If resource restrictions prevent a relay node from servicing all shards
used by the encapsulating application,
it MAY choose to support some shards as a non-relay node.
#### Bootstrapping and discovery
Nodes MAY use any method to bootstrap connection to the network,
but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect to using [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459).
Relay nodes SHOULD use [33/WAKU2-DISCV5](../33/discv5.md) to continually discover other peers in the network.
but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect
to using [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459).
Relay nodes SHOULD use [33/WAKU2-DISCV5](../33/discv5.md) to continually discover
other peers in the network.
Each relay node MUST encode its supported shards into its discoverable ENR,
as described in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#discovery).
The ENR MUST be updated if the set of supported shards change.
A node MAY choose to ignore discovered peers that do not support any of the shards in its own subscribed set.
A node MAY choose to ignore discovered peers that do not support any of the shards
in its own subscribed set.
#### Transports
Relay nodes MUST follow [10/WAKU2](../10/waku2.md) specifications with regards to supporting different transports.
If TCP transport is available, each relay node MUST support it as transport for both dialing and listening.
In addition, a relay node SHOULD support secure websockets for bidirectional communication streams,
Relay nodes MUST follow [10/WAKU2](../10/waku2.md) specifications with regards
to supporting different transports.
If TCP transport is available,
each relay node MUST support it as transport for both dialing and listening.
In addition,
a relay node SHOULD support secure websockets for bidirectional communication streams,
for example to allow connections from and to web browser-based clients.
A relay node MAY support unsecure websockets if required by the application or running environment.
A relay node MAY support unsecure websockets if required by the application or
running environment.
#### Default services
For each supported shard,
each relay node SHOULD enable and support the following protocols as a service node:
1. [12/WAKU2-FILTER](../12/filter.md) to allow resource-restricted peers to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](../13/store.md) to allow other peers to request historical messages from this node.
3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md) to allow resource-restricted peers to request publishing a message to the network on their behalf.
4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md) to allow resource-restricted peers to discover more peers in a resource efficient way.
1. [12/WAKU2-FILTER](../12/filter.md) to allow resource-restricted peers to subscribe
to messages matching a specific content filter.
2. [13/WAKU2-STORE](../13/store.md) to allow other peers to request historical messages
from this node.
3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md) to allow resource-restricted peers to
request publishing a message to the network on their behalf.
4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md)
to allow resource-restricted peers to discover more peers
in a resource efficient way.
#### Store service nodes
Each relay node SHOULD support [13/WAKU2-STORE](../13/store.md) as a store service node,
for each supported shard.
The store SHOULD be configured to retain at least `12` hours of messages per supported shard.
Store service nodes SHOULD only store messages with a valid [`rate_limit_proof`](#message-attributes) attribute.
Each relay node SHOULD support [13/WAKU2-STORE](../13/store.md)
as a store service node, for each supported shard.
The store SHOULD be configured to retain at least `12` hours of messages
per supported shard.
Store service nodes SHOULD only store messages
with a valid [`rate_limit_proof`](#message-attributes) attribute.
#### Non-relay nodes
Nodes MAY opt out of relay functionality on any network shard
and instead request services from relay nodes as clients
using any of the defined service protocols:
1. [12/WAKU2-FILTER](../12/filter.md) to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](../13/store.md) to request historical messages matching a specific content filter.
3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md) to request publishing a message to the network.
4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md) to discover more peers in a resource efficient way.
1. [12/WAKU2-FILTER](../12/filter.md)
to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](../13/store.md)
to request historical messages matching a specific content filter.
3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md)
to request publishing a message to the network.
4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md)
to discover more peers in a resource efficient way.
#### Store client nodes
Nodes MAY request historical messages from [13/WAKU2-STORE](../13/store.md) service nodes as store clients.
A store client SHOULD discard any messages retrieved from a store service node that do not contain a valid [`rate_limit_proof`](#message-attributes) attribute.
The client MAY consider service nodes returning messages without a valid [`rate_limit_proof`](#message-attributes) attribute as untrustworthy.
Nodes MAY request historical messages from [13/WAKU2-STORE](../13/store.md)
service nodes as store clients.
A store client SHOULD discard any messages retrieved from a store service node
that do not contain a valid [`rate_limit_proof`](#message-attributes) attribute.
The client MAY consider service nodes returning messages
without a valid [`rate_limit_proof`](#message-attributes) attribute as untrustworthy.
The mechanism by which this may happen is currently underdefined.
### Applications
Applications are the higher-layer projects or platforms that make use of the generalized messaging capability of the network.
In other words, an application defines a payload used in the various [10/WAKU2](../10/waku2.md) protocols.
Any participant in an application SHOULD make use of an underlying node in order to communicate on the network.
Applications are the higher-layer projects or
platforms that make use of the generalized messaging capability of the network.
In other words,
an application defines a payload used in the various [10/WAKU2](../10/waku2.md) protocols.
Any participant in an application SHOULD make use of an underlying node
in order to communicate on the network.
Applications SHOULD make use of an [autosharding](#autosharding) API
to allow the underlying node to automatically select the target shard on the Waku Network.
to allow the underlying node to automatically select the target shard
on the Waku Network.
See the section on [autosharding](#autosharding) for more.
## RLN rate-limiting
The [17/WAKU2-RLN-RELAY](../17/rln-relay.md) protocol uses [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md) proofs
to ensure that a pre-agreed rate limit of `x` messages every `y` seconds is not exceeded by any publisher.
The [17/WAKU2-RLN-RELAY](../17/rln-relay.md) protocol uses [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md)
proofs to ensure that a pre-agreed rate limit
of `x` messages every `y` seconds is not exceeded by any publisher.
While the network is under capacity,
individual relayers MAY choose to freely route messages without RLN proofs
up to a discretionary bandwidth limit,
after which messages without proofs MUST be discarded by relay nodes.
This bandwidth limit SHOULD be enforced using a [bandwidth validation mechanism](#free-bandwidth-exceeded) separate from a RLN rate-limiting.
This implies that quality of service and reliability is significantly lower for messages without proofs
This bandwidth limit SHOULD be enforced using a [bandwidth validation mechanism](#free-bandwidth-exceeded)
separate from a RLN rate-limiting.
This implies that quality of service and
reliability is significantly lower for messages without proofs
and at times of high network utilization these messages may not be relayed at all.
### RLN Parameters
The Waku Network uses the following RLN parameters:
* `rlnRelayUserMessageLimit=100`: Amount of messages that a membership is allowed to publish per epoch. Configurable between `0` and `MAX_MESSAGE_LIMIT`.
* `rlnRelayUserMessageLimit=100`:
Amount of messages that a membership is allowed to publish per epoch.
Configurable between `0` and `MAX_MESSAGE_LIMIT`.
* `rlnEpochSizeSec=600`: Size of the epoch in seconds.
* `rlnRelayChainId=11155111`: Network in which the RLN contract is deployed, aka Sepolia.
* `rlnRelayEthContractAddress=0xCB33Aa5B38d79E3D9Fa8B10afF38AA201399a7e3`: Network address where RLN memberships are stored.
* `staked_fund=0`: In other words, the Waku Network does not use RLN staking. Registering a membership just requires to pay gas.
* `MAX_MESSAGE_LIMIT=100`: Maximum amount of messages allowed per epoch for any membership. Enforced in the contract.
* `max_epoch_gap=20`: Maximum allowed gap in seconds into the past or future compared to the validator's clock.
* `rlnRelayChainId=11155111`: Network in which the RLN contract is deployed,
aka Sepolia.
* `rlnRelayEthContractAddress=0xCB33Aa5B38d79E3D9Fa8B10afF38AA201399a7e3`:
Network address where RLN memberships are stored.
* `staked_fund=0`: In other words, the Waku Network does not use RLN staking.
Registering a membership just requires to pay gas.
* `MAX_MESSAGE_LIMIT=100`:
Maximum amount of messages allowed per epoch for any membership.
Enforced in the contract.
* `max_epoch_gap=20`: Maximum allowed gap in seconds into the past or
future compared to the validator's clock.
Nodes MUST _reject_ messages not respecting any of these parameters.
Nodes SHOULD use Network Time Protocol (NTP) to synchronize their own clocks,
@@ -189,36 +241,53 @@ according to the rules discussed under [message validation](#message-validation)
### Message Attributes
- The mandatory `payload` attribute MUST contain the message data payload as crafted by the application.
- The mandatory `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering.
* The mandatory `payload` attribute MUST contain the message data payload
as crafted by the application.
* The mandatory `content_topic` attribute MUST specify a string identifier
that can be used for content-based filtering.
This is also crafted by the application.
See [Autosharding](#autosharding) for more on the content topic format.
- The optional `meta` attribute MAY be omitted.
* The optional `meta` attribute MAY be omitted.
If present, will form part of the message uniqueness vector described in [14/WAKU2-MESSAGE](../14/message.md).
- The optional `version` attribute SHOULD be set to `0`. It MUST be interpreted as `0` if not present.
- The mandatory `timestamp` attribute MUST contain the Unix epoch time at which the message was generated by the application.
* The optional `version` attribute SHOULD be set to `0`.
It MUST be interpreted as `0` if not present.
* The mandatory `timestamp` attribute MUST contain the Unix epoch time
at which the message was generated by the application.
The value MUST be in nanoseconds.
It MAY contain a fudge factor of up to 1 seconds in either direction to improve resistance to timing attacks.
- The optional `ephemeral` attribute MUST be set to `true` if the message should not be persisted by the Waku Network.
- The optional `rate_limit_proof` attribute SHOULD be populated with the RLN proof as set out in [RLN Proofs](#rln-proofs).
It MAY contain a fudge factor of up to 1 seconds in either direction
to improve resistance to timing attacks.
* The optional `ephemeral` attribute MUST be set to `true`,
if the message should not be persisted by the Waku Network.
* The optional `rate_limit_proof` attribute SHOULD be populated with the RLN proof
as set out in [RLN Proofs](#rln-proofs).
Messages with this field unpopulated MAY be discarded from the network by relayers.
This field MUST be populated if the message should be persisted by the Waku Network.
### Message Size
Any [14/WAKU2-MESSAGE](../14/message.md) published to the network MUST NOT exceed an absolute maximum size of `150` kilobytes.
This limit applies to the entire message after protobuf serialization, including attributes.
It is RECOMMENDED not to exceed an average size of `4` kilobytes for [14/WAKU2-MESSAGE](../14/message.md) published to the network.
Any [14/WAKU2-MESSAGE](../14/message.md) published to the network
MUST NOT exceed an absolute maximum size of `150` kilobytes.
This limit applies to the entire message after protobuf serialization,
including attributes.
It is RECOMMENDED not to exceed an average size of `4` kilobytes
for [14/WAKU2-MESSAGE](../14/message.md) published to the network.
### Message Validation
Relay nodes MUST apply [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) to each relayed message and
SHOULD apply all of the rules set out in the section below to determine the validity of a message.
Relay nodes MUST apply [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators)
to each relayed message and
SHOULD apply all of the rules set out in the section below
to determine the validity of a message.
Validation has one of three outcomes,
repeated here from the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for ease of reference:
1. Accept - the message is considered valid and it MUST be delivered and forwarded to the network.
2. Reject - the message is considered invalid, MUST be rejected and SHOULD trigger a gossipsub scoring penalty against the transmitting peer.
3. Ignore - the message SHOULD NOT be delivered and forwarded to the network, but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer.
repeated here from the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators)
for ease of reference:
1. Accept - the message is considered valid and
it MUST be delivered and forwarded to the network.
2. Reject - the message is considered invalid, MUST be rejected and
SHOULD trigger a gossipsub scoring penalty against the transmitting peer.
3. Ignore - the message SHOULD NOT be delivered and forwarded to the network,
but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer.
The following validation rules are defined:
@@ -266,7 +335,8 @@ If a message contains an RLN proof
and the relay node detects double signaling
according to the verification process described in [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md),
the relay node MUST _reject_ the message
for violating the agreed rate limit of `rlnRelayUserMessageLimit` messages every `rlnEpochSizeSec` second.
for violating the agreed rate limit of `rlnRelayUserMessageLimit` messages
every `rlnEpochSizeSec` second.
This SHOULD trigger a penalty against the transmitting peer.
## Autosharding
@@ -281,7 +351,7 @@ Applications using autosharding MUST use content topics in the format
defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#content-topics-format-for-autosharding)
and SHOULD use the short length format:
```
```text
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}
```

23
waku/standards/core/66/metadata.md
View File

@@ -8,25 +8,30 @@ contributors:
---
## Abstract
This specification describes the metadata that can be associated with a [10/WAKU2](../10/waku2.md) node.
This specification describes the metadata
that can be associated with a [10/WAKU2](../10/waku2.md) node.
## Metadata Protocol
Waku specifies a req/resp protocol that provides information about the node's medatadata.
Such metadata is meant to be used by the node to decide if a peer is worth connecting or not.
The node that makes the request, includes its metadata so that the receiver is aware of it,
without requiring an extra interaction.
Waku specifies a req/resp protocol that provides information about the node's medatadata.
Such metadata is meant to be used by the node to decide if a peer is worth connecting
or not.
The node that makes the request,
includes its metadata so that the receiver is aware of it,
without requiring an extra interaction.
The parameters are the following:
* `clusterId`: Unique identifier of the cluster that the node is running in.
* `shards`: Shard indexes that the node is subscribed to.
***Protocol Identifier***
/vac/waku/metadata/1.0.0
> /vac/waku/metadata/1.0.0
### Request
```proto
```protobuf
message WakuMetadataRequest {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
@@ -35,7 +40,7 @@ message WakuMetadataRequest {
### Response
```proto
```protobuf
message WakuMetadataResponse {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
@@ -48,4 +53,4 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References
- [10/WAKU2](../10/waku2.md)
* [10/WAKU2](../10/waku2.md)

BIN
waku/standards/legacy/.DS_Store vendored Normal file
View File

Binary file not shown.

459
waku/standards/legacy/6/waku1.md
View File

@@ -11,11 +11,23 @@ contributors:
- Kim De Mey <kimdemey@status.im>
---
This specification describes the format of Waku packets within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic envelopes (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
This specification describes the format of Waku packets within the ÐΞVp2p Wire Protocol.
This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627).
Waku is a fork of the original Whisper protocol
that enables better usability for resource restricted devices,
such as mostly-offline bandwidth-constrained smartphones.
It does this through (a) light node support, (b) historic envelopes
(with a mailserver) (c) expressing topic interest for better bandwidth usage
and (d) basic rate limiting.
## Motivation
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku packets in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
Waku was created to incrementally improve in areas that Whisper is lacking in,
with special attention to resource restricted devices.
We specify the standard for Waku packets in order to ensure forward compatibility
of different Waku clients, backwards compatibility with Whisper clients,
as well as to allow multiple implementations of Waku and its capabilities.
We also modify the language to be more unambiguous, concise and consistent.
## Definitions
@@ -30,34 +42,60 @@ Waku was created to incrementally improve in areas that Whisper is lacking in, w
### Use of DevP2P
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
For nodes to communicate, they MUST implement devp2p and run RLPx.
They MUST have some way of connecting to other nodes.
Node discovery is largely out of scope for this spec,
but see the appendix for some suggestions on how to do this.
This protocol needs to advertise the `waku/1` [capability](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
### Gossip based routing
In Whisper, envelopes are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Envelopes are eligible for retransmission until their TTL expires. A node SHOULD relay envelopes to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [8/WAKU-MAIL](../8/mail.md) response. A node SHOULD NOT send an envelope to a peer that it has already sent before.
In Whisper, envelopes are gossiped between peers.
Whisper is a form of rumor-mongering protocol
that works by flooding to its connected peers based on some factors.
Envelopes are eligible for retransmission until their TTL expires.
A node SHOULD relay envelopes to all connected nodes
if an envelope matches their PoW and bloom filter settings.
If a node works in light mode, it MAY choose not to forward envelopes.
A node MUST NOT send expired envelopes,
unless the envelopes are sent as a [8/WAKU-MAIL](../8/mail.md) response.
A node SHOULD NOT send an envelope to a peer that it has already sent before.
### Maximum Packet Size
Nodes SHOULD limit the maximum size of both packets and envelopes. If a packet or envelope exceeds its limit, it MUST be dropped.
Nodes SHOULD limit the maximum size of both packets and envelopes.
If a packet or envelope exceeds its limit, it MUST be dropped.
- **RLPx Packet Size** - This size MUST be checked before a message is decoded.
- **Waku Envelope Size** - Each envelope contained in an RLPx packet MUST then separately be checked against the maximum envelope size.
- **Waku Envelope Size** - Each envelope contained in an RLPx packet
MUST then separately be checked against the maximum envelope size.
Clients MAY use their own maximum packet and envelope sizes. The default values are `1.5mb` for the RLPx Packet and `1mb` for a Waku envelope.
Clients MAY use their own maximum packet and envelope sizes.
The default values are `1.5mb` for the RLPx Packet and `1mb` for a Waku envelope.
## Wire Specification
### Use of RLPx transport protocol
All Waku packets are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
All Waku packets are sent as devp2p RLPx transport protocol, version 5[^1] packets.
These packets MUST be RLP-encoded arrays of data containing two objects:
packet code followed by another object (whose type depends on the packet code).
See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and
the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf)
for more details on RLP.
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
Waku is a RLPx subprotocol called `waku` with version `0`.
The version number corresponds to the major version in the header spec.
Minor versions should not break compatibility of `waku`,
this would result in a new major.
(Some exceptions to this apply in the Draft stage
of where client implementation is rapidly change).
### ABNF specification
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
we have the following format:
```abnf
; Packet codes 0 - 127 are reserved for Waku protocol
@@ -185,13 +223,15 @@ optional-packet = 11 batch-ack /
packet = "[" required-packet [ optional-packet ] "]"
```
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
All primitive types are RLP encoded. Note that, per RLP specification,
integers are encoded starting from `0x00`.
### Packet Codes
The packet codes reserved for Waku protocol: 0 - 127.
Packets with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
Packets with unknown codes MUST be ignored without generating any error,
for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
@@ -219,17 +259,25 @@ The Status packet serves as a Waku handshake and peers MUST exchange this
packet upon connection. It MUST be sent after the RLPx handshake and prior to
any other Waku packets.
A Waku node MUST await the Status packet from a peer before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status packet from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
A Waku node MUST await the Status packet from a peer
before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status packet from a peer,
before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status packet, the node SHOULD validate the packet
received and validated the Status packet. Note that its peer might not be in
the same state.
When a node is receiving other Waku packets from a peer before a Status
packet is received, the node MUST ignore these packets and SHOULD disconnect from that peer. Status packets received after the handshake is completed MUST also be ignored.
packet is received, the node MUST ignore these packets and
SHOULD disconnect from that peer.
Status packets received after the handshake is completed MUST also be ignored.
The Status packet MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
The Status packet MUST contain an association list containing various options.
All options within this association list are OPTIONAL,
ordering of the key-value pairs is not guaranteed and
therefore MUST NOT be relied on.
Unknown keys in the association list SHOULD be ignored.
#### Messages
@@ -237,75 +285,105 @@ This packet is used for sending the standard Waku envelopes.
#### Status Update
The Status Update packet is used to communicate an update of the settings of the node.
The Status Update packet is used to communicate an update
of the settings of the node.
The format is the same as the Status packet, all fields are optional.
If none of the options are specified the packet MUST be ignored and considered a noop.
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
be transmitted.
If none of the options are specified the packet MUST be ignored and
considered a noop.
Fields that are omitted are considered unchanged,
fields that haven't changed SHOULD not be transmitted.
##### PoW Requirement Field
When PoW Requirement is updated, peers MUST NOT deliver envelopes with PoW lower than the PoW Requirement specified.
When PoW Requirement is updated,
peers MUST NOT deliver envelopes with PoW lower than the PoW Requirement specified.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by envelope size and TTL:
PoW is defined as average number of iterations,
required to find the current BestBit
(the number of leading zero bits in the hash), divided by envelope size and TTL:
PoW = (2**BestBit) / (size * TTL)
PoW = (2**BestBit) / (size * TTL)
PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
where size is the size of the RLP-encoded envelope,
excluding `env_nonce` field (size of `short_rlp(envelope)`).
##### Bloom Filter Field
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
The bloom filter is used to identify a number of topics to a peer without compromising
(too much) privacy over precisely what topics are of interest.
Precise control over the information content (and thus efficiency of the filter)
may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
Blooms are formed by the bitwise OR operation on a number of bloomed topics.
The bloom function takes the topic and projects them onto a 512-bit slice.
At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
The projection function is defined as a mapping
from a 4-byte slice S to a 512-bit slice D; for ease of explanation,
S will dereference to bytes, whereas D will dereference to bits.
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
A full bloom filter (all the bits set to 1)
means that the node is to be considered a `Full Node` and it will accept any topic.
If both topic interest and bloom filter are specified, topic interest always takes precedence and bloom filter MUST be ignored.
If both topic interest and bloom filter are specified,
topic interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current topic interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
If only bloom filter is specified,
the current topic interest MUST be discarded and
only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
A bloom filter with all bits set to 0 signals that the node is not currently
interested in receiving any envelope.
##### Topic Interest Field
Topic interest is used to share a node's interest in envelopes with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
Topic interest is used to share a node's interest in envelopes with specific topics.
It does this in a more bandwidth considerate way,
at the expense of some metadata protection.
Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics.
If you are interested in more topics than that, this is currently underspecified
and likely requires updating it. The constant is subject to change.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only topic interest is specified, the current bloom filter MUST be discarded and
only the updated topic interest MUST be used when forwarding or posting envelopes.
If only topic interest is specified, the current bloom filter MUST be discarded and only the updated topic interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node is not currently interested in receiving any envelope.
An empty array signals that the node is not currently interested in receiving
any envelope.
##### Rate Limits Field
Rate limits is used to inform other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
In order to provide basic Denial-of-Service attack protection,
each node SHOULD define its own rate limits.
The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the `status-update` packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD broadcast its rate limits to its peers
using the `status-update` packet.
The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
Each node SHOULD respect rate limits advertised by its peers.
The number of packets SHOULD be throttled in order not to exceed peer's rate limits.
If the limit gets exceeded, the connection MAY be dropped by the peer.
Two rate limits strategies are applied:
@@ -318,193 +396,320 @@ The size limit SHOULD be greater or equal than the maximum packet size.
##### Light Node Field
When the node's `light-node` field is set to true, the node SHOULD NOT forward Envelopes from its peers.
When the node's `light-node` field is set to true,
the node SHOULD NOT forward Envelopes from its peers.
A node connected to a peer with the `light-node` field set to true MUST NOT depend on the peer for forwarding Envelopes.
A node connected to a peer with the `light-node` field set to true
MUST NOT depend on the peer for forwarding Envelopes.
##### Confirmations Enabled Field
When the node's `confirmations-enabled` field is set to true, the node SHOULD send [message confirmations](#batch-ack-and-message-response) to its peers.
When the node's `confirmations-enabled` field is set to true,
the node SHOULD send [message confirmations](#batch-ack-and-message-response)
to its peers.
#### Batch Ack and Message Response
Message confirmations tell a node that an envelope originating from it has been received by its peers, allowing a node to know whether an envelope has or has not been received.
Message confirmations tell a node that an envelope
originating from it has been received by its peers,
allowing a node to know whether an envelope has or has not been received.
A node MAY send a message confirmation for any batch of envelopes received with a Messages packet (`0x01`).
A node MAY send a message confirmation for any batch of envelopes
received with a Messages packet (`0x01`).
A message confirmation is sent using Batch Ack packet (`0x0B`) or Message Response packet (`0x0C`). The message confirmation is specified in the [ABNF specification](#abnf-specification).
A message confirmation is sent using Batch Ack packet (`0x0B`) or
Message Response packet (`0x0C`).
The message confirmation is specified in the [ABNF specification](#abnf-specification).
The current `version` in the `confirmation` is `1`.
The supported error codes:
- `1`: time sync error which happens when an envelope is too old or was created in the future (typically because of an unsynchronized clock of a node).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent envelope, a corresponding confirmation is broadcast by one or more peers.
- `1`: time sync error which happens when an envelope is too old or
was created in the future (typically because of an unsynchronized clock of a node).
The drawback of sending message confirmations is that it increases the noise
in the network because for each sent envelope,
a corresponding confirmation is broadcast by one or more peers.
#### P2P Request
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting historic (expired) envelopes from the [Waku Mail Server](../8/mail.md).
This packet is used for sending Dapp-level peer-to-peer requests,
e.g. Waku Mail Client requesting historic (expired)
envelopes from the [Waku Mail Server](../8/mail.md).
#### P2P Message
This packet is used for sending the peer-to-peer envelopes, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of historic (expired) envelopes, which is otherwise not allowed.
This packet is used for sending the peer-to-peer envelopes,
which are not supposed to be forwarded any further.
E.g. it might be used by the Waku Mail Server for delivery of historic (expired)
envelopes, which is otherwise not allowed.
#### P2P Request Complete
This packet is used to indicate that all envelopes, requested earlier with a P2P Request packet (`0x7E`), have been sent via one or more P2P Message packets (`0x7F`).
This packet is used to indicate that all envelopes,
requested earlier with a P2P Request packet (`0x7E`),
have been sent via one or more P2P Message packets (`0x7F`).
The content of the packet is explained in the [Waku Mail Server](../8/mail.md) specification.
### Payload Encryption
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme
with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
### Packet code Rationale
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions.
Packet code `0x02` and `0x03` were previously used in Whisper but
are deprecated as of Waku v0.4
Packet code `0x22` is used to dynamically change the settings of a node.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without the P2P Message packet it would be impossible to deliver the historic envelopes, since they will be recognized as expired, and the peer will be disconnected for violating the Waku protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and
Client.
Without the P2P Message packet it would be impossible to deliver the historic envelopes,
since they will be recognized as expired, and
the peer will be disconnected for violating the Waku protocol.
They might be useful for other purposes
when it is not possible to spend time on PoW,
e.g. if a stock exchange will want to provide live feed about the latest trades.
## Additional capabilities
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Waku supports multiple capabilities.
These include light node, rate limiting and bridging of traffic.
Here we list these capabilities, how they are identified,
what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on [specification](../8/mail.md).
Additionally,
there is the capability of a mailserver which is documented in its on [specification](../8/mail.md).
### Light node
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
The rationale for light nodes is to allow for interaction with waku
on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming envelopes, they MUST only send their own envelopes. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in envelopes being dropped between the two.
Light nodes MUST NOT forward any incoming envelopes,
they MUST only send their own envelopes.
When light nodes happen to connect to each other, they SHOULD disconnect.
As this would result in envelopes being dropped between the two.
Light nodes are identified by the `light_node` value in the Status packet.
### Accounting for resources (experimental)
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
Nodes MAY implement accounting, keeping track of resource usage.
It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf),
and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Each node keeps track of resource usage with all other nodes.
Whenever an envelope is received from a node that is expected
(fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
Every epoch (say, every minute or every time an event happens)
statistics SHOULD be aggregated and saved by the client:
| peer | sent | received |
|-------|------|----------|
| peer1 | 0 | 123 |
| peer2 | 10 | 40 |
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
In later versions this will be amended by nodes communication thresholds,
settlements and disconnect logic.
## Upgradability and Compatibility
### General principles and policy
The currently advertised capability is `waku/1`. This needs to be advertised in the `hello` `ÐΞVp2p` [packet](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
If a node supports multiple versions of `waku`, those needs to be explicitly advertised. For example if both `waku/0` and `waku/1` are supported, both `waku/0` and `waku/1` MUST be advertised.
The currently advertised capability is `waku/1`.
This needs to be advertised in the `hello` `ÐΞVp2p` [packet](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
If a node supports multiple versions of `waku`, those needs to be explicitly advertised.
For example if both `waku/0` and `waku/1` are supported,
both `waku/0` and `waku/1` MUST be advertised.
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
These are policies that guide how we make decisions when it comes to upgradability,
compatibility, and extensibility:
1. Waku aims to be compatible with previous and future versions.
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
2. In cases where we want to break this compatibility,
we do so gracefully and as a single decision point.
3. To achieve this, we employ the following two general strategies:
- a) Accretion (including protocol negotiation) over changing data
- b) When we want to change things, we give it a new name (for example, a version number).
- b) When we want to change things, we give it a new name
(for example, a version number).
Examples:
- We enable bridging between `shh/6` and `waku/1` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
- When we we want to provide a new set of packets that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
- We enable bridging between `shh/6` and
`waku/1` until such a time as when we are ready to gracefully drop support
for `shh/6` (1, 2, 3).
- When we add parameter fields,
we (currently) do so by accreting them in a list,
so old clients can ignore new fields (dynamic list) and
new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future,
we will likely release a new version that gives better support for open,
growable maps (association lists or native map type) (3)
- When we we want to provide a new set of packets that have different requirements,
we do so under a new protocol version and employ protocol versioning.
This is a form of accretion at a level above -
it ensures a client can support both protocols at once and
drop support for legacy versions gracefully. (1,2,3)
### Backwards Compatibility
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
Waku is a different subprotocol from Whisper so it isn't directly compatible.
However, the data format is the same,
so compatibility can be achieved by the use of a bridging mode as described below.
Any client which does not implement certain packet codes
should gracefully ignore the packets with those codes.
This will ensure the forward compatibility.
### Waku-Whisper bridging
`waku/1` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
`waku/1` and `shh/6` are different DevP2P subprotocols,
however they share the same data format making their envelopes compatible.
This means we can bridge the protocols naively, this works as follows.
**Roles:**
- Waku client A, only Waku capability
- Whisper client B, only Whisper capability
- WakuWhisper bridge C, both Waku and Whisper capability
**Flow:**
1. A posts envelope; B posts envelope.
2. C picks up envelope from A and B and relays them both to Waku and Whisper.
3. A receives envelope on Waku; B on Whisper.
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a envelope between C1 and C2. I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
**Note**: This flow means if another bridge C1 is active,
we might get duplicate relaying for a envelope between C1 and C2.
I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B.
Theoretically this bridging chain can get as long as TTL permits.
### Forward Compatibility
It is desirable to have a strategy for maintaining forward compatibility between `waku/1` and future version of waku. Here we outline some concerns and strategy for this.
It is desirable to have a strategy for maintaining forward compatibility
between `waku/1` and future version of waku.
Here we outline some concerns and strategy for this.
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished is by negotiating the versions of subprotocols, within the `hello` packet nodes transmit their capabilities along with a version. The highest common version should then be used.
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
- **Connecting to nodes with multiple versions:**
The way this SHOULD be accomplished is by negotiating the versions of subprotocols,
within the `hello` packet nodes transmit their capabilities along with a version.
The highest common version should then be used.
- **Adding new packet codes:**
New packet codes can be added easily due to the available packet codes.
Unknown packet codes SHOULD be ignored.
Upgrades that add new packet codes SHOULD implement some fallback mechanism
if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:**
New options can be added to the `status-options` association list in the `status`
and `status-update` packet as options are OPTIONAL and
unknown option keys SHOULD be ignored.
A node SHOULD NOT disconnect from a peer when receiving `status-options`
with unknown option keys.
## Appendix A: Security considerations
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](../8/mail.md#security-considerations) can be found in their respective specifications.
There are several security considerations to take into account when running Waku.
Chief among them are: scalability, DDoS-resistance and privacy.
These also vary depending on what capabilities are used.
The security considerations for extra capabilities,
such as [mailservers](../8/mail.md#security-considerations)
can be found in their respective specifications.
### Scalability and UX
#### Bandwidth usage:
#### Bandwidth usage
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
In version 0 of Waku, bandwidth usage is likely to be an issue.
For more investigation into this,
see the theoretical scaling model described
[here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
#### Gossip-based routing:
#### Gossip-based routing
Use of gossip-based routing doesn't necessarily scale. It means each node can see an envelope multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
Use of gossip-based routing doesn't necessarily scale.
It means each node can see an envelope multiple times, and
having too many light nodes can cause propagation probability that is too low.
See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/)
for more and a possible Kademlia based alternative.
#### Lack of incentives:
#### Lack of incentives
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
Waku currently lacks incentives to run nodes,
which means node operators are more likely to create centralized choke points.
### Privacy
#### Light node privacy:
#### Light node privacy
The main privacy concern with a light node is that it has to reveal its topic interests (in addition to its IP/ID) to its directed peers. This is because when a light node publishes an envelope, its directed peers will know that the light node owns that envelope (as light nodes do not relay other envelopes). Therefore, the directed peers of a light node can make assumptions about what envelopes (topics) the light node is interested in.
The main privacy concern with a light node
is that it has to reveal its topic interests
(in addition to its IP/ID) to its directed peers.
This is because when a light node publishes an envelope,
its directed peers will know that the light node owns that envelope
(as light nodes do not relay other envelopes).
Therefore, the directed peers of a light node can make assumptions about what envelopes
(topics) the light node is interested in.
#### Mailserver client privacy
#### Mailserver client privacy:
A mailserver client fetches archival envelopes from a mailserver
through a direct connection.
In this direct connection,
the client discloses its IP/ID as well as the topics/ bloom filter
it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs
to their topic interests and build a profile for each client over time.
As such, the mailserver client has to trust the mailserver with this level of information.
A mailserver client fetches archival envelopes from a mailserver through a direct connection.
In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs to their topic interests and build a profile for each client over time.
As such, the mailserver client has to trust the mailserver with this level of information.
#### Bloom filter privacy
#### Bloom filter privacy:
By having a bloom filter where only the topics you are interested in are set,
you reveal which envelopes you are interested in.
This is a fundamental tradeoff between bandwidth usage and privacy,
though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf)
[trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
By having a bloom filter where only the topics you are interested in are set, you reveal which envelopes you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
#### Privacy guarantees not rigorous
#### Privacy guarantees not rigorous:
Privacy for Whisper / Waku haven't been studied rigorously for various threat models
like global passive adversary, local active attacker, etc.
This is unlike e.g. Tor and mixnets.
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
#### Topic hygiene
#### Topic hygiene:
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
Similar to bloom filter privacy,
if you use a very specific topic you reveal more information.
See scalability model linked above.
### Spam resistance
**PoW bad for heterogeneous devices:**
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
Proof of work is a poor spam prevention mechanism.
A mobile device can only have a very low PoW
in order not to use too much CPU / burn up its phone battery.
This means someone can spin up a powerful node and overwhelm the network.
### Censorship resistance
**Devp2p TCP port blockable:**
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
By default Devp2p runs on port `30303`,
which is not commonly used for any other service.
This means it is easy to censor, e.g. airport WiFi.
This can be mitigated somewhat by running on e.g. port `80` or `443`,
but there are still outstanding issues.
See libp2p and Tor's Pluggable Transport for how this can be improved.
## Appendix B: Implementation Notes
@@ -519,17 +724,23 @@ By default Devp2p runs on port `30303`, which is not commonly used for any other
Notes useful for implementing Waku mode.
1. Avoid duplicate envelopes
1.Avoid duplicate envelopes
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
To avoid duplicate envelopes,
only connect to one Waku node.
Benign duplicate envelopes is an intrinsic property of Whisper
which often leads to a N factor increase in traffic,
where N is the number of peers you are connected to.
2. Topic specific recommendations
2.Topic specific recommendations
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
Consider partition topics based on some usage,
to avoid too much traffic on a single topic.
### Node discovery
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
to discover nodes.
Known static nodes MAY also be used.
@@ -567,16 +778,21 @@ Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f2417984
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
- Clarify the preferred way of handling unknown keys
in the `status-options` association list.
- Correct spec/implementation mismatch:
Change RLP keys to be the their int values in order to reflect production behavior
### Version 0.4
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
- Simplify implementation matrix with latest state
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
- Introduces a new required packet code Status Code (`0x22`)
for communicating option changes
- Deprecates the following packet codes: PoW Requirement (`0x02`),
Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) -
all superseded by the new Status Code (`0x22`)
- Increased `topic-interest` capacity from 1000 to 10000
### Version 0.3
@@ -586,7 +802,8 @@ Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954
- Recommend DNS based node discovery over other Discovery methods.
- Mark spec as Draft mode in terms of its lifecycle.
- Simplify Changelog and misc formatting.
- Handshake/Status packet not compatible with shh/6 nodes; specifying options as association list.
- Handshake/Status packet not compatible with shh/6 nodes;
specifying options as association list.
- Include topic-interest in Status handshake.
- Upgradability policy.
- `topic-interest` packet code.
@@ -602,7 +819,8 @@ Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/wak
- More details on handshake modifications.
- Accounting for resources mode (experimental)
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
- Appendix with implementation notes and
implementation matrix across various clients with breakdown per capability.
- More details on handshake and parameters.
- Describe rate limits in more detail.
- More details on mailserver and mail client API.
@@ -629,7 +847,6 @@ confirmations-enabled and rate-limits
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## Footnotes
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.

41
waku/standards/legacy/7/data.md
View File

@@ -9,24 +9,29 @@ contributors:
- Kim De Mey <kimdemey@status.im>
---
This specification describes the encryption, decryption and signing of the content in the [data field used in Waku](../6/waku1.md/#abnf-specification).
This specification describes the encryption,
decryption and signing of the content in the [data field used in Waku](../6/waku1.md/#abnf-specification).
## Specification
The `data` field is used within the `waku envelope`, the field MUST contain the encrypted payload of the envelope.
The `data` field is used within the `waku envelope`,
the field MUST contain the encrypted payload of the envelope.
The fields that are concatenated and encrypted as part of the `data` field are:
- flags
- auxiliary field
- payload
- padding
- signature
In case of symmetric encryption, a `salt` (a.k.a. AES Nonce, 12 bytes) field MUST be appended.
- flags
- auxiliary field
- payload
- padding
- signature
In case of symmetric encryption, a `salt`
(a.k.a. AES Nonce, 12 bytes) field MUST be appended.
### ABNF
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
we have the following format:
```abnf
; 1 byte; first two bits contain the size of auxiliary field,
@@ -53,11 +58,23 @@ data = flags auxiliary-field payload padding [signature] [salt]
### Signature
Those unable to decrypt the envelope data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the `R`, `S` and `V` parameters of the SECP-256k1 ECDSA signature, in that order. `R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned. `V` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
Those unable to decrypt the envelope data are also unable to access the signature.
The signature, if provided,
is the ECDSA signature of the Keccak-256 hash of the unencrypted data
using the secret key of the originator identity.
The signature is serialized as the concatenation of the `R`, `S` and
`V` parameters of the SECP-256k1 ECDSA signature, in that order.
`R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned.
`V` MUST be an 8-bit big-endian encoded,
non-normalized and should be either 27 or 28.
### Padding
The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
The padding field is used to align data size,
since data size alone might reveal important metainformation.
Padding can be arbitrary size.
However, it is recommended that the size of Data Field (excluding the Salt)
before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
## Copyright

86
waku/standards/legacy/8/mail.md
View File

@@ -12,19 +12,30 @@ contributors:
## Abstract
In this specification, we describe Mailservers. These are nodes responsible for archiving envelopes and delivering them to peers on-demand.
In this specification, we describe Mailservers.
These are nodes responsible for archiving envelopes and
delivering them to peers on-demand.
## Specification
A node which wants to provide mailserver functionality MUST store envelopes from incoming Messages packets (Waku packet-code `0x01`). The envelopes can be stored in any format, however they MUST be serialized and deserialized to the Waku envelope format.
A node which wants to provide mailserver functionality MUST store envelopes
from incoming Messages packets (Waku packet-code `0x01`).
The envelopes can be stored in any format,
however they MUST be serialized and deserialized to the Waku envelope format.
A mailserver SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
A mailserver SHOULD store envelopes for all topics
to be generally useful for any peer,
however for specific use cases it MAY store envelopes for a subset of topics.
### Requesting Historic Envelopes
In order to request historic envelopes, a node MUST send a packet P2P Request (`0x7e`) to a peer providing mailserver functionality. This packet requires one argument which MUST be a Waku envelope.
In order to request historic envelopes,
a node MUST send a packet P2P Request (`0x7e`)
to a peer providing mailserver functionality.
This packet requires one argument which MUST be a Waku envelope.
In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
In the Waku envelope's payload section,
there MUST be RLP-encoded information about the details of the request:
```abnf
; UNIX time in seconds; oldest requested envelope's creation time
@@ -55,21 +66,44 @@ payload-with-topic = "[" lower upper bloom limit cursor [ topics ] "]"
payload = payload-with-topic | payload-without-topic
```
The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit` so that the requester can send another request using the obtained `Cursor` value. What exactly is in the `Cursor` is up to the implementation. The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request to another mailserver because the format or the result MAY be different.
The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and
`Upper` is greater than `Limit` so that the requester can send another request
using the obtained `Cursor` value.
What exactly is in the `Cursor` is up to the implementation.
The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request
to another mailserver because the format or the result MAY be different.
The envelope MUST be encrypted with a symmetric key agreed between the requester and Mailserver.
The envelope MUST be encrypted with a symmetric key agreed between the requester
and Mailserver.
If `Topics` is used the `Cursor` field MUST be specified for the argument order to be unambiguous. However, it MAY be set to `null`. `Topics` is used to specify which topics a node is interested in. If `Topics` is not empty, a mailserver MUST only send envelopes that belong to a topic from `Topics` list and `Bloom` value MUST be ignored.
If `Topics` is used the `Cursor` field MUST be specified
for the argument order to be unambiguous.
However, it MAY be set to `null`.
`Topics` is used to specify which topics a node is interested in.
If `Topics` is not empty,
a mailserver MUST only send envelopes that belong to a topic from `Topics` list and
`Bloom` value MUST be ignored.
### Receiving Historic Envelopes
Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes. A Mailserver MUST limit the amount of messages sent, either by the `Limit` specified in the request or limited to the maximum [RLPx packet size](./waku#maximum-packet-size), whichever limit comes first.
Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`)
followed by an array of Waku envelopes.
A Mailserver MUST limit the amount of messages sent,
either by the `Limit` specified in the request or
limited to the maximum [RLPx packet size](./waku#maximum-packet-size),
whichever limit comes first.
In order to receive historic envelopes from a mailserver, a node MUST trust the selected mailserver, that is allow to receive expired packets with the P2P Message code. By default, such packets are discarded.
In order to receive historic envelopes from a mailserver,
a node MUST trust the selected mailserver,
that is allow to receive expired packets with the P2P Message code.
By default, such packets are discarded.
Received envelopes MUST be passed through the Whisper envelope pipelines so that they are picked up by registered filters and passed to subscribers.
Received envelopes MUST be passed through the Whisper envelope pipelines
so that they are picked up by registered filters and passed to subscribers.
For a requester, to know that all envelopes have been sent by mailserver, it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by a list with:
For a requester, to know that all envelopes have been sent by mailserver,
it SHOULD handle P2P Request Complete code (`0x7d`).
This code is followed by a list with:
```abnf
; array with a Keccak-256 hash of the envelope containing the original request.
@@ -84,27 +118,39 @@ cursor = *OCTET
payload = "[" request-id last-envelope-hash [ cursor ] "]"
```
If `Cursor` is not empty, it means that not all envelopes were sent due to the set `Limit` in the request. One or more consecutive requests MAY be sent with `Cursor` field filled in in order to receive the rest of the envelopes.
If `Cursor` is not empty,
it means that not all envelopes were sent due to the set `Limit` in the request.
One or more consecutive requests MAY be sent with `Cursor` field filled
in order to receive the rest of the envelopes.
### Security considerations
There are several security considerations to take into account when running or interacting with Mailservers. Chief among them are: scalability, DDoS-resistance and privacy.
There are several security considerations to take into account when running or
interacting with Mailservers.
Chief among them are: scalability, DDoS-resistance and privacy.
**Mailserver High Availability requirement:**
A mailserver has to be online to receive envelopes for other nodes, this puts a high availability requirement on it.
A mailserver has to be online to receive envelopes for other nodes,
this puts a high availability requirement on it.
**Mailserver client privacy:**
A mailserver client fetches archival envelopes from a mailserver through a direct connection.
In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs to their topic interests and build a profile for each client over time.
A mailserver client fetches archival envelopes from a mailserver
through a direct connection.
In this direct connection,
the client discloses its IP/ID as well as the topics/ bloom filter
it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs
to their topic interests and build a profile for each client over time.
As such, the mailserver client has to trust the mailserver with this level of information.
A similar concern exists for the light nodes and their direct peers which is discussed in the security considerations of [6/WAKU1](../6/waku1.md).
A similar concern exists for the light nodes and
their direct peers which is discussed in the security considerations of [6/WAKU1](../6/waku1.md).
**Mailserver trusted connection:**
A mailserver has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning mailserver can overwhelm an individual node.
A mailserver has a direct TCP connection, which means they are trusted to send traffic.
This means a malicious or malfunctioning mailserver can overwhelm an individual node.
## Changelog

248
waku/standards/legacy/9/rpc.md
View File

@@ -9,8 +9,10 @@ contributors:
- Oskar Thorén <oskarth@titanproxy.com>
---
This specification describes the RPC API that Waku nodes MAY adhere to. The unified API allows clients to easily
be able to connect to any node implementation. The API described is privileged as a node stores the keys of clients.
This specification describes the RPC API that Waku nodes MAY adhere to.
The unified API allows clients to easily
be able to connect to any node implementation.
The API described is privileged as a node stores the keys of clients.
## Introduction
@@ -20,7 +22,10 @@ This API is based off the [Whisper V6 RPC API](https://github.com/ethereum/go-et
### Transport
Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API that can be accessed. The JSON RPC version SHOULD be `2.0`. Below is an example request:
Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API
that can be accessed.
The JSON RPC version SHOULD be `2.0`.
Below is an example request:
```json
{
@@ -46,7 +51,11 @@ In this section you will find objects used throughout the JSON RPC API.
#### Message
The message object represents a Waku message. Below you will find the description of the attributes contained in the message object. A message is the decrypted payload and padding of an [envelope](../7/data.md) along with all of its metadata and other extra information such as the hash.
The message object represents a Waku message.
Below you will find the description of the attributes contained in the message object.
A message is the decrypted payload and
padding of an [envelope](../7/data.md) along with all of its metadata and
other extra information such as the hash.
| Field | Type | Description |
| ----: | :--: | ----------- |
@@ -62,7 +71,8 @@ The message object represents a Waku message. Below you will find the descriptio
#### Filter
The filter object represents filters that can be applied to retrieve messages. Below you will find the description of the attributes contained in the filter object.
The filter object represents filters that can be applied to retrieve messages.
Below you will find the description of the attributes contained in the filter object.
| Field | Type | Description |
| ----: | :--: | ----------- |
@@ -73,7 +83,9 @@ The filter object represents filters that can be applied to retrieve messages. B
| `topics` | array | Array of possible topics, this can also contain partial topics |
| `allowP2P` | boolean | Indicates if this filter allows processing of direct peer-to-peer messages |
All fields are optional, however `symKeyID` or `privateKeyID` must be present, it cannot be both. Additionally, the `topics` field is only optional when an asymmetric key is used.
All fields are optional, however `symKeyID` or `privateKeyID` must be present,
it cannot be both.
Additionally, the `topics` field is only optional when an asymmetric key is used.
### Methods
@@ -93,11 +105,11 @@ none
The `waku_info` method returns information about a Waku node.
##### Parameters
Parameters
none
##### Response
Response
The response is an `Object` containing the following fields:
@@ -108,202 +120,231 @@ The response is an `Object` containing the following fields:
#### `waku_setMaxEnvelopeSize`
Sets the maximum envelope size allowed by this node. Any envelopes larger than this size both incoming and outgoing will be rejected. The envelope size can never exceed the underlying envelope size of `10mb`.
Sets the maximum envelope size allowed by this node.
Any envelopes larger than this size both incoming and outgoing will be rejected.
The envelope size can never exceed the underlying envelope size of `10mb`.
##### Parameters
Parameters
- **number** - The message size in bytes.
- **number** - The message size in bytes.
##### Response
Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_setMinPoW`
Sets the minimal PoW required by this node.
##### Parameters
Parameters
- **number** - The new PoW requirement.
- **number** - The new PoW requirement.
##### Response
Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_markTrustedPeer`
Marks a specific peer as trusted allowing it to send expired messages.
##### Parameters
Parameters
- **string** - `enode` of the peer.
##### Response
Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_newKeyPair`
Generates a keypair used for message encryption and decryption.
##### Parameters
Parameters
none
##### Response
Response
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** -
Key ID on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_addPrivateKey`
Stores a key and returns its ID.
##### Parameters
Parameters
- **string** - Private key as hex bytes.
##### Response
Response
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - Key ID on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_deleteKeyPair`
Deletes a specific key if it exists.
##### Parameters
Parameters
- **string** - ID of the Key pair.
##### Response
Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** - `true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_hasKeyPair`
Checks if the node has a private key of a key pair matching the given ID.
##### Parameters
Parameters
- **string** - ID of the Key pair.
##### Response
Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** -
`true` or
`false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getPublicKey`
Returns the public key for an ID.
##### Parameters
Parameters
- **string** - ID of the Key.
##### Response
Response
- **string** - The public key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - The public key or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getPrivateKey`
Returns the private key for an ID.
##### Parameters
Parameters
- **string** - ID of the Key.
##### Response
Response
- **string** - The private key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - The private key or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_newSymKey`
Generates a random symmetric key and stores it under an ID. This key can be used to encrypt and decrypt messages where the key is known to both parties.
Generates a random symmetric key and stores it under an ID.
This key can be used to encrypt and
decrypt messages where the key is known to both parties.
##### Parameters
Parameters
none
##### Response
Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - The key ID or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_addSymKey`
Stores the key and returns its ID.
##### Parameters
Parameters
- **string** - The raw key for symmetric encryption hex encoded.
##### Response
Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - The key ID or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_generateSymKeyFromPassword`
Generates the key from a password and stores it.
##### Parameters
Parameters
- **string** - The password.
- **string** - The password.
##### Response
Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object)
on failure.
#### `waku_hasSymKey`
Returns whether there is a key associated with the ID.
##### Parameters
Parameters
- **string** - ID of the Key.
##### Response
Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object)
on failure.
#### `waku_getSymKey`
Returns the symmetric key associated with an ID.
##### Parameters
Parameters
- **string** - ID of the Key.
##### Response
Response
- **string** - Raw key on success or an [error](https://www.jsonrpc.org/specification#error_object) of failure.
- **string** - Raw key on success or
an [error](https://www.jsonrpc.org/specification#error_object) of failure.
#### `waku_deleteSymKey`
Deletes the key associated with an ID.
##### Parameters
Parameters
- **string** - ID of the Key.
##### Response
Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** -
`true` or `false` or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_subscribe`
Creates and registers a new subscription to receive notifications for inbound Waku messages.
Creates and
registers a new subscription to receive notifications for inbound Waku messages.
##### Parameters
Parameters
The parameters for this request is an array containing the following fields:
1. **string** - The ID of the function call, in case of Waku this must contain the value "messages".
1. **string** - The ID of the function call,
in case of Waku this must contain the value "messages".
2. **object** - The [message filter](#filter).
##### Response
Response
- **string** - ID of the subscription or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **string** - ID of the subscription or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
###### Notifications
Notifications
Notifications received by the client contain a [message](#message) matching the filter. Below is an example notification:
Notifications received by the client contain a [message](#message) matching the filter.
Below is an example notification:
```json
{
@@ -328,81 +369,90 @@ Notifications received by the client contain a [message](#message) matching the
#### `waku_unsubscribe`
Cancels and removes an existing subscription. The node MUST stop sending the client notifications.
Cancels and removes an existing subscription.
The node MUST stop sending the client notifications.
##### Parameters
Parameters
- **string** - The subscription ID.
- **string** - The subscription ID.
##### Response
Response
- **bool** - `true` or `false`
#### `waku_newMessageFilter`
Creates a new message filter within the node. This filter can be used to poll for new messages that match the criteria.
Creates a new message filter within the node.
This filter can be used to poll for new messages that match the criteria.
##### Parameters
Parameters
The request must contain a [message filter](#filter) as its parameter.
The request must contain a [message filter](#filter) as its parameter.
##### Response
Response
- **string** - The ID of the filter.
- **string** - The ID of the filter.
#### `waku_deleteMessageFilter`
Removes a message filter from the node.
##### Parameters
Parameters
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newmessagefilter).
##### Response
Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
- **bool** - `true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getFilterMessages`
Retrieves messages that match a filter criteria and were received after the last time this function was called.
Retrieves messages that match a filter criteria and
were received after the last time this function was called.
##### Parameters
Parameters
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newmessagefilter).
##### Response
Response
The response contains an array of [messages](#messages) or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
The response contains an array of [messages](#message) or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_post`
The `waku_post` method creates a waku envelope and propagates it to the network.
##### Parameters
Parameters
The parameters is an `Object` containing the following fields:
- **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
- **`pubKey` [string]** `optional` - The public key for message encryption.
- **`sig` [string]** `optional` - The ID of the signing key.
- **`ttl` [number]** - The time-to-live in seconds.
- **`topic` [string]** - 4 bytes message topic.
- **`payload` [string]** - The payload to be encrypted.
- **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
- **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
- **`powTarget` [number]** - Minimal PoW target required for this message.
- **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
*Either the **`symKeyID`** or the **`pubKey`** need to be present. It can not be both.*
#### Response
- **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
- **`pubKey` [string]** `optional` - The public key for message encryption.
- **`sig` [string]** `optional` - The ID of the signing key.
- **`ttl` [number]** - The time-to-live in seconds.
- **`topic` [string]** - 4 bytes message topic.
- **`payload` [string]** - The payload to be encrypted.
- **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
- **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
- **`powTarget` [number]** - Minimal PoW target required for this message.
- **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
*Either the **`symKeyID`** or the **`pubKey`** need to be present.
It can not be both.*
Response
- **bool** -
`true` on success or
an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Changelog
| Version | Comment |
| :--------------------------------------------------------------------------------------:| ---------------- |
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. |
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. |
## Copyright