vacp2p/rfc.vac.dev
1
0
Fork 0
mirror of https://github.com/vacp2p/rfc.vac.dev.git synced 2026-01-09 22:27:59 -05:00
Code Issues Packages Projects Releases Wiki Activity

add fetch script and populate content

Browse Source
This commit is contained in:
jinhojang6
2024-03-13 01:12:23 +09:00
parent 2b10d5086b
commit 6955a5443e
82 changed files with 15368 additions and 141 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
waku/README.md Normal file
View File

@@ -0,0 +1,5 @@
## Waku RFCs
Waku builds a family of privacy-preserving, censorship-resistant communication protocols for web3 applications.
Contributors can visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.

555
waku/deprecated/5/waku0.md Normal file
View File

@@ -0,0 +1,555 @@
---
title: 5/WAKU0
name: Waku v0
status: deprecated
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Adam Babik \<adam@status.im\>
- Andrea Maria Piana \<andreap@status.im\>
- Dean Eigenmann \<dean@status.im\>
- Kim De Mey \<kimdemey@status.im\>
sidebar_position: 1
---
This specification describes the format of Waku messages within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic messages (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
## Motivation
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku messages in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
## Definitions
| Term | Definition |
| --------------- | ----------------------------------------------------- |
| **Light node** | A Waku node that does not forward any messages. |
| **Envelope** | Messages sent and received by Waku nodes. |
| **Node** | Some process that is able to communicate for Waku. |
## Underlying Transports and Prerequisites
### Use of DevP2P
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
### Gossip based routing
In Whisper, messages are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Messages are eligible for retransmission until their TTL expires. A node SHOULD relay messages to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [mailserver](./mailserver.md) response. A node SHOULD NOT send a message to a peer that it has already sent before.
## Wire Specification
### Use of RLPx transport protocol
All Waku messages are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
### ABNF specification
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
```abnf
; Packet codes 0 - 127 are reserved for Waku protocol
packet-code = 1*3DIGIT
; rate limits
limit-ip = 1*DIGIT
limit-peerid = 1*DIGIT
limit-topic = 1*DIGIT
rate-limits = "[" limit-ip limit-peerid limit-topic "]"
pow-requirement-key = 48
bloom-filter-key = 49
light-node-key = 50
confirmations-enabled-key = 51
rate-limits-key = 52
topic-interest-key = 53
status-options = "["
[ pow-requirement-key pow-requirement ]
[ bloom-filter-key bloom-filter ]
[ light-node-key light-node ]
[ confirmations-enabled-key confirmations-enabled ]
[ rate-limits-key rate-limits ]
[ topic-interest-key topic-interest ]
"]"
status = "[" version status-options "]"
status-update = status-options
; version is "an integer (as specified in RLP)"
version = DIGIT
confirmations-enabled = BIT
light-node = BIT
; pow is "a single floating point value of PoW.
; This value is the IEEE 754 binary representation
; of a 64-bit floating point number.
; Values of qNAN, sNAN, INF and -INF are not allowed.
; Negative values are also not allowed."
pow = 1*DIGIT "." 1*DIGIT
pow-requirement = pow
; bloom filter is "a byte array"
bloom-filter = *OCTET
waku-envelope = "[" expiry ttl topic data nonce "]"
; List of topics interested in
topic-interest = "[" *10000topic "]"
; 4 bytes (UNIX time in seconds)
expiry = 4OCTET
; 4 bytes (time-to-live in seconds)
ttl = 4OCTET
; 4 bytes of arbitrary data
topic = 4OCTET
; byte array of arbitrary size
; (contains encrypted message)
data = OCTET
; 8 bytes of arbitrary data
; (used for PoW calculation)
nonce = 8OCTET
messages = 1*waku-envelope
; mail server / client specific
p2p-request = waku-envelope
p2p-message = 1*waku-envelope
; packet-format needs to be paired with its
; corresponding packet-format
packet-format = "[" packet-code packet-format "]"
required-packet = 0 status /
1 messages /
22 status-update /
optional-packet = 126 p2p-request / 127 p2p-message
packet = "[" required-packet [ optional-packet ] "]"
```
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
### Packet Codes
The message codes reserved for Waku protocol: 0 - 127.
Messages with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
| Name | Int Value |
| -------------------------- | ------------- |
| Status | 0 |
| Messages | 1 |
| Status Update | 22 |
The following message codes are optional, but they are reserved for specific purpose.
| Name | Int Value | Comment |
|----------------------------|-----------|---------|
| Batch Ack | 11 | |
| Message Response | 12 | |
| P2P Request | 126 | |
| P2P Message | 127 | |
### Packet usage
#### Status
The Status message serves as a Waku handshake and peers MUST exchange this
message upon connection. It MUST be sent after the RLPx handshake and prior to
any other Waku messages.
A Waku node MUST await the Status message from a peer before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status message from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status message, the node SHOULD validate the message
received and validated the Status message. Note that its peer might not be in
the same state.
When a node is receiving other Waku messages from a peer before a Status
message is received, the node MUST ignore these messages and SHOULD disconnect from that peer. Status messages received after the handshake is completed MUST also be ignored.
The status message MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
#### Messages
This packet is used for sending the standard Waku envelopes.
#### Status Update
The Status Update message is used to communicate an update of the settings of the node.
The format is the same as the Status message, all fields are optional.
If none of the options are specified the message MUST be ignored and considered a noop.
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
be transmitted.
**PoW Requirement update**
When PoW is updated, peers MUST NOT deliver the sender envelopes with PoW lower than specified in this message.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by message size and TTL:
PoW = (2**BestBit) / (size * TTL)
PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
**Bloom filter update**
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
If both Topic Interest and bloom filter are specified, Topic Interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current Topic Interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
**Topic Interest update**
This packet is used by Waku nodes for sharing their interest in messages with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only Topic Interest is specified, the current bloom filter MUST be discarded and only the updated Topic Interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node is not currently interested in receiving any envelope.
**Rate Limits update**
This packet is used for informing other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the rate limits packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
**Message Confirmations update**
Message confirmations tell a node that a message originating from it has been received by its peers, allowing a node to know whether a message has or has not been received.
A node MAY send a message confirmation for any batch of messages received with a packet Messages Code.
A message confirmation is sent using Batch Acknowledge packet or Message Response packet. The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data.
The current `version` of the message response is `1`.
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
```abnf
; a version of the Message Response
version = 1*DIGIT
; keccak256 hash of the envelopes batch data (raw bytes) for which the confirmation is sent
hash = *OCTET
hasherror = *OCTET
; error code
code = 1*DIGIT
; a descriptive error message
description = *ALPHA
error = "[" hasherror code description "]"
errors = *error
response = "[" hash errors "]"
confirmation = "[" version response "]"
```
The supported codes:
`1`: means time sync error which happens when an envelope is too old or created in the future (the root cause is no time sync between nodes).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcast by one or more peers.
#### P2P Request
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting old messages from the [Waku Mail Server](./mailserver.md).
#### P2P Message
This packet is used for sending the peer-to-peer messages, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of old (expired) messages, which is otherwise not allowed.
### Payload Encryption
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
### Packet code Rationale
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
Packet code `0x22` is used to dynamically change the settings of a node.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without P2P messages it would be impossible to deliver the old messages, since they will be recognized as expired, and the peer will be disconnected for violating the Whisper protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
## Additional capabilities
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on [specification](mailserver.md).
### Light node
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming messages, they MUST only send their own messages. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in messages being dropped between the two.
Light nodes are identified by the `light_node` value in the status message.
### Accounting for resources (experimental)
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
| peer | sent | received |
|-------|------|----------|
| peer1 | 0 | 123 |
| peer2 | 10 | 40 |
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
## Upgradability and Compatibility
### General principles and policy
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
1. Waku aims to be compatible with previous and future versions.
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
3. To achieve this, we employ the following two general strategies:
- a) Accretion (including protocol negotiation) over changing data
- b) When we want to change things, we give it a new name (for example, a version number).
Examples:
- We enable bridging between `shh/6` and `waku/0` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
- When we we want to provide a new set of messages that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
### Backwards Compatibility
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
### Waku-Whisper bridging
`waku/0` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
**Roles:**
- Waku client A, only Waku capability
- Whisper client B, only Whisper capability
- WakuWhisper bridge C, both Waku and Whisper capability
**Flow:**
1. A posts message; B posts message.
2. C picks up message from A and B and relays them both to Waku and Whisper.
3. A receives message on Waku; B on Whisper.
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a message between C1 and C2. I.e. Whisper(\<\>Waku\<\>Whisper)\<\>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
### Forward Compatibility
It is desirable to have a strategy for maintaining forward compatibility between `waku/0` and future version of waku. Here we outline some concerns and strategy for this.
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished in the future is by negotiating the versions of subprotocols, within the `hello` message nodes transmit their capabilities along with a version. As suggested in [EIP-8](https://eips.ethereum.org/EIPS/eip-8), if a node connects that has a higher version number for a specific capability, the node with a lower number SHOULD assume backwards compatibility. The node with the higher version will decide if compatibility can be assured between versions, if this is not the case it MUST disconnect.
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
## Appendix A: Security considerations
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](./mailserver.md#security-considerations) can be found in their respective specifications.
### Scalability and UX
**Bandwidth usage:**
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
**Gossip-based routing:**
Use of gossip-based routing doesn't necessarily scale. It means each node can see a message multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
**Lack of incentives:**
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
### Privacy
**Light node privacy:**
The main privacy concern with light nodes is that directly connected peers will know that a message originates from them (as it are the only ones it sends). This means nodes can make assumptions about what messages (topics) their peers are interested in.
**Bloom filter privacy:**
By having a bloom filter where only the topics you are interested in are set, you reveal which messages you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
**Privacy guarantees not rigorous:**
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
**Topic hygiene:**
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
### Spam resistance
**PoW bad for heterogeneous devices:**
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
### Censorship resistance
**Devp2p TCP port blockable:**
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
## Appendix B: Implementation Notes
### Implementation Matrix
| Client | Spec supported | Details |
|--------|----------------|---------|
| **Status-go** | 0.5 | [details](https://github.com/status-im/status-go/blob/develop/WAKU.md) |
| **Nimbus** | 0.4 | [details](https://github.com/status-im/nimbus/tree/8747fe1ecd36fe778bb92b97634db84d364fede8/waku) |
### Recommendations for clients
Notes useful for implementing Waku mode.
1. Avoid duplicate envelopes
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
2. Topic specific recommendations
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
### Node discovery
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
Known static nodes MAY also be used.
## Changelog
### Version 0.6
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
- Mark spec as Deprecated mode in terms of its lifecycle.
### Version 0.5
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
### Version 0.4
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
- Simplify implementation matrix with latest state
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
- Increased `topic-interest` capacity from 1000 to 10000
### Version 0.3
Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954ab4c315e1b8d210ac7631b6d1428).
- Recommend DNS based node discovery over other Discovery methods.
- Mark spec as Draft mode in terms of its lifecycle.
- Simplify Changelog and misc formatting.
- Handshake/Status message not compatible with shh/6 nodes; specifying options as association list.
- Include topic-interest in Status handshake.
- Upgradability policy.
- `topic-interest` packet code.
### Version 0.2
Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/waku.md).
- General style improvements.
- Fix ABNF grammar.
- Mailserver requesting/receiving.
- New packet codes: topic-interest (experimental), rate limits (experimental).
- More details on handshake modifications.
- Accounting for resources mode (experimental)
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
- More details on handshake and parameters.
- Describe rate limits in more detail.
- More details on mailserver and mail client API.
- Accounting for resources mode (very experimental).
- Clarify differences with Whisper.
### Version 0.1
Initial version. Released [November 21, 2019](https://github.com/vacp2p/specs/blob/b59b9247f2ac1bf45c75bd3227a2e5dd87b6d7b0/waku.md).
### Differences between shh/6 and waku/0
Summary of main differences between this spec and Whisper v6, as described in [EIP-627](https://eips.ethereum.org/EIPS/eip-627):
- RLPx subprotocol is changed from `shh/6` to `waku/0`.
- Light node capability is added.
- Optional rate limiting is added.
- Status packet has following additional parameters: light-node,
confirmations-enabled and rate-limits
- Mail Server and Mail Client functionality is now part of the specification.
- P2P Message packet contains a list of envelopes instead of a single envelope.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## Footnotes
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.

6
waku/deprecated/README.md Normal file
View File

@@ -0,0 +1,6 @@
## Deprecated RFCs
Deprecated specifications are no longer used in Waku products.
This subdirectory is for achrive purpose and
should not be used in production ready implementations.
Visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.

52
waku/informational/22/toy-chat.md Normal file
View File

@@ -0,0 +1,52 @@
---
title: 22/TOY-CHAT
name: Waku v2 Toy Chat
status: draft
editor: Franck Royer \<franck@status.im\>
contributors:
- Hanno Cornelius \<hanno@status.im\>
sidebar_position: 1
---
**Content Topic**: `/toy-chat/2/huilong/proto`.
This specification explains a toy chat example using Waku v2.
This protocol is mainly used to:
1. Dogfood Waku v2,
2. Show an example of how to use Waku v2.
Currently, all main Waku v2 implementations support the toy chat protocol:
[nim-waku](https://github.com/status-im/nim-waku/blob/master/examples/v2/chat2.nim),
js-waku ([NodeJS](https://github.com/status-im/js-waku/tree/main/examples/cli-chat) and [web](https://github.com/status-im/js-waku/tree/main/examples/web-chat))
and [go-waku](https://github.com/status-im/go-waku/tree/master/examples/chat2).
Note that this is completely separate from the protocol the Status app is using for its chat functionality.
# Design
The chat protocol enables sending and receiving messages in a chat room.
There is currently only one chat room, which is tied to the content topic.
The messages SHOULD NOT be encrypted.
The `contentTopic` MUST be set to `/toy-chat/2/huilong/proto`.
# Payloads
```protobuf
syntax = "proto3";
message Chat2Message {
uint64 timestamp = 1;
string nick = 2;
bytes payload = 3;
}
```
- `timestamp`: The time at which the message was sent, in Unix Epoch seconds,
- `nick`: The nickname of the user sending the message,
- `payload`: The text of the messages, UTF-8 encoded.
# Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

170
waku/informational/23/topics.md Normal file
View File

@@ -0,0 +1,170 @@
---
title: 23/WAKU2-TOPICS
name: Waku v2 Topic Usage Recommendations
status: draft
category: Informational
editor: Oskar Thoren \<oskarth@titanproxy.com\>
contributors:
- Hanno Cornelius \<hanno@status.im\>
- Daniel Kaiser \<danielkaiser@status.im\>
sidebar_position: 1
---
This document outlines recommended usage of topic names in Waku v2.
In [10/WAKU2 spec](../../standards/core/10/waku2.md) there are two types of topics:
- pubsub topics, used for routing
- Content topics, used for content-based filtering
## Pubsub Topics
Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](../../standards/core/11/relay.md)),
and can be named implicitly by Waku sharding (see [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
This document comprises recommendations for explicitly naming pubsub topics (e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
### Pubsub Topic Format
Pubsub topics SHOULD follow the following structure:
`/waku/2/{topic-name}`
This namespaced structure makes compatibility, discoverability, and automatic handling of new topics easier.
The first two parts indicate
1) it relates to the Waku protocol domain, and
2) the version is 2.
If applicable, it is RECOMMENDED to structure `{topic-name}` in a hierarchical way as well.
\> *Note*: In previous versions of this document, the structure was `/waku/2/{topic-name}/{encoding}`.
The now deprecated `/{encoding}` was always set to `/proto`,
which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition) in pubsub is serialized/encoded as protobuf.
The inspiration for this format was taken from
[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md) must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
which specifies the wire format as protobuf,`/proto` is the only valid encoding.
This makes the `/proto` indication obsolete.
The encoding of the `payload` field of a Waku Message is indicated by the `/{encoding}` part of the content topic name.
Specifying an encoding is only significant for the actual payload/data field.
Waku preserves this option by allowing to specify an encoding for the WakuMessage payload field as part of the content topic name.
### Default PubSub Topic
The Waku v2 default pubsub topic is:
`/waku/2/default-waku/proto`
The `{topic name}` part is `default-waku/proto`, which indicates it is default topic for exchanging WakuMessages;
`/proto` remains for backwards compatibility.
### Application Specific Names
Larger apps can segregate their pubsub meshes using topics named like:
```
/waku/2/status/
/waku/2/walletconnect/
```
This indicates that these networks carry WakuMessages, but for different domains completely.
### Named Topic Sharding Example
The following is an example of named sharding, as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md).
```
waku/2/waku-9_shard-0/
...
waku/2/waku-9_shard-9/
```
This indicates explicitly that the network traffic has been partitioned into 10 buckets.
## Content Topics
The other type of topic that exists in Waku v2 is a content topic.
This is used for content based filtering.
See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md) for where this is specified.
Note that this doesn't impact routing of messages between relaying nodes,
but it does impact how request/reply protocols such as
[12/WAKU2-FILTER](../../standards/core/14/filter.md) and [13/WAKU2-STORE](../../standards/core/13/store.md) are used.
This is especially useful for nodes that have limited bandwidth,
and only want to pull down messages that match this given content topic.
Since all messages are relayed using the relay protocol regardless of content topic,
you MAY use any content topic you wish without impacting how messages are relayed.
### Content Topic Format
The format for content topics is as follows:
`/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}`
The name of a content topic is application-specific.
As an example, here's the content topic used for an upcoming testnet:
`/toychat/2/huilong/proto`
### Content Topic Naming Recommendations
Application names should be unique to avoid conflicting issues with other protocols.
Applications should specify their version (if applicable) in the version field.
The `{content-topic-name}` portion of the content topic is up to the application,
and depends on the problem domain.
It can be hierarchical, for instance to separate content, or to indicate different bandwidth and privacy guarantees.
The encoding field indicates the serialization/encoding scheme for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
## Differences with Waku v1
In [5/WAKU1](../../deprecated/5/waku0.md) there is no actual routing.
All messages are sent to all other nodes.
This means that we are implicitly using the same pubsub topic that would be something like:
```
/waku/1/default-waku/rlp
```
Topics in Waku v1 correspond to Content Topics in Waku v2.
### Bridging Waku v1 and Waku v2
To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](../../standards/core/15/bridge.md).
For mapping Waku v1 topics to Waku v2 content topics,
the following structure for the content topic SHOULD be used:
```
/waku/1/\<4bytes-waku-v1-topic\>/rfc26
```
The `\<4bytes-waku-v1-topic\>` SHOULD be the lowercase hex representation of the 4-byte Waku v1 topic.
A `0x` prefix SHOULD be used.
`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](../../standards/application/26/payload.md).
See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description of the bridged fields.
This creates a direct mapping between the two protocols.
For example:
```
/waku/1/0x007f80ff/rfc26
```
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
* [10/WAKU2 spec](../../standards/core/10/waku2.md)
* [11/WAKU2-RELAY](../../standards/core/11/relay.md)
* [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)
* [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
* [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
* [12/WAKU2-FILTER](../../standards/core/14/filter.md)
* [13/WAKU2-STORE](../../standards/core/13/store.md)
* [6/WAKU1](../../deprecated/5/waku0.md)
* [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
* [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)

99
waku/informational/27/peers.md Normal file
View File

@@ -0,0 +1,99 @@
---
title: 27/WAKU2-PEERS
name: Waku v2 Client Peer Management Recommendations
status: draft
editor: Hanno Cornelius \<hanno@status.im\>
contributors:
sidebar_position: 1
---
`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and peer management features to be implemented by Waku v2 clients.
In this context, peer _storage_ refers to a client's ability to keep track of discovered or statically-configured peers and their metadata.
It also deals with matters of peer _persistence_,
or the ability to store peer data on disk to resume state after a client restart.
Peer _management_ is a closely related concept and refers to the set of actions a client MAY choose to perform based on its knowledge of its connected peers,
e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
## Peer store
The peer store SHOULD be an in-memory data structure where information about discovered or configured peers are stored.
It SHOULD be considered the main source of truth for peer-related information in a Waku v2 client.
Clients MAY choose to persist this store on-disk.
### Tracked peer metadata
It is RECOMMENDED that a Waku v2 client tracks at least the following information about each of its peers in a peer store:
| Metadata | Description |
| --- | --- |
| _Public key_ | The public key for this peer. This is related to the libp2p [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/). |
| _Addresses_ | Known transport layer [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/) for this peer. |
| _Protocols_ | The libp2p [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids) supported by this peer. This can be used to track the client's connectivity to peers supporting different Waku v2 protocols, e.g. [`11/WAKU2-RELAY`](../../standards/core/11/relay.md) or [`13/WAKU2-STORE`](../../standards/core/13/store.md). |
| _Connectivity_ | Tracks the peer's current connectedness state. See [**Peer connectivity**](#peer-connectivity) below. |
| _Disconnect time_ | The timestamp at which this peer last disconnected. This becomes important when managing [peer reconnections](#reconnecting-peers) |
### Peer connectivity
A Waku v2 client SHOULD track _at least_ the following connectivity states for each of its peers:
- **`NotConnected`**: The peer has been discovered or configured on this client,
but no attempt has yet been made to connect to this peer.
This is the default state for a new peer.
- **`CannotConnect`**: The client attempted to connect to this peer, but failed.
- **`CanConnect`**: The client was recently connected to this peer and disconnected gracefully.
- **`Connected`**: The client is actively connected to this peer.
This list does not preclude clients from tracking more advanced connectivity metadata,
such as a peer's blacklist status (see [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)).
### Persistence
A Waku v2 client MAY choose to persist peers across restarts,
using any offline storage technology, such as an on-disk database.
Peer persistence MAY be used to resume peer connections after a client restart.
## Peer management
Waku v2 clients will have different requirements when it comes to managing the peers tracked in the [**peer store**](#peer-store).
It is RECOMMENDED that clients support:
- [automatic reconnection](#reconnecting-peers) to peers under certain conditions
- [connection keep-alive](#connection-keep-alive)
### Reconnecting peers
A Waku v2 client MAY choose to reconnect to previously connected, managed peers under certain conditions.
Such conditions include, but are not limited to:
- Reconnecting to all `relay`-capable peers after a client restart. This will require [persistent peer storage](#persistence).
If a client chooses to automatically reconnect to previous peers,
it MUST respect the [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange) specified for GossipSub v1.1 before attempting to reconnect.
This requires keeping track of the [last time each peer was disconnected](#tracked-peer-metadata).
### Connection keep-alive
A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers.
If a client chooses to implement keep-alive on a connection,
it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping) as per `10/WAKU2` [client recommendations](../../standards/core/10/WAKU2.md/#recommendations-for-clients).
The recommended period between pings SHOULD be _at most_ 50% of the shortest idle connection timeout for the specific client and transport.
For example, idle TCP connections often times out after 10 to 15 minutes.
\> **Implementation note:** the `nim-waku` client currently implements a keep-alive mechanism every `5 minutes`,
in response to a TCP connection timeout of `10 minutes`.
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/)
- [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/)
- [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids)
- [`11/WAKU2-RELAY`](../../standards/core/11/relay.md)
- [`13/WAKU2-STORE`](../../standards/core/13/store.md)
- [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)
- [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
- [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
- [`10/WAKU2` client recommendations](https://rfc.vac.dev/spec/10/#recommendations-for-clients)

74
waku/informational/29/config.md Normal file
View File

@@ -0,0 +1,74 @@
---
title: 29/WAKU2-CONFIG
name: Waku v2 Client Parameter Configuration Recommendations
status: draft
editor: Hanno Cornelius \<hanno@status.im\>
contributors:
sidebar_position: 1
---
`29/WAKU2-CONFIG` describes the RECOMMENDED values to assign to configurable parameters for Waku v2 clients.
Since Waku v2 is built on [libp2p](https://github.com/libp2p/specs),
most of the parameters and reasonable defaults are derived from there.
Waku v2 relay messaging is specified in [`11/WAKU2-RELAY`](../../standards/core/11/relay.md),
a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
GossipSub behaviour is controlled by a series of adjustable parameters.
Waku v2 clients SHOULD configure these parameters to the recommended values below.
## GossipSub v1.0 parameters
GossipSub v1.0 parameters are defined in the [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters).
We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
| Parameter | Purpose | RECOMMENDED value |
|----------------------|-------------------------------------------------------|-------------------|
| `D` | The desired outbound degree of the network | 6 |
| `D_low` | Lower bound for outbound degree | 4 |
| `D_high` | Upper bound for outbound degree | 8 |
| `D_lazy` | (Optional) the outbound degree for gossip emission | `D` |
| `heartbeat_interval` | Time between heartbeats | 1 second |
| `fanout_ttl` | Time-to-live for each topic's fanout state | 60 seconds |
| `mcache_len` | Number of history windows in message cache | 5 |
| `mcache_gossip` | Number of history windows to use when emitting gossip | 3 |
| `seen_ttl` | Expiry time for cache of seen message ids | 2 minutes |
## GossipSub v1.1 parameters
GossipSub v1.1 extended GossipSub v1.0 and introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
We repeat the global parameters here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
| Parameter | Description | RECOMMENDED value |
|----------------|------------------------------------------------------------------------|-------------------|
| `PruneBackoff` | Time after pruning a mesh peer before we consider grafting them again. | 1 minute |
| `FloodPublish` | Whether to enable flood publishing | true |
| `GossipFactor` | % of peers to send gossip to, if we have more than `D_lazy` available | 0.25 |
| `D_score` | Number of peers to retain by score when pruning from oversubscription | `D_low` |
| `D_out` | Number of outbound connections to keep in the mesh. | `D_low` - 1 |
`11/WAKU2-RELAY` clients SHOULD implement a peer scoring mechanism with the parameter constraints as [specified by libp2p](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
## Other configuration
The following behavioural parameters are not specified by `libp2p`,
but nevertheless describes constraints that `11/WAKU2-RELAY` clients MAY choose to implement.
| Parameter | Description | RECOMMENDED value |
|--------------------|---------------------------------------------------------------------------|-------------------|
| `BackoffSlackTime` | Slack time to add to prune backoff before attempting to graft again | 2 seconds |
| `IWantPeerBudget` | Maximum number of IWANT messages to accept from a peer within a heartbeat | 25 |
| `IHavePeerBudget` | Maximum number of IHAVE messages to accept from a peer within a heartbeat | 10 |
| `IHaveMaxLength` | Maximum number of messages to include in an IHAVE message | 5000 |
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [libp2p](https://github.com/libp2p/specs)
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
- [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
- [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters)
- [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters)

102
waku/informational/30/adaptive-nodes.md Normal file
View File

@@ -0,0 +1,102 @@
---
title: 30/ADAPTIVE-NODES
name: Adaptive nodes
status: draft
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
sidebar_position: 1
---
This is an informational spec that show cases the concept of adaptive nodes.
## Node types - a continuum
We can look at node types as a continuum, from more restricted to less restricted, fewer resources to more resources.
![Node types - a continuum](./images/adaptive_node_continuum2.png)
### Possible limitations
- Connectivity: Not publicly connectable vs static IP and DNS
- Connectivity: Mostly offline to mostly online to always online
- Resources: Storage, CPU, Memory, Bandwidth
### Accessibility and motivation
Some examples:
- Opening browser window: costs nothing, but contribute nothing
- Desktop: download, leave in background, contribute somewhat
- Cluster: expensive, upkeep, but can contribute a lot
These are also illustrative, so a node in a browser in certain environment might contribute similarly to Desktop.
### Adaptive nodes
We call these nodes *adaptive nodes* to highlights different modes of contributing, such as:
- Only leeching from the network
- Relaying messages for one or more topics
- Providing services for lighter nodes such as lightpush and filter
- Storing historical messages to various degrees
- Ensuring relay network can't be spammed with RLN
### Planned incentives
Incentives to run a node is currently planned around:
- SWAP for accounting and settlement of services provided
- RLN RELAY for spam protection
- Other incentivization schemes are likely to follow and is an area of active research
## Node protocol selection
Each node can choose which protocols to support, depending on its resources and goals.
![Protocol selection](./images/adaptive_node_protocol_selection2.png)
In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md) etc (12, 13, 19, 21) these correspond to Libp2p protocols.
However, other protocols like 16/WAKU2-RPC (local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY, Discovery v5 (DevP2P) or interfacing with distributed storage, are running on different network stacks.
This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE, 26/WAKU2-PAYLOAD, or application specific ones. As well as specs that act more as recommendations, such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
## Waku network visualization
We can better visualize the network with some illustrative examples.
### Topology and topics
The first one shows an example topology with different PubSub topics for the relay protocol.
![Waku Network visualization](./images/adaptive_node_network_topology_protocols2.png)
### Legend
![Waku Network visualization legend](./images/adaptive_node_network_topology_protocols_legend.png)
The dotted box shows what content topics (application-specific) a node is interested in.
A node that is purely providing a service to the network might not care.
In this example, we see support for toy chat, a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
### Auxiliary network
This is a separate component with its own topology.
Behavior and interaction with other protocols specified in Vac RFCs, e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
### Node Cross Section
This one shows a cross-section of nodes in different dimensions and shows how the connections look different for different protocols.
![Node Cross Section](./images/adaptive_node_cross_section2.png)
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)

BIN
waku/informational/30/images/adaptive_network_topology_protocols.jpg Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_continuum.jpg Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_continuum2.png Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_cross_section.jpg Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_cross_section2.png Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_network_topology_protocols2.png Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_network_topology_protocols_legend.png Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_node_protocol_selection2.png Normal file
View File

Binary file not shown.

BIN
waku/informational/30/images/adaptive_protocol_selection.jpg Normal file
View File

Binary file not shown.

188
waku/standards/application/18/swap.md Normal file
View File

@@ -0,0 +1,188 @@
---
title: 18/WAKU2-SWAP
name: Waku SWAP Accounting
status: draft
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributor: Ebube Ud \<ebube@status.im\>
sidebar_position: 1
---
## Abstract
This specification outlines how we do accounting and settlement based on the provision and usage of resources, most immediately bandwidth usage and/or storing and retrieving of Waku message. This enables nodes to cooperate and efficiently share resources, and in the case of unequal nodes to settle the difference through a relaxed payment mechanism in the form of sending cheques.
**Protocol identifier***: `/vac/waku/swap/2.0.0-beta1`
## Motivation
The Waku network makes up a service network, and some nodes provide a useful service to other nodes. We want to account for that, and when imbalances arise, settle this. The core of this approach has some theoretical backing in game theory, and variants of it have practically been proven to work in systems such as Bittorrent. The specific model use was developed by the Swarm project (previously part of Ethereum), and we re-use contracts that were written for this purpose.
By using a delayed payment mechanism in the form of cheques, a barter-like mechanism can arise, and nodes can decide on their own policy as opposed to be strictly tied to a specific payment scheme. Additionally, this delayed settlement eases requirements on the underlying network in terms of transaction speed or costs.
Theoretically, nodes providing and using resources over a long, indefinite, period of time can be seen as an iterated form of [Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma). Specifically, and more intuitively, since we have a cost and benefit profile for each provision/usage (of Waku Message's, e.g.), and the pricing can be set such that mutual cooperation is incentivized, this can be analyzed as a form of donations game.
## Game Theory - Iterated prisoner's dilemma / donation game
What follows is a sketch of what the game looks like between two nodes. We can look at it as a special case of iterated prisoner's dilemma called a [Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game) where each node can cooperate with some benefit `b` at a personal cost `c`, where `b\>c`.
From A's point of view:
A/B | Cooperate | Defect
-----|----------|-------
Cooperate | b-c | -c
Defect | b | 0
What this means is that if A and B cooperates, A gets some benefit `b` minus a cost `c`. If A cooperates and B defects she only gets the cost, and if she defects and B cooperates A only gets the benefit. If both defect they get neither benefit nor cost.
The generalized form of PD is:
A/B | Cooperate | Defect
-----|----------|-------
Cooperate | R | S
Defect | T | P
With R=reward, S=Sucker's payoff, T=temptation, P=punishment
And the following holds:
- `T\>R\>P\>S`
- `2R\>T+S`
In our case, this means `b\>b-c\>0\>-c` and `2(b-c)\> b-c` which is trivially true.
As this is an iterated game with no clear finishing point in most circumstances, a tit-for-tat strategy is simple, elegant and functional. To be more theoretically precise, this also requires reasonable assumptions on error rate and discount parameter. This captures notions such as "does the perceived action reflect the intended action" and "how much do you value future (uncertain) actions compared to previous actions". See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation) for more details. In specific circumstances, nodes can choose slightly different policies if there's a strong need for it. A policy is simply how a node chooses to act given a set of circumstances.
A tit-for-tat strategy basically means:
- cooperate first (perform service/beneficial action to other node)
- defect when node stops cooperating (disconnect and similar actions), i.e. when it stops performing according to set parameters re settlement
- resume cooperation if other node does so
This can be complemented with node selection mechanisms.
## SWAP protocol overview
We use SWAP for accounting and settlement in conjunction with other request/reply protocols in Waku v2,
where accounting is done in a pairwise manner.
It is an acronym with several possible meanings (as defined in the Book
of Swarm), for example:
- service wanted and provided
- settle with automated payments
- send waiver as payment
- start without a penny
This approach is based on communicating payment thresholds and sending cheques as indications of later payments.
Communicating payment thresholds MAY be done out-of-band or as part of the handshake.
Sending cheques is done once payment threshold is hit.
See [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth) section 3.2. on Peer-to-peer accounting etc., for more context and details.
### Accounting
Nodes perform their own accounting for each relevant peer based on some "volume"/bandwidth metric. For now we take this to mean the number of `WakuMessage`s exchanged.
Additionally, a price is attached to each unit. Currently, this is simply a "karma counter" and equal to 1 per message.
Each accounting balance SHOULD be w.r.t. to a given protocol it is accounting for.
NOTE: This may later be complemented with other metrics, either as part of SWAP or more likely outside of it. For example, online time can be communicated and attested to as a form of enhanced quality of service to inform peer selection.
### Flow
Assuming we have two store nodes, one operating mostly as a client (A) and another as server (B).
1. Node A performs a handshake with B node. B node responds and both nodes communicate their payment threshold.
2. Node A and B creates an accounting entry for the other peer, keep track of peer and current balance.
3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`. Based on the number of WakuMessages in the response, both nodes update their accounting records.
4. When payment threshold is reached, Node A sends over a cheque to reach a neutral balance. Settlement of this is currently out of scope, but would occur through a SWAP contract (to be specified). (mock and hard phase).
5. If disconnect threshold is reached, Node B disconnects Node A (mock and hard phase).
Note that not all of these steps are mandatory in initial stages, see below for more details. For example, the payment threshold MAY initially be set out of bounds, and policy is only activated in the mock and hard phase.
### Protobufs
We use protobuf to specify the handshake and signature. This current protobuf is a work in progress. This is needed for mock and hard phase.
A handshake gives initial information about payment thresholds and possibly other information. A cheque is best thought of as a promise to pay at a later date.
```protobuf
message Handshake {
bytes payment_threshold = 1;
}
// TODO Signature?
// Should probably be over the whole Cheque type
message Cheque {
bytes beneficiary = 1;
// TODO epoch time or block time?
uint32 date = 2;
// TODO ERC20 extension?
// For now karma counter
uint32 amount = 3;
}
```
## Incremental integration and roll-out
To incrementally integrate this into Waku v2, we have divided up the roll-out into three phases:
- Soft - accounting only
- Mock - send mock cheques and take word for it
- Hard Test - blockchain integration and deployed to public testnet (Goerli, Optimism testnet or similar)
- Hard Main - deployed to a public mainnet
An implementation MAY support any of these phases.
### Soft phase
In the soft phase only accounting is performed, without consequence for the
peers. No disconnect or sending of cheques is performed at this tage.
SWAP protocol is performed in conjunction with another request-reply protocol to account for its usage.
It SHOULD be done for [13/WAKU2-STORE](../../standards/core/13/store.md)
and it MAY be done for other request/reply protocols.
A client SHOULD log accounting state per peer
and SHOULD indicate when a peer is out of bounds (either of its thresholds met).
### Mock phase
In the mock phase, we send mock cheques and send cheques/disconnect peers as appropriate.
- If a node reaches a disconnect threshold, which MUST be outside the payment threshold, it SHOULD disconnect the other peer.
- If a node is within payment balance, the other node SHOULD stay connected to it.
- If a node receives a valid Cheque it SHOULD update its internal accounting records.
- If any node behaves badly, the other node is free to disconnect and pick another node.
- Peer rating is out of scope of this specification.
### Hard phase
In the hard phase, in addition to sending cheques and activating policy, this is
done with blockchain integration on a public testnet. More details TBD.
This also includes settlements where cheques can be redeemed.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [Prisoner's Dilemma](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma)
2. [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
3. [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth)
4. [13/WAKU2-STORE](../../standards/core/13/store.md)
\<!--
General TODOs:
- Find new link for book of swarm
- Illustrate payment and disconnection thresholds (mscgen not great for this?)
- Elaborate on how accounting works with amount in the context of e.g. store
- Illustrate flow
- Specify chequeboo
--\>

224
waku/standards/application/20/toy-eth-pm.md Normal file
View File

@@ -0,0 +1,224 @@
---
title: 20/TOY-ETH-PM
name: Toy Ethereum Private Message
status: draft
editor: Franck Royer \<franck@status.im\>
contributors:
sidebar_position: 1
---
**Content Topics**:
- Public Key Broadcast: `/eth-pm/1/public-key/proto`,
- Private Message: `/eth-pm/1/private-message/proto`.
This specification explains the Toy Ethereum Private Message protocol
which enables a peer to send an encrypted message to another peer
using the Waku v2 network, and the peer's Ethereum address.
The main purpose of this specification is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
using Ethereum accounts for identity.
This protocol caters for Web3 wallets restrictions, allowing it to be implemented only using standard Web3 API.
In the current state, the protocol has privacy and features [limitations](#limitations), has not been audited
and hence is not fit for production usage.
We hope this can be an inspiration for developers wishing to build on top of Waku v2.
## Goal
Alice wants to send an encrypted message to Bob, where only Bob can decrypt the message.
Alice only knows Bob's Ethereum Address.
## Variables
Here are the variables used in the protocol and their definition:
- `B` is Bob's Ethereum address (or account),
- `b` is the private key of `B`, and is only known by Bob.
- `B'` is Bob's Encryption Public Key, for which `b'` is the private key.
- `M` is the private message that Alice sends to Bob.
## Design Requirements
The proposed protocol MUST adhere to the following design requirements:
1. Alice knows Bob's Ethereum address,
2. Bob is willing to participate to Eth-PM, and publishes `B'`,
3. Bob's ownership of `B'` MUST be verifiable,
4. Alice wants to send message `M` to Bob,
5. Bob SHOULD be able to get `M` using [10/WAKU2 spec](../../standards/core/10/waku2.md),
6. Participants only have access to their Ethereum Wallet via the Web3 API,
7. Carole MUST NOT be able to read `M`'s content even if she is storing it or relaying it,
8. [Waku Message Version 1](../../standards/application/26/payload.md) Asymmetric Encryption is used for encryption purposes.
## Limitations
Alice's details are not included in the message's structure,
meaning that there is no programmatic way for Bob to reply to Alice
or verify her identity.
Private messages are sent on the same content topic for all users.
As the recipient data is encrypted, all participants must decrypt all messages which can lead to scalability issues.
This protocol does not guarantee Perfect Forward Secrecy nor Future Secrecy:
If Bob's private key is compromised, past and future messages could be decrypted.
A solution combining regular [X3DH](https://www.signal.org/docs/specifications/x3dh/)
bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) encryption would remove these limitations;
See the [Status secure transport spec](https://specs.status.im/spec/5) for an example of a protocol that achieves this in a peer-to-peer setting.
Bob MUST decide to participate in the protocol before Alice can send him a message.
This is discussed in more in details in [Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
## The protocol
### Generate Encryption KeyPair
First, Bob needs to generate a keypair for Encryption purposes.
Bob SHOULD get 32 bytes from a secure random source as Encryption Private Key, `b'`.
Then Bob can compute the corresponding SECP-256k1 Public Key, `B'`.
### Broadcast Encryption Public Key
For Alice to encrypt messages for Bob,
Bob SHOULD broadcast his Encryption Public Key `B'`.
To prove that the Encryption Public Key `B'` is indeed owned by the owner of Bob's Ethereum Account `B`,
Bob MUST sign `B'` using `B`.
### Sign Encryption Public Key
To prove ownership of the Encryption Public Key,
Bob must sign it using [EIP-712](https://eips.ethereum.org/EIPS/eip-712) v3,
meaning calling `eth_signTypedData_v3` on his Wallet's API.
Note: While v4 also exists,
it is not available on all wallets and the features brought by v4 is not needed for the current use case.
The `TypedData` to be passed to `eth_signTypedData_v3` MUST be as follows, where:
- `encryptionPublicKey` is Bob's Encryption Public Key, `B'`, in hex format, **without** `0x` prefix.
- `bobAddress` is Bob's Ethereum address, corresponding to `B`, in hex format, **with** `0x` prefix.
```js
const typedData = {
domain: {
chainId: 1,
name: 'Ethereum Private Message over Waku',
version: '1',
},
message: {
encryptionPublicKey: encryptionPublicKey,
ownerAddress: bobAddress,
},
primaryType: 'PublishEncryptionPublicKey',
types: {
EIP712Domain: [
{ name: 'name', type: 'string' },
{ name: 'version', type: 'string' },
{ name: 'chainId', type: 'uint256' },
],
PublishEncryptionPublicKey: [
{ name: 'encryptionPublicKey', type: 'string' },
{ name: 'ownerAddress', type: 'string' },
],
},
}
```
### Public Key Message
The resulting signature is then included in a `PublicKeyMessage`, where
- `encryption_public_key` is Bob's Encryption Public Key `B'`, not compressed,
- `eth_address` is Bob's Ethereum Address `B`,
- `signature` is the EIP-712 as described above.
```protobuf
syntax = "proto3";
message PublicKeyMessage {
bytes encryption_public_key = 1;
bytes eth_address = 2;
bytes signature = 3;
}
```
This MUST be wrapped in a Waku Message version 0, with the Public Key Broadcast content topic.
Finally, Bob SHOULD publish the message on Waku v2.
## Consideration for a non-interactive/uncoordinated protocol
Alice has to get Bob's public Key to send a message to Bob.
Because an Ethereum Address is part of the hash of the public key's account,
it is not enough in itself to deduce Bob's Public Key.
This is why the protocol dictates that Bob MUST send his Public Key first,
and Alice MUST receive it before she can send him a message.
Moreover, nim-waku, the reference implementation of [13/WAKU2-STORE](../../standards/core/13/store.md)),
stores messages for a maximum period of 30 days.
This means that Bob would need to broadcast his public key at least every 30 days to be reachable.
Below we are reviewing possible solutions to mitigate this "sign up" step.
### Retrieve the public key from the blockchain
If Bob has signed at least one transaction with his account then his Public Key can be extracted from the transaction's ECDSA signature.
The challenge with this method is that standard Web3 Wallet API does not allow Alice to specifically retrieve all/any transaction sent by Bob.
Alice would instead need to use the `eth.getBlock` API to retrieve Ethereum blocks one by one.
For each block, she would need to check the `from` value of each transaction until she finds a transaction sent by Bob.
This process is resource intensive and can be slow when using services such as Infura due to rate limits in place,
which makes it inappropriate for a browser or mobile phone environment.
An alternative would be to either run a backend that can connect directly to an Ethereum node,
use a centralized blockchain explorer
or use a decentralized indexing service such as [The Graph](https://thegraph.com/).
Note that these would resolve a UX issue only if a sender wants to proceed with _air drops_.
Indeed, if Bob does not publish his Public Key in the first place
then it can be an indication that he simply does not participate in this protocol and hence will not receive messages.
However, these solutions would be helpful if the sender wants to proceed with an _air drop_ of messages:
Send messages over Waku for users to retrieve later, once they decide to participate in this protocol.
Bob may not want to participate first but may decide to participate at a later stage
and would like to access previous messages.
This could make sense in an NFT offer scenario:
Users send offers to any NFT owner,
NFT owner may decide at some point to participate in the protocol and retrieve previous offers.
### Publishing the public in long term storage
Another improvement would be for Bob not having to re-publish his public key every 30 days or less.
Similarly to above, if Bob stops publishing his public key then it may be an indication that he does not participate in the protocol anymore.
In any case, the protocol could be modified to store the Public Key in a more permanent storage, such as a dedicated smart contract on the blockchain.
## Send Private Message
Alice MAY monitor the Waku v2 to collect Ethereum Address and Encryption Public Key tuples.
Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives are valid as per EIP-712.
She SHOULD drop any message without a signature or with an invalid signature.
Using Bob's Encryption Public Key, retrieved via [10/WAKU2](../../standards/core/10/waku2.md), Alice MAY now send an encrypted message to Bob.
If she wishes to do so, Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](../../standards/application/26/payload.md/#asymmetric).
Alice SHOULD now publish this message on the Private Message content topic.
# Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [10/WAKU2 spec](../../standards/core/10/waku2.md)
- [Waku Message Version 1](../../standards/application/26/payload.md)
- [X3DH](https://www.signal.org/docs/specifications/x3dh/)
- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
- [Status secure transport spec](https://specs.status.im/spec/5)
- [EIP-712](https://eips.ethereum.org/EIPS/eip-712)
- [13/WAKU2-STORE](../../standards/core/13/store.md))
- [The Graph](https://thegraph.com/)

74
waku/standards/application/21/fault-tolerant-store.md Normal file
View File

@@ -0,0 +1,74 @@
---
title: 21/WAKU2-FAULT-TOLERANT-STORE
name: Waku v2 Fault-Tolerant Store
status: draft
editor: Sanaz Taheri \<sanaz@status.im\>
contributors:
sidebar_position: 1
---
The reliability of [13/WAKU2-STORE](../../standards/core/13/store.md) protocol heavily relies on the fact that full nodes i.e., those who persist messages have high availability and uptime and do not miss any messages.
If a node goes offline, then it will risk missing all the messages transmitted in the network during that time.
In this specification, we provide a method that makes the store protocol resilient in presence of faulty nodes.
Relying on this method, nodes that have been offline for a time window will be able to fix the gap in their message history when getting back online.
Moreover, nodes with lower availability and uptime can leverage this method to reliably provide the store protocol services as a full node.
## Method description
As the first step towards making the [13/WAKU2-STORE](../../standards/core/13/store.md) protocol fault-tolerant, we introduce a new type of time-based query through which nodes fetch message history from each other based on their desired time window.
This method operates based on the assumption that the querying node knows some other nodes in the store protocol which have been online for that targeted time window.
## Security Consideration
The main security consideration to take into account while using this method is that a querying node has to reveal its offline time to the queried node.
This will gradually result in the extraction of the node's activity pattern which can lead to inference attacks.
## Wire Specification
We extend the [HistoryQuery](../../standards/core/13/store.md/#payloads) protobuf message with two fields of `start_time` and `end_time` to signify the time range to be queried.
### Payloads
```diff
syntax = "proto3";
message HistoryQuery {
// the first field is reserved for future use
string pubsubtopic = 2;
repeated ContentFilter contentFilters = 3;
PagingInfo pagingInfo = 4;
+ sint64 start_time = 5;
+ sint64 end_time = 6;
}
```
### HistoryQuery
RPC call to query historical messages.
- `start_time`: this field MAY be filled out to signify the starting point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../standards/core/14/message.md/#Payloads) is larger than or equal to the `start_time`.
- `end_time` this field MAY be filled out to signify the ending point of the queried time window.
This field holds the Unix epoch time in nanoseconds.
The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../standards/core/14/message.md/#Payloads) is less than or equal to the `end_time`.
A time-based query is considered valid if its `end_time` is larger than or equal to the `start_time`.
Queries that do not adhere to this condition will not get through e.g. an open-end time query in which the `start_time` is given but no `end_time` is supplied is not valid.
If both `start_time` and `end_time` are omitted then no time-window filter takes place.
In order to account for nodes asynchrony, and assuming that nodes may be out of sync for at most 20 seconds (i.e., 20000000000 nanoseconds), the querying nodes SHOULD add an offset of 20 seconds to their offline time window.
That is if the original window is [`l`,`r`] then the history query SHOULD be made for `[start_time: l - 20s, end_time: r + 20s]`.
Note that `HistoryQuery` preserves `AND` operation among the queried attributes.
As such, The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages that satisfy the indicated `pubsubtopic` AND `contentFilters` AND the time range [`start_time`, `end_time`].
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [13/WAKU2-STORE](../../standards/core/13/store.md)
- [`timestamp`](../../standards/core/14/message.md/#Payloads)

154
waku/standards/application/26/payload.md Normal file
View File

@@ -0,0 +1,154 @@
---
title: 26/WAKU2-PAYLOAD
name: Waku Message Payload Encryption
status: draft
editor: Oskar Thoren \<oskarth@titanproxy.com\>
contributors:
sidebar_position: 1
---
This specification describes how Waku provides confidentiality, authenticity, and integrity, as well as some form of unlinkability.
Specifically, it describes how encryption, decryption and signing works in [6/WAKU1](../../standards/core/6/waku1.md) and in [10/WAKU2 spec](../../standards/core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../standards/core/14/message.md/#version1).
This specification effectively replaces [7/WAKU-DATA](../../standards/application/7/DATA.md) as well as [6/WAKU1 Payload encryption](../../standards/core/6/waku1.md/#payload-encryption) but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
Large sections of the specification originate from [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) with some modifications.
## Design requirements
- *Confidentiality*: The adversary should not be able to learn what data is being sent from one Waku node to one or several other Waku nodes.
- *Authenticity*: The adversary should not be able to cause Waku endpoint to accept data from any third party as though it came from the other endpoint.
- *Integrity*: The adversary should not be able to cause a Waku endpoint to accept data that has been tampered with.
Notable, *forward secrecy* is not provided for at this layer.
If this property is desired, a more fully featured secure communication protocol can be used on top, such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
It also provides some form of *unlinkability* since:
- only participants who are able to decrypt a message can see its signature
- payload are padded to a fixed length
## Cryptographic primitives
- AES-256-GCM (for symmetric encryption)
- ECIES
- ECDSA
- KECCAK-256
ECIES is using the following cryptosystem:
- Curve: secp256k1
- KDF: NIST SP 800-56 Concatenation Key Derivation Function, with SHA-256 option
- MAC: HMAC with SHA-256
- AES: AES-128-CTR
## Specification
For 6/WAKU1, the `data` field is used in the `waku envelope`, and the field MAY contain the encrypted payload.
For 10/WAKU2, the `payload` field is used in `WakuMessage` and MAY contain the encrypted payload.
The fields that are concatenated and encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
- flags
- payload-length
- payload
- padding
- signature
### ABNF
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
```abnf
; 1 byte; first two bits contain the size of payload-length field,
; third bit indicates whether the signature is present.
flags = 1OCTET
; contains the size of payload.
payload-length = 4*OCTET
; byte array of arbitrary size (may be zero).
payload = *OCTET
; byte array of arbitrary size (may be zero).
padding = *OCTET
; 65 bytes, if present.
signature = 65OCTET
data = flags payload-length payload padding [signature]
; This field is called payload in Waku v2
payload = data
```
### Signature
Those unable to decrypt the payload/data are also unable to access the signature.
The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity.
The signature is serialized as the concatenation of the `r`, `s` and `v` parameters of the SECP-256k1 ECDSA signature, in that order.
`r` and `s` MUST be big-endian encoded, fixed-width 256-bit unsigned.
`v` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
See [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf) for more information on signature generation, parameters and public key recovery.
### Encryption
#### Symmetric
Symmetric encryption uses AES-256-GCM for [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption).
The output of encryption is of the form (`ciphertext`, `tag`, `iv`) where `ciphertext` is the encrypted message, `tag` is a 16 byte message authentication tag and `iv` is a 12 byte initialization vector (nonce).
The message authentication `tag` and initialization vector `iv` field MUST be appended to the resulting `ciphertext`, in that order.
Note that previous specifications and some implementations might refer to `iv` as `nonce` or `salt`.
#### Asymmetric
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme (ECIES) with SECP-256k1 public key.
#### ECIES
This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) spec with minor modifications.
The cryptosystem used is:
- The elliptic curve secp256k1 with generator `G`.
- `KDF(k, len)`: the NIST SP 800-56 Concatenation Key Derivation Function.
- `MAC(k, m)`: HMAC using the SHA-256 hash function.
- `AES(k, iv, m)`: the AES-128 encryption function in CTR mode.
Special notation used: `X || Y` denotes concatenation of `X` and `Y`.
Alice wants to send an encrypted message that can be decrypted by Bob's static private key `kB`. Alice knows about Bobs static public key `KB`.
To encrypt the message `m`, Alice generates a random number `r` and corresponding elliptic curve public key `R = r * G` and computes the shared secret `S = Px` where `(Px, Py) = r * KB`.
She derives key material for encryption and authentication as `kE || kM = KDF(S, 32)` as well as a random initialization vector `iv`.
Alice sends the encrypted message `R || iv || c || d` where `c = AES(kE, iv , m)` and `d = MAC(sha256(kM), iv || c)` to Bob.
For Bob to decrypt the message `R || iv || c || d`, he derives the shared secret `S = Px` where `(Px, Py) = kB * R` as well as the encryption and authentication keys `kE || kM = KDF(S, 32)`.
Bob verifies the authenticity of the message by checking whether `d == MAC(sha256(kM), iv || c)` then obtains the plaintext as `m = AES(kE, iv || c)`.
### Padding
The padding field is used to align data size, since data size alone might reveal important metainformation.
Padding can be arbitrary size.
However, it is recommended that the size of Data Field (excluding the IV and tag) before encryption (i.e. plain text) SHOULD be a multiple of 256 bytes.
### Decoding a message
In order to decode a message, a node SHOULD try to apply both symmetric and asymmetric decryption operations.
This is because the type of encryption is not included in the message.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [6/WAKU1](../../standards/core/6/waku1.md)
2. [10/WAKU2 spec](../../standards/core/10/waku2.md)
3. [14/WAKU-MESSAGE version 1](../../standards/core/14/message.md/#version1)
4. [7/WAKU-DATA](../../standards/application/7/DATA.md)
5. [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627)
6. [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
7. [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
8. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
9. [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
10. [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption)

257
waku/standards/application/53/x3dh.md Normal file
View File

@@ -0,0 +1,257 @@
---
title: 53/WAKU2-X3DH
name: X3DH usage for Waku payload encryption
status: draft
category: Standards Track
editor: Aaryamann Challani \<aaryamann@status.im\>
contributors:
- Andrea Piana \<andreap@status.im\>
- Pedro Pombeiro \<pedro@status.im\>
- Corey Petty \<corey@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
- Dean Eigenmann \<dean@status.im\>
sidebar_position: 1
---
## Abstract
This document describes a method that can be used to provide a secure channel between two peers, and thus provide confidentiality, integrity, authenticity and forward secrecy.
It is transport-agnostic and works over asynchronous networks.
It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications, with some adaptations to operate in a decentralized environment.
## Motivation
Nodes on a network may want to communicate with each other in a secure manner, without other nodes network being able to read their messages.
## Specification
### Definitions
- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols which provide assurances that session keys will not be compromised even if the private keys of the participants are compromised.
Specifically, past messages cannot be decrypted by a third-party who manages to get a hold of a private key.
- **Secret channel** describes a communication channel where a Double Ratchet algorithm is in use.
### Design Requirements
- **Confidentiality**: The adversary should not be able to learn what data is being exchanged between two Status clients.
- **Authenticity**: The adversary should not be able to cause either endpoint to accept data from any third party as though it came from the other endpoint.
- **Forward Secrecy**: The adversary should not be able to learn what data was exchanged between two clients if, at some later time, the adversary compromises one or both of the endpoints.
- **Integrity**: The adversary should not be able to cause either endpoint to accept data that has been tampered with.
All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
### Conventions
Types used in this specification are defined using the [Protobuf](https://developers.google.com/protocol-buffers/) wire format.
## Specification
### End-to-End Encryption
End-to-end encryption (E2EE) takes place between two clients.
The main cryptographic protocol is a Double Ratchet protocol, which is derived from the [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html), using a different ratchet.
[The Waku v2 protocol](../../standards/core/10/waku2.md) subsequently encrypts the message payload, using symmetric key encryption.
Furthermore, the concept of prekeys (through the use of [X3DH](https://signal.org/docs/specifications/x3dh/)) is used to allow the protocol to operate in an asynchronous environment.
It is not necessary for two parties to be online at the same time to initiate an encrypted conversation.
### Cryptographic Protocols
This protocol uses the following cryptographic primitives:
- X3DH
- Elliptic curve Diffie-Hellman key exchange (secp256k1)
- KECCAK-256
- ECDSA
- ECIES
- Double Ratchet
- HMAC-SHA-256 as MAC
- Elliptic curve Diffie-Hellman key exchange (Curve25519)
- AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
The node achieves key derivation using [HKDF](https://www.rfc-editor.org/rfc/rfc5869).
### Pre-keys
Every client SHOULD initially generate some key material which is stored locally:
- Identity keypair based on secp256k1 - `IK`
- A signed prekey based on secp256k1 - `SPK`
- A prekey signature - `Sig(IK, Encode(SPK))`
More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles).
Prekey bundles MAY be extracted from any peer's messages, or found via searching for their specific topic, `{IK}-contact-code`.
The following methods can be used to retrieve prekey bundles from a peer's messages:
- contact codes;
- public and one-to-one chats;
- QR codes;
- ENS record;
- Decentralized permanent storage (e.g. Swarm, IPFS).
- Waku
Waku SHOULD be used for retrieving prekey bundles.
Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys, the bundle MAY be rotated every 24 hours, and distributed via Waku.
### Flow
The key exchange can be summarized as follows:
1. Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles, and derive a shared secret.
2. Double Ratchet: The two parties use the shared secret to derive a new encryption key for each message they send.
3. Chain key update: The two parties update their chain keys. The chain key is used to derive new encryption keys for future messages.
4. Message key derivation: The two parties derive a new message key from their chain key, and use it to encrypt a message.
#### 1. Initial key exchange flow (X3DH)
[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message) describes the initial key exchange flow, with some additional context:
- The peers' identity keys `IK_A` and `IK_B` correspond to their public keys;
- Since it is not possible to guarantee that a prekey will be used only once in a decentralized world, the one-time prekey `OPK_B` is not used in this scenario;
- Nodes SHOULD not send Bundles to a centralized server, but instead provide them in a decentralized way as described in the [Pre-keys section](#pre-keys).
Alice retrieves Bob's prekey bundle, however it is not specific to Alice. It contains:
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12))
**Wire format:**
``` protobuf
// X3DH prekey bundle
message Bundle {
// Identity key 'IK_B'
bytes identity = 1;
// Signed prekey 'SPK_B' for each device, indexed by 'installation-id'
map\<string,SignedPreKey\> signed_pre_keys = 2;
// Prekey signature 'Sig(IK_B, Encode(SPK_B))'
bytes signature = 4;
// When the bundle was created locally
int64 timestamp = 5;
}
```
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L5))
``` protobuf
message SignedPreKey {
bytes signed_pre_key = 1;
uint32 version = 2;
}
```
The `signature` is generated by sorting `installation-id` in lexicographical order, and concatenating the `signed-pre-key` and `version`:
`installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2`
#### 2. Double Ratchet
Having established the initial shared secret `SK` through X3DH, it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
Refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/) for more details.
The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage` ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65)) containing a map of `DirectMessageProtocol` indexed by `installation-id` ([reference wire format](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
``` protobuf
message ProtocolMessage {
// The installation id of the sender
string installation_id = 2;
// A sequence of bundles
repeated Bundle bundles = 3;
// One to one message, encrypted, indexed by installation_id
map\<string,DirectMessageProtocol\> direct_message = 101;
// Public message, not encrypted
bytes public_message = 102;
}
```
``` protobuf
message EncryptedMessageProtocol {
X3DHHeader X3DH_header = 1;
DRHeader DR_header = 2;
DHHeader DH_header = 101;
// Encrypted payload
// if a bundle is available, contains payload encrypted with the Double Ratchet algorithm;
// otherwise, payload encrypted with output key of DH exchange (no Perfect Forward Secrecy).
bytes payload = 3;
}
```
Where:
- `X3DH_header`: the `X3DHHeader` field in `DirectMessageProtocol` contains:
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
``` protobuf
message X3DHHeader {
// Alice's ephemeral key `EK_A`
bytes key = 1;
// Bob's bundle signed prekey
bytes id = 4;
}
```
- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)). Used when Bob's public bundle is available:
``` protobuf
message DRHeader {
// Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet))
bytes key = 1;
// number of the message in the sending chain
uint32 n = 2;
// length of the previous sending chain
uint32 pn = 3;
// Bob's bundle ID
bytes id = 4;
}
```
- `DH_header`: Diffie-Hellman header (used when Bob's bundle is not available):
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
``` protobuf
message DHHeader {
// Alice's compressed ephemeral public key.
bytes key = 1;
}
```
#### 3. Chain key update
The chain key MUST be updated according to the `DR_Header` received in the `EncryptedMessageProtocol` message, described in [2.Double Ratchet](#2-double-ratchet).
#### 4. Message key derivation
The message key MUST be derived from a single ratchet step in the symmetric-key ratchet as described in [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
The message key MUST be used to encrypt the next message to be sent.
## Security Considerations
1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
2. Inherits the security considerations of the [Waku v2 protocol](../../standards/core/10/waku2.md).
3. The protocol is designed to be used in a decentralized manner, however, it is possible to use a centralized server to serve prekey bundles. In this case, the server is trusted.
## Privacy Considerations
1. This protocol does not provide message unlinkability. It is possible to link messages signed by the same keypair.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [X3DH](https://signal.org/docs/specifications/x3dh/)
- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
- [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
- [Protobuf](https://developers.google.com/protocol-buffers/)
- [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html)
- [The Waku v2 protocol](../../standards/core/10/waku2.md)
- [HKDF](https://www.rfc-editor.org/rfc/rfc5869)
- [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles)
- [reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12)
- [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
-
-

163
waku/standards/application/54/x3dh-sessions.md Normal file
View File

@@ -0,0 +1,163 @@
---
title: 54/WAKU2-X3DH-SESSIONS
name: Session management for Waku X3DH
status: draft
category: Standards Track
editor: Aaryamann Challani \<aaryamann@status.im\>
contributors:
- Andrea Piana \<andreap@status.im\>
- Pedro Pombeiro \<pedro@status.im\>
- Corey Petty \<corey@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
- Dean Eigenmann \<dean@status.im\>
sidebar_position: 1
---
## Abstract
This document specifies how to manage sessions based on an X3DH key exchange.
This includes how to establish new sessions, how to re-establish them, how to maintain them, and how to close them.
[53/WAKU2-X3DH](../../standards/application/53/X3DH.md) specifies the Waku `X3DH` protocol for end-to-end encryption.
Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
## Session Establishment
A node identifies a peer by their `installation-id` which MAY be interpreted as a device identifier.
### Discovery of pre-key bundles
The node's pre-key bundle MUST be broadcast on a content topic derived from the node's public key, so that the first message may be PFS-encrypted.
Each peer MUST publish their pre-key bundle periodically to this topic, otherwise they risk not being able to perform key-exchanges with other peers.
Each peer MAY publish to this topic when their metadata changes, so that the other peer can update their local record.
If peer A wants to send a message to peer B, it MUST derive the topic from peer B's public key, which has been shared out of band.
Partitioned topics have been used to balance privacy and efficiency of broadcasting pre-key bundles.
The number of partitions that MUST be used is 5000.
The topic MUST be derived as follows:
```
var partitionsNum *big.Int = big.NewInt(5000)
var partition *big.Int = big.NewInt(0).Mod(peerBPublicKey, partitionsNum)
partitionTopic := "contact-discovery-" + strconv.FormatInt(partition.Int64(), 10)
var hash []byte = keccak256(partitionTopic)
var topicLen int = 4
if len(hash) \< topicLen {
topicLen = len(hash)
}
var contactCodeTopic [4]byte
for i = 0; i \< topicLen; i++ {
contactCodeTopic[i] = hash[i]
}
```
### Initialization
A node initializes a new session once a successful X3DH exchange has taken place.
Subsequent messages will use the established session until re-keying is necessary.
### Negotiated topic to be used for the session
After the peers have performed the initial key exchange, they MUST derive a topic from their shared secret to send messages on.
To obtain this value, take the first four bytes of the keccak256 hash of the shared secret encoded in hexadecimal format.
```
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
ecies.ImportECDSAPublic(theirPublicKey),
16,
16,
)
hexEncodedKey := hex.EncodeToString(sharedKey)
var hash []byte = keccak256(hexEncodedKey)
var topicLen int = 4
if len(hash) \< topicLen {
topicLen = len(hash)
}
var topic [4]byte
for i = 0; i \< topicLen; i++ {
topic[i] = hash[i]
}
```
To summarize, following is the process for peer B to establish a session with peer A:
1. Listen to peer B's Contact Code Topic to retrieve their bundle information, including a list of active devices
2. Peer A sends their pre-key bundle on peer B's partitioned topic
3. Peer A and peer B perform the key-exchange using the shared pre-key bundles
3. The negotiated topic is derived from the shared secret
4. Peers A & B exchange messages on the negotiated topic
### Concurrent sessions
If a node creates two sessions concurrently between two peers, the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
### Re-keying
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
### Multi-device support
Multi-device support is quite challenging as there is not a central place where information on which and how many devices (identified by their respective `installation-id`) a peer has, is stored.
Furthermore, account recovery always needs to be taken into consideration, where a user wipes clean the whole device and the node loses all the information about any previous sessions.
Taking these considerations into account, the way the network propagates multi-device information using X3DH bundles, which will contain information about paired devices as well as information about the sending device.
This means that every time a new device is paired, the bundle needs to be updated and propagated with the new information, the user has the responsibility to make sure the pairing is successful.
The method is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/).
### Pairing
A new `installation-id` MUST be generated on a per-device basis.
The device should be paired as soon as possible if other devices are present.
If a bundle is received, which has the same `IK` as the keypair present on the device, the devices MAY be paired.
Once a user enables a new device, a new bundle MUST be generated which includes pairing information.
The bundle MUST be propagated to contacts through the usual channels.
Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
### Sending messages to a paired group
When sending a message, the peer SHOULD send a message to other `installation-id` that they have seen.
The node caps the number of devices to `n`, ordered by last activity.
The node sends messages using pairwise encryption, including their own devices.
Where `n` is the maximum number of devices that can be paired.
### Account recovery
Account recovery is the same as adding a new device, and it MUST be handled the same way.
### Partitioned devices
In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own `installation-id`.
In this case an empty message containing bundle information MUST be sent back, which will notify the receiving end not to include the device in any further communication.
## Security Considerations
1. Inherits all security considerations from [53/WAKU2-X3DH](../../standards/application/53/X3DH.md).
### Recommendations
1. The value of `n` SHOULD be configured by the app-protocol.
- The default value SHOULD be 3, since a larger number of devices will result in a larger bundle size, which may not be desirable in a peer-to-peer network.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [53/WAKU2-X3DH](../../standards/application/53/X3DH.md)
2. [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)

33
waku/standards/core/10/images/overview.msc Normal file
View File

@@ -0,0 +1,33 @@
# Sequence diagram for Waku v2 (WakuMessage, WakuData, Relay, Store, Filter)
# PNG generated with https://mscgen.js.org
msc {
hscale="1",
wordwraparcs=true;
a [label="A\nrelay\n(0)"],
b [label="B relay(pubtopic1)\n(0)"],
c [label="C relay(pubtopic2)\n(0)"],
d [label="D relay(pubtopic1), store(pubtopic1), filter\n(0)"],
e [label="E\nrelay, store\n(0)"],
f [label="F\nrelay, filter\n(0)"];
a rbox a [label="msg1=WakuMessage(contentTopic1, data) [14/WAKU2-MESSAGE] (1)"];
a note a [label="If version=1, encrypt data per [7/WAKU-DATA] (1)"];
f => d [label="FilterRequest(pubtopic1, contentTopic1) [12/WAKU2-FILTER] (2)"];
d rbox d [label="Subscribe F to filter [12/WAKU2-FILTER] (2)"];
a => b [label="Publish msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
b => d [label="relay msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
d rbox d [label="store: saves msg1 [13/WAKU2-STORE] (4)"];
d => f [label="MessagePush(msg1)[12/WAKU2-FILTER] (5)"];
---;
e note e [label="E comes online (6)"];
e => d [label="HistoryQuery(pubtopic1, contentTopic1) [13/WAKU2-STORE] (6)"];
d => e [label="HistoryResponse(msg1, ...) [13/WAKU2-STORE] (6)"];
}

BIN
waku/standards/core/10/images/overview.png Normal file
View File

Binary file not shown.

499
waku/standards/core/10/waku2.md Normal file
View File

@@ -0,0 +1,499 @@
---
title: 10/WAKU2
name: Waku v2
status: draft
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Sanaz Taheri \<sanaz@status.im\>
- Hanno Cornelius \<hanno@status.im\>
- Reeshav Khan \<reeshav@status.im\>
- Daniel Kaiser \<danielkaiser@status.im\>
sidebar_position: 1
---
## Abstract
Waku v2 is family of modular peer-to-peer protocols for secure communication.
The protocols are designed to be secure, privacy-preserving, censorship-resistant and being able to run in resource restricted environments.
At a high level, it implements Pub/Sub over [libp2p](https://github.com/libp2p/specs) and adds a set of capabilities to it.
These capabilities are things such as:
(i) retrieving historical messages for mostly-offline devices
(ii) adaptive nodes, allowing for heterogeneous nodes to contribute to the network
(iii) preserving bandwidth usage for resource-restriced devices
This makes Waku ideal for running a p2p protocol on mobile and in similarly restricted environments.
Historically, it has its roots in [6/WAKU1](../6/waku1.md),
which stems from [Whisper](https://eips.ethereum.org/EIPS/eip-627), originally part of the Ethereum stack.
However, Waku v2 acts more as a thin wrapper for PubSub and has a different API.
It is implemented in an iterative manner where initial focus is on porting essential functionality to libp2p.
See [rough road map (2020)](https://vac.dev/waku-v2-plan) for more historical context.
## Motivation and goals
Waku as a family of protocols is designed to have a set of properties that are useful for many applications:
1. **Useful for generalized messaging.**
Many applications require some form of messaging protocol to communicate between different subsystems or different nodes.
This messaging can be human-to-human or machine-to-machine or a mix.
Waku is designed to work for all these scenarios.
2. **Peer-to-peer.**
Applications sometimes have requirements that make them suitable for peer-to-peer solutions:
- Censorship-resistant with no single point of failure
- Adaptive and scalable network
- Shared infrastructure
3. **Runs anywhere.**
Applications often run in restricted environments, where resources or the environment is restricted in some fashion.
For example:
- Limited bandwidth, CPU, memory, disk, battery, etc
- Not being publicly connectable
- Only being intermittently connected; mostly-offline
4. **Privacy-preserving.**
Applications often have a desire for some privacy guarantees, such as:
- Pseudonymity and not being tied to any personally identifiable information (PII)
- Metadata protection in transit
- Various forms of unlinkability, etc
5. **Modular design.**
Applications often have different trade-offs when it comes to what properties they and their users value.
Waku is designed in a modular fashion where an application protocol or node can choose what protocols they run.
We call this concept *adaptive nodes*.
For example:
- Resource usage vs metadata protection
- Providing useful services to the network vs mostly using it
- Stronger guarantees for spam protection vs economic registration cost
For more on the concept of adaptive nodes and what this means in practice,
please see the [30/ADAPTIVE-NODES](../../../informational/30/adaptive-nodes.md) spec.
## Network interaction domains
While Waku is best thought of as a single cohesive thing, there are three network interaction domains:
(a) gossip domain
(b) discovery domain
(c) req/resp domain
### Protocols and identifiers
Since Waku v2 is built on top of libp2p, many protocols have a libp2p protocol identifier.
The current main [protocol identifiers](https://docs.libp2p.io/concepts/protocols/) are:
1. `/vac/waku/relay/2.0.0`
2. `/vac/waku/store/2.0.0-beta4`
3. `/vac/waku/filter/2.0.0-beta1`
4. `/vac/waku/lightpush/2.0.0-beta1`
This is in addition to protocols that specify messages, payloads, and recommended usages.
Since these aren't negotiated libp2p protocols, they are referred to by their RFC ID.
For example:
- [14/WAKU2-MESSAGE](../14/message.md) and [26/WAKU-PAYLOAD](../../application/26/payload.md) for message payloads
- [23/WAKU2-TOPICS](../../../informational/23/topics.md) and [27/WAKU2-PEERS](../../../informational/27/peers.md) for recommendations around usage
There are also more experimental libp2p protocols such as:
1. `/vac/waku/swap/2.0.0-beta1`
2. `/vac/waku/waku-rln-relay/2.0.0-alpha1`
These protocols and their semantics are elaborated on in their own specs.
### Use of libp2p and protobuf
Unless otherwise specified, all protocols are implemented over libp2p and use Protobuf by default.
Since messages are exchanged over a [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/),
as a convention, libp2p protocols prefix binary message payloads with the length of the message in bytes.
This length integer is encoded as a [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints).
### Gossip domain
Waku is using gossiping to disseminate messages throughout the network.
**Protocol identifier**: `/vac/waku/relay/2.0.0`
See [11/WAKU2-RELAY](../11/relay.md) spec for more details.
For an experimental privacy-preserving economic spam protection mechanism, see [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
See [23/WAKU2-TOPICS](../../../informational/23/topics.md) for more information about recommended topic usage.
### Direct use of libp2p protocols
In addition to `/vac/waku/*` protocols, Waku v2 MAY directly use the following libp2p protocols:
* [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping) with protocol id
```
/ipfs/ping/1.0.0
```
for liveness checks between peers, or to keep peer-to-peer connections alive.
* [libp2p identity and identity/push](https://docs.libp2p.io/concepts/protocols/#identify) with protocol IDs
```
/ipfs/id/1.0.0
```
and
```
/ipfs/id/push/1.0.0
```
respectively, as basic means for capability discovery.
These protocols are anyway used by the libp2p connection establishment layer Waku v2 is built on.
We plan to introduce a new Vac capability discovery protocol with better anonymity properties and more functionality.
# Transports
Waku v2 is built in top of libp2p, and like libp2p it strives to be transport agnostic.
We define a set of recommended transports in order to achieve a baseline of interoperability between clients.
This section describes these recommended transports.
Waku client implementations SHOULD support the TCP transport.
Where TCP is supported it MUST be enabled for both dialing and listening, even if other transports are available.
Waku v2 nodes where the environment do not allow to use TCP directly, MAY use other transports.
A Waku v2 node SHOULD support secure websockets for bidirectional communication streams, for example in a web browser context.
A node MAY support unsecure websockets if required by the application or running environment.
### Discovery domain
#### Discovery methods
Waku v2 can retrieve a list of nodes to connect to using DNS-based discovery as per [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459).
While this is a useful way of bootstrapping connection to a set of peers,
it MAY be used in conjunction with an [ambient peer discovery](https://docs.libp2p.io/concepts/publish-subscribe/#discovery) procedure to find still other nodes to connect to,
such as [Node Discovery v5](https://github.com/ethereum/devp2p/blob/8fd5f7e1c1ec496a9d8dc1640a8548b8a8b5986b/discv5/discv5.md).
More ambient peer discovery methods are being tested for Waku v2,
and will be specified for wider adoption.
It is possible to bypass the discovery domain by specifying static nodes.
#### Use of ENR
[31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md) describes the usage of [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778) for Waku v2 discovery purposes.
It introduces two new ENR fields, `multiaddrs` and `waku2`, that a Waku v2 node MAY use for discovery purposes.
These fields MUST be used under certain conditions, as set out in the spec.
Both EIP-1459 DNS-based discovery and Node Discovery v5 operates on ENR,
and it's reasonable to expect even wider utility for ENR in Waku v2 networks in future.
### Request/Reply domain
In addition to the Gossip domain,
Waku provides a set of Request/Reply protocols.
They are primarily used in order to get Waku to run in resource restricted environments,
such as low bandwidth or being mostly offline.
#### Historical message support
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
This is used to fetch historical messages for mostly offline devices.
See [13/WAKU2-STORE spec](../13/store.md) spec for more details.
There is also an experimental fault-tolerant addition to the store protocol that relaxes the high availability requirement.
See [21/WAKU2-FT-STORE](../../application/21/ft-store.md)
#### Content filtering
**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
This is used to make fetching of a subset of messages more bandwidth preserving.
See [12/WAKU2-FILTER](../12/filter.md) spec for more details.
#### Light push
**Protocol identifier***: `/vac/waku/lightpush/2.0.0-beta1`
This is used for nodes with short connection windows and limited bandwidth to publish messages into the Waku network.
See [19/WAKU2-LIGHTPUSH](../19/lightpush.md) spec for more details.
#### Other protocols
The above is a non-exhaustive list,
and due to the modular design of Waku there may be other protocols here that provide a useful service to the Waku network.
### Overview of protocol interaction
See the sequence diagram below for an overview of how different protocols interact.
![Overview of how protocols interact in Waku v2.](./images/overview.png)
0. We have six nodes, A-F.
The protocols initially mounted are indicated as such.
The PubSub topics `pubtopic1` and `pubtopic2` is used for routing and indicates that it is subscribed to messages on that topic for relay, see [11/WAKU2-RELAY](../11/relay.md) for details.
Ditto for [13/WAKU2-STORE](../13/store.md) where it indicates that these messages are persisted on that node.
1. Node A creates a WakuMessage `msg1` with a ContentTopic `contentTopic1`.
See [14/WAKU2-MESSAGE](../core/14/message.md) for more details.
If WakuMessage version is set to 1, we use the [6/WAKU1](../6/waku1.md) compatible `data` field with encryption.
See [7/WAKU-DATA](../../application/7/data.md) for more details.
2. Node F requests to get messages filtered by PubSub topic `pubtopic1` and ContentTopic `contentTopic1`.
Node D subscribes F to this filter and will in the future forward messages that match that filter.
See [12/WAKU2-FILTER](../12/filter.md) for more details.
3. Node A publishes `msg1` on `pubtopic1` and subscribes to that relay topic pick it up.
It then gets relayed further from B to D, but not C since it doesn't subscribe to that topic.
See [11/WAKU2-RELAY](../11/relay.md).
4. Node D saves `msg1` for possible later retrieval by other nodes.
See [13/WAKU2-STORE](../13/store.md).
5. Node D also pushes `msg1` to F, as it has previously subscribed F to this filter.
See [12/WAKU2-FILTER](../12/filter.md).
6. At a later time, Node E comes online.
It then requests messages matching `pubtopic1` and `contentTopic1` from Node D.
Node D responds with messages meeting this (and possibly other) criteria. See [13/WAKU2-STORE](../13/store.md).
## Appendix A: Upgradability and Compatibility
### Compatibility with Waku v1
Waku v1 and Waku v2 are different protocols all together.
They use a different transport protocol underneath; Waku v1 is devp2p RLPx based while Waku v2 uses libp2p.
The protocols themselves also differ as does their data format.
Compatibility can be achieved only by using a bridge that not only talks both devp2p RLPx and libp2p, but that also transfers (partially) the content of a packet from one version to the other.
See [15/WAKU-BRIDGE](../15/bridge.md) for details on a bidirectional bridge mode.
# Appendix B: Security
Each protocol layer of Waku v2 provides a distinct service and is associated with a separate set of security features and concerns.
Therefore, the overall security of Waku v2 depends on how the different layers are utilized.
In this section, we overview the security properties of Waku v2 protocols against a static adversarial model which is described below.
Note that a more detailed security analysis of each Waku protocol is supplied in its respective specification as well.
## Primary Adversarial Model
In the primary adversarial model, we consider adversary as a passive entity that attempts to collect information from others to conduct an attack,
but it does so without violating protocol definitions and instructions.
The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers
(unless the adversary is one end of the communication).
Specifically, the communication channels are assumed to be secure.
## Security Features
### Pseudonymity
Waku v2 by default guarantees pseudonymity for all of the protocol layers since parties do not have to disclose their true identity
and instead they utilize libp2p `PeerID` as their identifiers.
While pseudonymity is an appealing security feature, it does not guarantee full anonymity since the actions taken under the same pseudonym
i.e., `PeerID` can be linked together and potentially result in the re-identification of the true actor.
### Anonymity / Unlinkability
At a high level, anonymity is the inability of an adversary in linking an actor to its data/performed action (the actor and action are context-dependent).
To be precise about linkability, we use the term Personally Identifiable Information (PII) to refer to any piece of data that could potentially be used to uniquely identify a party.
For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
Notice that users' actions can be traced through their PIIs (e.g., signatures) and hence result in their re-identification risk.
As such, we seek anonymity by avoiding linkability between actions and the actors / actors' PII. Concerning anonymity, Waku v2 provides the following features:
**Publisher-Message Unlinkability**:
This feature signifies the unlinkability of a publisher to its published messages in the 11/WAKU2-RELAY protocol.
The [Publisher-Message Unlinkability](../11/relay.md/#security-analysis) is enforced through the `StrictNoSign` policy due to which the data fields of pubsub messages that count as PII for the publisher must be left unspecified.
**Subscriber-Topic Unlinkability**:
This feature stands for the unlinkability of the subscriber to its subscribed topics in the 11/WAKU2-RELAY protocol.
The [Subscriber-Topic Unlinkability](../11/relay.md/#security-analysis) is achieved through the utilization of a single PubSub topic.
As such, subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of subscribers).
Note that there is no hard limit on the number of the pubsub topics, however, the use of one topic is recommended for the sake of anonymity.
### Spam protection
This property indicates that no adversary can flood the system (i.e., publishing a large number of messages in a short amount of time), either accidentally or deliberately, with any kind of message i.e. even if the message content is valid or useful.
Spam protection is partly provided in `11/WAKU2-RELAY` through the [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures) provided for by GossipSub v1.1.
At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
### Data confidentiality, Integrity, and Authenticity
Confidentiality can be addressed through data encryption whereas integrity and authenticity are achievable through digital signatures.
These features are provided for in [14/WAKU2-MESSAGE (version 1)](../14/message.md/#version-1)` through payload encryption as well as encrypted signatures.
## Security Considerations
**Lack of anonymity/unlinkability in the protocols involving direct connections including `13/WAKU2-STORE` and `12/WAKU2-FILTER` protocols**:
The anonymity/unlinkability is not guaranteed in the protocols like `13/WAKU2-STORE` and `12/WAKU2-FILTER` where peers need to have direct connections to benefit from the designated service.
This is because during the direct connections peers utilize `PeerID` to identify each other,
therefore the service obtained in the protocol is linkable to the beneficiary's `PeerID` (which counts as PII).
For `13/WAKU2-STORE`, the queried node would be able to link the querying node's `PeerID` to its queried topics.
Likewise, in the `12/WAKU2-FILTER`, a full node can link the light node's `PeerID`s to its content filter.
\<!-- TODO: to inspect the nim-libp2p codebase and figure out the exact use of PeerIDs in direct communication, it might be the case that the requester does not have to disclose its PeerID--\>
\<!--TODO: might be good to add a figure visualizing the Waku protocol stack and the security features of each layer--\>
## Appendix C: Implementation Notes
### Implementation Matrix
There are multiple implementations of Waku v2 and its protocols:
- [nim-waku (Nim)](https://github.com/status-im/nim-waku/)
- [go-waku (Go)](https://github.com/status-im/go-waku/)
- [js-waku (NodeJS and Browser)](https://github.com/status-im/js-waku/)
Below you can find an overview of the specs that they implement as they relate to Waku v2.
This includes Waku v1 specs, as they are used for bridging between the two networks.
| Spec | nim-waku (Nim) | go-waku (Go) | js-waku (Node JS) | js-waku (Browser JS) |
| ---- | -------------- | ------------ | ----------------- | -------------------- |
|[6/WAKU1](../6/waku1.md)|✔|||
|[7/WAKU-DATA](../7/data.md)|✔|✔||
|[8/WAKU-MAIL](../../application/8/mail.md)|✔|||
|[9/WAKU-RPC](../9/waku2-rpc.md)|✔|||
|[10/WAKU2](../10/waku2.md)|✔|🚧|🚧|🚧|
|[11/WAKU2-RELAY](../11/relay.md)|✔|✔|✔|✔|
|[12/WAKU2-FILTER](../12/filter.md)|✔|✔||
|[13/WAKU2-STORE](../13/store.md)|✔|✔|✔\*|✔\*|
|[14/WAKU2-MESSAGE](../14/message.md))|✔|✔|✔|✔|
|[15/WAKU2-BRIDGE](../15/bridge.md)|✔|||
|[16/WAKU2-RPC](../16/rpc.md)|✔|||
|[17/WAKU2-RLN-RELAY](../17/rln-relay.md)|🚧|||
|[18/WAKU2-SWAP](../../application/18/swap.md)|🚧|||
|[19/WAKU2-LIGHTPUSH](../19/lightpush.md)|✔|✔|✔\**|✔\**|
|[21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)|✔|✔||
*js-waku implements [13/WAKU2-STORE](../13/store.md) as a querying node only.
**js-waku only implements [19/WAKU2-LIGHTPUSH](../19/lightpush.md) requests.
### Recommendations for clients
To implement a minimal Waku v2 client, we recommend implementing the following subset in the following order:
- [10/WAKU2](../10/waku2.md) - this spec
- [11/WAKU2-RELAY](../11/relay.md) - for basic operation
- [14/WAKU2-MESSAGE](../14/message.md) - version 0 (unencrypted)
- [13/WAKU2-STORE](../13/store.md) - for historical messaging (query mode only)
To get compatibility with Waku v1:
- [7/WAKU-DATA](../7/data.md)
- [14/WAKU2-MESSAGE](../14/message.md) - version 1 (encrypted with `7/WAKU-DATA`)
For an interoperable keep-alive mechanism:
- [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping),
with periodic pings to connected peers
## Appendix D: Future work
The following features are currently experimental and under research and initial implementation:
**Economic Spam resistance**:
We aim to enable an incentivized spam protection technique to enhance `11/WAKU2-RELAY` by using rate limiting nullifiers.
More details on this can be found in [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
In this advanced method, peers are limited to a certain rate of messaging per epoch and an immediate financial penalty is enforced for spammers who break this rate.
**Prevention of Denial of Service (DoS) and Node Incentivization**:
Denial of service signifies the case where an adversarial node exhausts another node's service capacity (e.g., by making a large number of requests) and makes it unavailable to the rest of the system.
DoS attack is to be mitigated through the accounting model as described in [18/WAKU2-SWAP](../../application/18/swap.md).
In a nutshell, peers have to pay for the service they obtain from each other.
In addition to incentivizing the service provider, accounting also makes DoS attacks costly for malicious peers.
The accounting model can be used in `13/WAKU2-STORE` and `12/WAKU2-FILTER` to protect against DoS attacks.
Additionally, this gives node operators who provide a useful service to the network an incentive to perform that service.
See [18/WAKU2-SWAP](../../application/18/swap.md) for more details on this piece of work.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [libp2p specs](https://github.com/libp2p/specs)
2. [6/WAKU1](../6/waku1.md)
3. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)
4. [Waku v2 plan](https://vac.dev/waku-v2-plan)
5. [30/ADAPTIVE-NODES](../../../informational/30/adaptive-nodes.md)
6. [Protocol Identifiers](https://docs.libp2p.io/concepts/protocols/)
7. [14/WAKU2-MESSAGE](../14/message.md)
8. [26/WAKU-PAYLOAD](../../application/26/payload.md)
9. [23/WAKU2-TOPICS](../../../informational/23/topics.md)
10. [27/WAKU2-PEERS](../../../informational/27/peers.md)
11. [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/)
12. [Protobuf varint encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints)
13. [11/WAKU2-RELAY spec](../11/relay.md)
14. [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
15. [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
16. [Ambient peer discovery](https://docs.libp2p.io/concepts/publish-subscribe/#discovery)
17. [Node Discovery v5](https://github.com/ethereum/devp2p/blob/8fd5f7e1c1ec496a9d8dc1640a8548b8a8b5986b/discv5/discv5.md)
18. [31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
19. [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778)
20. [13/WAKU2-STORE spec](../13/store.md)
21. [21/WAKU2-FT-STORE](../../application/21/ft-store.md)
22. [12/WAKU2-FILTER](../12/filter.md)
23. [19/WAKU2-LIGHTPUSH](../19/lightpush.md)
24. [7/WAKU-DATA](../../application/7/data.md)
25. [15/WAKU-BRIDGE](../15/bridge.md)
26. [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
27. [GossipSub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
28. [nim-waku (Nim)](https://github.com/status-im/nim-waku/)
29. [go-waku (Go)](https://github.com/status-im/go-waku/)
30. [js-waku (NodeJS and Browser)](https://github.com/status-im/js-waku/)
31. [8/WAKU-MAIL](../../application/8/mail.md)
32. [9/WAKU-RPC](../9/waku2-rpc.md)
33. [16/WAKU2-RPC](../16/rpc.md)
34. [18/WAKU2-SWAP spec](../../application/18/swap.md)
35. [21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)

196
waku/standards/core/11/relay.md Normal file
View File

@@ -0,0 +1,196 @@
---
title: 11/WAKU2-RELAY
name: Waku v2 Relay
status: stable
editor: Hanno Cornelius \<hanno@status.im\>
contributors:
- Oskar Thorén \<oskarth@titanproxy.com\>
- Sanaz Taheri \<sanaz@status.im\>
sidebar_position: 1
---
`11/WAKU2-RELAY` specifies a [Publish/Subscribe approach](https://docs.libp2p.io/concepts/publish-subscribe/) to peer-to-peer messaging with a strong focus on privacy, censorship-resistance, security and scalability.
Its current implementation is a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) and prescribes gossip-based dissemination.
As such the scope is limited to defining a separate [`protocol id`](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation) for `11/WAKU2-RELAY`, establishing privacy and security requirements, and defining how the underlying GossipSub is to be interpreted and implemented within the Waku and cryptoeconomic domain.
`11/WAKU2-RELAY` should not be confused with [libp2p circuit relay](https://github.com/libp2p/specs/tree/master/relay).
**Protocol identifier**: `/vac/waku/relay/2.0.0`
## Security Requirements
The `11/WAKU2-RELAY` protocol is designed to provide the following security properties under a static [Adversarial Model](#adversarial-model).
Note that data confidentiality, integrity, and authenticity are currently considered out of scope for `11/WAKU2-RELAY` and must be handled by higher layer protocols such as [`14/WAKU2-MESSAGE`](../14/message.md).
\<!-- May add the definition of the unsupported feature:
Confidentiality indicates that an adversary should not be able to learn the data carried by the `WakuRelay` protocol.
Integrity indicates that the data transferred by the `WakuRelay` protocol can not be tampered with by an adversarial entity without being detected.
Authenticity no adversary can forge data on behalf of a targeted publisher and make it accepted by other subscribers as if the origin is the target. --\>
- **Publisher-Message Unlinkability**:
This property indicates that no adversarial entity can link a published `Message` to its publisher.
This feature also implies the unlinkability of the publisher to its published topic ID as the `Message` embodies the topic IDs.
- **Subscriber-Topic Unlinkability**:
This feature stands for the inability of any adversarial entity from linking a subscriber to its subscribed topic IDs.
\<!-- TODO: more requirements can be added, but that needs further and deeper investigation--\>
### Terminology
_Personally identifiable information_ (PII) refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
## Adversarial Model
- Any entity running the `11/WAKU2-RELAY` protocol is considered an adversary.
This includes publishers, subscribers, and all the peers' direct connections.
Furthermore, we consider the adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions.
For example, under the passive adversarial model, no malicious subscriber hides the messages it receives from other subscribers as it is against the description of `11/WAKU2-RELAY`.
However, a malicious subscriber may learn which topics are subscribed to by which peers.
- The following are **not** considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
In other words, the communication channels are assumed to be secure.
## Wire Specification
The [PubSub interface specification](https://github.com/libp2p/specs/blob/master/pubsub/README.md) defines the protobuf RPC messages exchanged between peers participating in a GossipSub network.
We republish these messages here for ease of reference and define how `11/WAKU2-RELAY` uses and interprets each field.
### Protobuf definitions
The PubSub RPC messages are specified using [protocol buffers v2](https://developers.google.com/protocol-buffers/)
```protobuf
syntax = "proto2";
message RPC {
repeated SubOpts subscriptions = 1;
repeated Message publish = 2;
message SubOpts {
optional bool subscribe = 1;
optional string topicid = 2;
}
message Message {
optional string from = 1;
optional bytes data = 2;
optional bytes seqno = 3;
repeated string topicIDs = 4;
optional bytes signature = 5;
optional bytes key = 6;
}
}
```
\> **_NOTE:_**
The various [control messages](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#control-messages) defined for GossipSub are used as specified there.
\> **_NOTE:_**
The [`TopicDescriptor`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) is not currently used by `11/WAKU2-RELAY`.
### Message fields
The `Message` protobuf defines the format in which content is relayed between peers.
`11/WAKU2-RELAY` specifies the following usage requirements for each field:
- The `from` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
- The `data` field MUST be filled out with a `WakuMessage`.
See [`14/WAKU2-MESSAGE`](../14/message.md) for more details.
- The `seqno` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
- The `topicIDs` field MUST contain the content-topics that a message is being published on.
- The `signature` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
- The `key` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-solicy).
### SubOpts fields
The `SubOpts` protobuf defines the format in which subscription options are relayed between peers.
A `11/WAKU2-RELAY` node MAY decide to subscribe or unsubscribe from topics by sending updates using `SubOpts`.
The following usage requirements apply:
- The `subscribe` field MUST contain a boolean, where `true` indicates subscribe and `false` indicates unsubscribe to a topic.
- The `topicid` field MUST contain the pubsub topic.
\> Note: The `topicid` refering to pubsub topic and
`topicId` refering to content-topic are detailed in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
### Signature Policy
The [`StrictNoSign` option](https://github.com/libp2p/specs/blob/master/pubsub/README.md#signature-policy-options) MUST be used, to ensure that messages are built without the `signature`, `key`, `from` and `seqno` fields.
Note that this does not merely imply that these fields be empty, but that they MUST be _absent_ from the marshalled message.
## Security Analysis
\<!-- TODO: realized that the prime security objective of the `WakuRelay` protocol is to provide peers unlinkability as such this feature is prioritized over other features e.g., unlinkability is preferred over authenticity and integrity. It might be good to motivate unlinkability and its impact on the relay protocol or other protocols invoking relay protocol.--\>
- **Publisher-Message Unlinkability**:
To address publisher-message unlinkability, one should remove any PII from the published message.
As such, `11/WAKU2-RELAY` follows the `StrictNoSign` policy as described in [libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing).
As the result of the `StrictNoSign` policy, `Message`s should be built without the `from`, `signature` and `key` fields since each of these three fields individually counts as PII for the author of the message (one can link the creation of the message with libp2p peerId and thus indirectly with the IP address of the publisher).
Note that removing identifiable information from messages cannot lead to perfect unlinkability.
The direct connections of a publisher might be able to figure out which `Message`s belong to that publisher by analyzing its traffic.
The possibility of such inference may get higher when the `data` field is also not encrypted by the upper-level protocols. \<!-- TODO: more investigation on traffic analysis attacks and their success probability--\>
- **Subscriber-Topic Unlinkability:**
To preserve subscriber-topic unlinkability, it is recommended by [`10/WAKU2`](../10/waku2.md) to use a single PubSub topic in the `11/WAKU2-RELAY` protocol.
This allows an immediate subscriber-topic unlinkability where subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID.
This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of participants of Waku relay protocol).
However, note that `11/WAKU2-RELAY` supports the use of more than one topic.
In case that more than one topic id is utilized, preserving unlinkability is the responsibility of the upper-level protocols which MAY adopt [partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic) to achieve K-anonymity for the subscribed peers.
## Future work
- **Economic spam resistance**:
In the spam-protected `11/WAKU2-RELAY` protocol, no adversary can flood the system with spam messages (i.e., publishing a large number of messages in a short amount of time).
Spam protection is partly provided by GossipSub v1.1 through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures).
At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
`11/WAKU2-RELAY` aims at enabling an advanced spam protection mechanism with economic disincentives by utilizing Rate Limiting Nullifiers.
In a nutshell, peers must conform to a certain message publishing rate per a system-defined epoch, otherwise, they get financially penalized for exceeding the rate.
More details on this new technique can be found in [`17/WAKU2-RLN-RELAY`](../17/rln-relay.md).
\<!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1 are taken in the nim-libp2p as well, may need to audit the code --\>
- Providing **Unlinkability**, **Integrity** and **Authenticity** simultaneously:
Integrity and authenticity are typically addressed through digital signatures and Message Authentication Code (MAC) schemes, however, the usage of digital signatures (where each signature is bound to a particular peer) contradicts with the unlinkability requirement (messages signed under a certain signature key are verifiable by a verification key that is bound to a particular publisher).
As such, integrity and authenticity are missing features in `11/WAKU2-RELAY` in the interest of unlinkability.
In future work, advanced signature schemes like group signatures can be utilized to enable authenticity, integrity, and unlinkability simultaneously.
In a group signature scheme, a member of a group can anonymously sign a message on behalf of the group as such the true signer is indistinguishable from other group members. \<!-- TODO: shall I add a reference for group signatures?--\>
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [`10/WAKU2`](../10/waku2.md)
1. [`14/WAKU2-MESSAGE`](../14/message.md)
1. [`17/WAKU-RLN`](../17/rln-relay.md)
1. [GossipSub v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
1. [GossipSub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
1. [K-anonimity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
1. [`libp2p` concepts: Publish/Subscribe](https://docs.libp2p.io/concepts/publish-subscribe/)
1. [`libp2p` protocol negotiation](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation)
1. [Partitioned topics](https://specs.status.im/spec/10#partitioned-topic)
1. [Protocol Buffers](https://developers.google.com/protocol-buffers/)
1. [PubSub interface for libp2p (r2, 2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
1. [Waku v1 spec](../6/waku1.md)
1. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)

266
waku/standards/core/12/filter.md Normal file
View File

@@ -0,0 +1,266 @@
---
title: 12/WAKU2-FILTER
name: Waku v2 Filter
status: draft
version: 01
editor: Hanno Cornelius \<hanno@status.im\>
contributors:
- Dean Eigenmann \<dean@status.im\>
- Oskar Thorén \<oskar@status.im\>
- Sanaz Taheri \<sanaz@status.im\>
- Ebube Ud \<ebube@status.im\>
sidebar_position: 1
---
previous versions: [00](./previous-versions00)
---
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
## Content filtering
**Protocol identifiers**:
- _filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
- _filter-push_: `/vac/waku/filter-push/2.0.0-beta1`
Content filtering is a way to do [message-based
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
Currently the only content filter being applied is on `contentTopic`. This
corresponds to topics in Waku v1.
## Rationale
Unlike the `store` protocol for historical messages, this protocol allows for
native lower latency scenarios such as instant messaging. It is thus
complementary to it.
Strictly speaking, it is not just doing basic request response, but performs
sender push based on receiver intent. While this can be seen as a form of light
pub/sub, it is only used between two nodes in a direct fashion. Unlike the
Gossip domain, this is meant for light nodes which put a premium on bandwidth.
No gossiping takes place.
It is worth noting that a light node could get by with only using the `store`
protocol to query for a recent time window, provided it is acceptable to do
frequent polling.
## Design Requirements
The effectiveness and reliability of the content filtering service enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
## Security Consideration
Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section.
### Terminology
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
## Adversarial Model
Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
### Protobuf
```protobuf
syntax = "proto3";
// 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/
package waku.filter.v2;
// Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1
message FilterSubscribeRequest {
enum FilterSubscribeType {
SUBSCRIBER_PING = 0;
SUBSCRIBE = 1;
UNSUBSCRIBE = 2;
UNSUBSCRIBE_ALL = 3;
}
string request_id = 1;
FilterSubscribeType filter_subscribe_type = 2;
// Filter criteria
optional string pubsub_topic = 10;
repeated string content_topics = 11;
}
message FilterSubscribeResponse {
string request_id = 1;
uint32 status_code = 10;
optional string status_desc = 11;
}
// Protocol identifier: /vac/waku/filter-push/2.0.0-beta1
message MessagePush {
WakuMessage waku_message = 1;
optional string pubsub_topic = 2;
}
```
### Filter-Subscribe
A filter service node MUST support the _filter-subscribe_ protocol
to allow filter clients to subscribe, modify, refresh and unsubscribe a desired set of filter criteria.
The combination of different filter criteria for a specific filter client node is termed a "subscription".
A filter client is interested in receiving messages matching the filter criteria in its registered subscriptions.
Since a filter service node is consuming resources to provide this service,
it MAY account for usage and adapt its service provision to certain clients.
An incentive mechanism is currently planned but underspecified.
#### Filter Subscribe Request
A client node MUST send all filter requests in a `FilterSubscribeRequest` message.
This request MUST contain a `request_id`.
The `request_id` MUST be a uniquely generated string.
Each request MUST include a `filter_subscribe_type`, indicating the type of request.
#### Filter Subscribe Response
In return to any `FilterSubscribeRequest`,
a filter service node SHOULD respond with a `FilterSubscribeResponse` with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful or not.
Successful status codes are in the `2xx` range.
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
In addition, the filter service node MAY choose to provide a more detailed status description in the `status_desc` field.
#### Filter matching
In the description of each request type below,
the term "filter criteria" refers to the combination of `pubsub_topic` and a set of `content_topics`.
The request MAY include filter criteria, conditional to the selected `filter_subscribe_type`.
If the request contains filter criteria,
it MUST contain a `pubsub_topic`
and the `content_topics` set MUST NOT be empty.
A `WakuMessage` matches filter criteria when its `content_topic` is in the `content_topics` set
and it was published on a matching `pubsub_topic`.
#### Filter Subscribe Types
The following filter subscribe types are defined:
##### SUBSCRIBER_PING
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBER_PING`
requests that the service node SHOULD indicate if it has any active subscriptions for this client.
The filter client SHOULD exclude any filter criteria from the request.
The filter service node SHOULD respond with a success code if it has any active subscriptions for this client
or an error code if not.
The filter service node SHOULD ignore any filter criteria in the request.
##### SUBSCRIBE
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBE`
requests that the service node SHOULD push messages matching this filter to the client.
The filter client MUST include the desired filter criteria in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _additional_ filter criteria in a new request.
A client MAY use this request type to _refresh_ an existing subscription
by providing _the same_ filter criteria in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
or an error code if not.
The filter service node SHOULD respond with an error code and discard the request
if the subscribe request does not contain valid filter criteria,
i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
##### UNSUBSCRIBE
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE`
requests that the service node SHOULD _stop_ pushing messages matching this filter to the client.
The filter client MUST include the filter criteria it desires to unsubscribe from in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _a subset of_ the original filter criteria to unsubscribe from in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
or an error code if not.
The filter service node SHOULD respond with an error code and discard the request
if the unsubscribe request does not contain valid filter criteria,
i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
##### UNSUBSCRIBE_ALL
A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
requests that the service node SHOULD _stop_ pushing messages matching _any_ filter to the client.
The filter client SHOULD exclude any filter criteria from the request.
The filter service node SHOULD remove any existing subscriptions for this client.
It SHOULD respond with a success code if it successfully honored this request
or an error code if not.
### Filter-Push
A filter client node MUST support the _filter-push_ protocol
to allow filter service nodes to push messages matching registered subscriptions to this client.
A filter service node SHOULD push all messages
matching the filter criteria in a registered subscription
to the subscribed filter client.
These [`WakuMessage`s](../14/message.md) are likely to come from [`11/WAKU2-RELAY`](../11/relay.md),
but there MAY be other sources or protocols where this comes from.
This is up to the consumer of the protocol.
If a message push fails,
the filter service node MAY consider the client node to be unreachable.
If a specific filter client node is not reachable from the service node for a period of time,
the filter service node MAY choose to stop pushing messages to the client and remove its subscription.
This period is up to the service node implementation.
We consider `1 minute` to be a reasonable default.
#### Message Push
Each message MUST be pushed in a `MessagePush` message.
Each `MessagePush` MUST contain one (and only one) `waku_message`.
If this message was received on a specific `pubsub_topic`,
it SHOULD be included in the `MessagePush`.
A filter client SHOULD NOT respond to a `MessagePush`.
Since the filter protocol does not include caching or fault-tolerance,
this is a best effort push service with no bundling
or guaranteed retransmission of messages.
A filter client SHOULD verify that each `MessagePush` it receives
originated from a service node where the client has an active subscription
and that it matches filter criteria belonging to that subscription.
---
## Future Work
\<!-- Alternative title: Filter-subscriber unlinkability --\>
**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor\<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address--\>.
Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
## Changelog
### Next
- Added initial threat model and security analysis.
### 2.0.0-beta2
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
- Fix: Ensure contentFilter is a repeated field, on implementation
- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
### 2.0.0-beta1
Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [message-based
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
- [`WakuMessage`s](../14/message.md)
- [`11/WAKU2-RELAY`](../11/relay.md)
- [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains)
- previous versions: [00](./previous-versions00)
1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)

170
waku/standards/core/12/previous-versions00/filter.md Normal file
View File

@@ -0,0 +1,170 @@
---
title: 12/WAKU2-FILTER
name: Waku v2 Filter
status: draft
editor: Hanno Cornelius \<hanno@status.im\>
contributors:
- Dean Eigenmann \<dean@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
- Sanaz Taheri \<sanaz@status.im\>
- Ebube Ud \<ebube@status.im\>
sidebar_position: 1
---
version: 00
---
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
## Content filtering
**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
Content filtering is a way to do [message-based
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
Currently the only content filter being applied is on `contentTopic`. This
corresponds to topics in Waku v1.
## Rationale
Unlike the `store` protocol for historical messages, this protocol allows for
native lower latency scenarios such as instant messaging. It is thus
complementary to it.
Strictly speaking, it is not just doing basic request response, but performs
sender push based on receiver intent. While this can be seen as a form of light
pub/sub, it is only used between two nodes in a direct fashion. Unlike the
Gossip domain, this is meant for light nodes which put a premium on bandwidth.
No gossiping takes place.
It is worth noting that a light node could get by with only using the `store`
protocol to query for a recent time window, provided it is acceptable to do
frequent polling.
## Design Requirements
The effectiveness and reliability of the content filtering service enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
## Security Consideration
Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section.
### Terminology
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
## Adversarial Model
Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the nodes and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
### Protobuf
```protobuf
message FilterRequest {
bool subscribe = 1;
string topic = 2;
repeated ContentFilter contentFilters = 3;
message ContentFilter {
string contentTopic = 1;
}
}
message MessagePush {
repeated WakuMessage messages = 1;
}
message FilterRPC {
string requestId = 1;
FilterRequest request = 2;
MessagePush push = 3;
}
```
#### FilterRPC
A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
Filter protocol is not a request response based protocol but instead a push based system.
The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
matched the message causing it to be pushed.
#### FilterRequest
A `FilterRequest` contains an optional topic, zero or more content filters and
a boolean signifying whether to subscribe or unsubscribe to the given filters.
True signifies 'subscribe' and false signifies 'unsubscribe'.
A node that sends the RPC with a filter request and `subscribe` set to 'true'
requests that the filter node SHOULD notify the light requesting node of messages
matching this filter.
A node that sends the RPC with a filter request and `subscribe` set to 'false'
requests that the filter node SHOULD stop notifying the light requesting node
of messages matching this filter if it is currently doing so.
The filter matches when content filter and, optionally, a topic is matched.
Content filter is matched when a `WakuMessage` `contentTopic` field is the same.
A filter node SHOULD honor this request, though it MAY choose not to do so. If
it chooses not to do so it MAY tell the light why. The mechanism for doing this
is currently not specified. For notifying the light node a filter node sends a
MessagePush message.
Since such a filter node is doing extra work for a light node, it MAY also
account for usage and be selective in how much service it provides. This
mechanism is currently planned but underspecified.
#### MessagePush
A filter node that has received a filter request SHOULD push all messages that
match this filter to a light node. These [`WakuMessage`'s](../14/message.md) are likely to come from the
`relay` protocol and be kept at the Node, but there MAY be other sources or
protocols where this comes from. This is up to the consumer of the protocol.
A filter node MUST NOT send a push message for messages that have not been
requested via a FilterRequest.
If a specific light node isn't connected to a filter node for some specific
period of time (e.g. a TTL), then the filter node MAY choose to not push these
messages to the node. This period is up to the consumer of the protocol and node
implementation, though a reasonable default is one minute.
---
# Future Work
\<!-- Alternative title: Filter-subscriber unlinkability --\>
**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor\<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address--\>.
Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity.
- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
## Changelog
### Next
- Added initial threat model and security analysis.
### 2.0.0-beta2
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
- Fix: Ensure contentFilter is a repeated field, on implementation
- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
### 2.0.0-beta1
Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)

258
waku/standards/core/13/store.md Normal file
View File

@@ -0,0 +1,258 @@
---
title: 13/WAKU2-STORE
name: Waku v2 Store
status: draft
editor: Sanaz Taheri \<sanaz@status.im\>
contributors:
- Dean Eigenmann \<dean@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
- Aaryamann Challani \<aaryamann@status.im\>
sidebar_position: 1
---
## Abstract
This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
stored by other nodes.
It also supports pagination for more efficient querying of historical messages.
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
## Terminology
The term PII, Personally Identifiable Information,
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and hence count as PII.
## Design Requirements
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
SHOULD provide a complete and full view of message history.
As such, they are required to be *highly available* and
specifically have a *high uptime* to consistently receive and store network messages.
The high uptime requirement makes sure that no message is missed out hence a complete and
intact view of the message history is delivered to the querying nodes.
Nevertheless, in case storage provider nodes cannot afford high availability,
the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](../14/message.md).
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
## Adversarial Model
Any peer running the `13/WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary.
Furthermore,
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
it does so without violating protocol definitions and instructions.
As we evolve the protocol,
further adversarial models will be considered.
For example, under the passive adversarial model,
no malicious node hides or
lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
## Wire Specification
Peers communicate with each other using a request / response API.
The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
The following are the specifications of the Protobuf messages.
### Payloads
```protobuf
syntax = "proto3";
message Index {
bytes digest = 1;
sint64 receiverTime = 2;
sint64 senderTime = 3;
string pubsubTopic = 4;
}
message PagingInfo {
uint64 pageSize = 1;
Index cursor = 2;
enum Direction {
BACKWARD = 0;
FORWARD = 1;
}
Direction direction = 3;
}
message ContentFilter {
string contentTopic = 1;
}
message HistoryQuery {
// the first field is reserved for future use
string pubsubtopic = 2;
repeated ContentFilter contentFilters = 3;
PagingInfo pagingInfo = 4;
}
message HistoryResponse {
// the first field is reserved for future use
repeated WakuMessage messages = 2;
PagingInfo pagingInfo = 3;
enum Error {
NONE = 0;
INVALID_CURSOR = 1;
}
Error error = 4;
}
message HistoryRPC {
string request_id = 1;
HistoryQuery query = 2;
HistoryResponse response = 3;
}
```
#### Index
To perform pagination,
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts.
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node.
- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender.
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
#### PagingInfo
`PagingInfo` holds the information required for pagination. It consists of the following components.
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery`
(or retrieved `WakuMessage`s in a `HistoryResponse`).
- `cursor`: holds the `Index` of a `WakuMessage`.
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
#### ContentFilter
`ContentFilter` carries the information required for filtering historical messages.
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
#### HistoryQuery
RPC call to query historical messages.
- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved.
This field denotes the pubsub topic on which `WakuMessage`s are published.
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
Leaving this field empty means no filter on the pubsub topic of message history is requested.
This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
Leaving this field empty means no filter on the content topic of message history is required.
This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
- `PagingInfo` holds the information required for pagination.
Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`.
It is RECOMMENDED that the queried node defines a maximum page size internally.
If the querying node leaves the `pageSize` unspecified,
or if the `pageSize` exceeds the maximum page size,
the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
This allows mitigation of long response time for `HistoryQuery`.
In the forward pagination request,
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
(and vise versa for the backward pagination).
Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
#### Sorting Messages
The queried node MUST sort the `WakuMessage` based on their `Index`,
where the `senderTime` constitutes the most significant part and the `digest` comes next, and
then perform pagination on the sorted result.
As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one.
Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process.
However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and
consistent across all the nodes.
This has the benefit of `cursor` reusability i.e.,
a `cursor` obtained from one node can be consistently used to query from another node.
However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and
nodes' clock asynchrony.
#### HistoryResponse
RPC call to respond to a HistoryQuery call.
- The `messages` field MUST contain the messages found,
these are [14/WAKU2-MESSAGE](../14/message.md) types.
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries.
The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`).
The `direction` is the same direction as in the corresponding `HistoryQuery`.
In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging).
Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent.
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).
The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different.
- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`.
`NONE` stands for no error.
This is also the default value.
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
## Security Consideration
The main security consideration to take into account while using this protocol is that a querying node have to reveal their content filters of interest to the queried node, hence potentially compromising their privacy.
## Future Work
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md) they are interested in.
As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](../14/message.md).
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries,
as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the request i.e., anonymous communication.
That is the querying node shall hide all its PII in its history request e.g., its IP address.
This can happen by the utilization of a proxy server or by using Tor.
Note that the current structure of historical requests does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve query anonymity.
\<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address--\>
- By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic,
the queried node learns nothing about the query.
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
\<!-- TODO: add a reference for PSIs? --\> \<!-- TODO: more techniques to be included --\>
\<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).--\>
- **Robust and verifiable timestamps**: Messages timestamp is a way to show that the message existed prior to some point in time.
However, the lack of timestamp verifiability can create room for a range of attacks,
including injecting messages with invalid timestamps pointing to the far future.
To better understand the attack,
consider a store node whose current clock shows `2021-01-01 00:00:30` (and assume all the other nodes have a synchronized clocks +-20seconds).
The store node already has a list of messages,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
that are sorted based on their timestamp.
An attacker sends a message with an arbitrary large timestamp e.g.,
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
The store node places `m'` at the end of the list,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30)`.
Now another message arrives with a valid timestamp e.g.,
`(m11, 2021-01-01 00:00:45)`.
However, since its timestamp precedes the malicious message `m'`,
it gets placed before `m'` in the list i.e.,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
In fact, for the next 10 hours,
`m'` will always be considered as the most recent message and
served as the last message to the querying nodes irrespective of how many other messages arrive afterward.
A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp.
One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
block height in Blockchain-based timestamps.
That is, messages contain the most recent block height perceived by their senders at the time of message generation.
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
seconds (e.g., in Ethereum 2.0) from the time of origination.
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [14/WAKU2-MESSAGE](../14/message.md)
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
3. [11/WAKU2-RELAY](../11/relay.md)
4. [Open timestamps](https://opentimestamps.org/)

216
waku/standards/core/14/message.md Normal file
View File

@@ -0,0 +1,216 @@
---
title: 14/WAKU2-MESSAGE
name: Waku v2 Message
status: draft
category: Standards Track
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Sanaz Taheri \<sanaz@status.im\>
- Aaryamann Challani \<aaryamann@status.im\>
- Lorenzo Delgado \<lorenzo@status.im\>
- Abhimanyu Rawat \<abhi@status.im\>
sidebar_position: 1
---
## Abstract
Waku v2 is a family of modular peer-to-peer protocols for secure communication.
These protocols are designed to be secure, privacy-preserving, and censorship-resistant and can run in resource-restricted environments.
At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and adds capabilities.
The present document specifies the Waku v2 message format, a way to encapsulate the messages sent with specific information security goals, and Whisper/Waku v1 backward compatibility.
## Motivation
When sending messages over Waku, there are multiple requirements:
- One may have a separate encryption layer as part of the application.
- One may want to provide efficient routing for resource-restricted devices.
- One may want to provide compatibility with [Waku v1 envelopes](../6/waku1.md).
- One may want encrypted payloads by default.
- One may want to provide unlinkability to get metadata protection.
This specification attempts to provide for these various requirements.
## Semantics
### Waku Message
A Waku message is constituted by the combination of data payload and attributes that, for example, a *publisher* sends to a *topic* and is eventually delivered to *subscribers*.
Waku message attributes are key-value pairs of metadata associated with a message.
And the message data payload is the part of the transmitted Waku message that is the actual message information.
The data payload is also treated as a Waku message attribute for convenience.
### Message Attributes
* The `payload` attribute MUST contain the message data payload to be sent.
* The `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering,
as described in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
* The `meta` attribute, if present, contains an arbitrary application-specific variable-length byte array with a maximum length limit of 64 bytes.
This attribute can be utilized to convey supplementary details to various Waku protocols, thereby enabling customized processing based on its contents.
* The `version` attribute, if present, contains a version number to discriminate different types of payload encryption.
If omitted, the value SHOULD be interpreted as version 0.
* The `timestamp` attribute, if present, signifies the time at which the message was generated by its sender.
This attribute MAY contain the Unix epoch time in nanoseconds.
If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
* The `ephemeral` attribute, if present, signifies the transient nature of the message.
For example, an ephemeral message SHOULD not be persisted by the Waku network.
If this attribute is set to `true`, the message SHOULD be interpreted as ephemeral.
If, instead, the attribute is omitted or set to `false`, the message SHOULD be interpreted as non-ephemeral.
## Wire Format
The Waku message wire format is specified using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
```protobuf
syntax = "proto3";
message WakuMessage {
bytes payload = 1;
string content_topic = 2;
optional uint32 version = 3;
optional sint64 timestamp = 10;
optional bytes meta = 11;
optional bool ephemeral = 31;
}
```
An example proto file following this specification can be found [here (vacp2p/waku)](https://github.com/vacp2p/waku/blob/main/waku/message/v1/message.proto).
## Payload encryption
The Waku message payload MAY be encrypted.
The message `version` attribute indicates the schema used to encrypt the payload data.
- **Version 0:**
The payload SHOULD be interpreted as unencrypted; additionally, it CAN indicate that the message payload has been encrypted at the application layer.
- **Version 1:**
The payload SHOULD be encrypted using Waku v1 payload encryption specified in [26/WAKU-PAYLOAD](../../application/26/payload.md).
This provides asymmetric and symmetric encryption.
The key agreement is performed out of band.
And provides an encrypted signature and padding for some form of unlinkability.
- **Version 2:**
The payload SHOULD be encoded according to [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)).
Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
Any `version` value not included in this list is reserved for future specification.
And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
## Whisper/Waku v1 envelope compatibility
Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
* Whisper/Waku v1 `topic` field SHOULD be mapped to Waku v2 message's `content_topic` attribute.
* Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
Waku v2 implements a pub/sub messaging pattern over libp2p.
This makes redundant some Whisper/Waku v1 envelope fields (e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
## Deterministic message hashing
In Protocol Buffers v3, the deterministic serialization is not canonical across the different implementations and languages.
It is also unstable across different builds with schema changes due to unknown fields.
To overcome this interoperability limitation, a Waku v2 message's hash MUST be computed following this schema:
```
message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp))
```
If an optional attribute, such as `meta`, is absent, the concatenation of attributes SHOULD exclude it. This recommendation is made to ensure that the concatenation process proceeds smoothly when certain attributes are missing and to maintain backward compatibility.
This hashing schema is deemed appropriate for use cases where a cross-implementation deterministic hash is needed, such as message deduplication and integrity validation. The collision probability offered by this hashing schema can be considered negligible. This is due to the deterministic concatenation order of the message attributes, coupled with using a SHA-2 (256-bit) hashing algorithm.
### Test vectors
Waku message hash computation (`meta` size of 12 bytes):
```
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
message.meta = 0x73757065722d736563726574
message.timestamp = 0x175789bfa23f8400
message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05
```
Waku message hash computation (`meta` size of 64 bytes):
```
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
message.meta = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f
message.timestamp = 0x175789bfa23f8400
message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27
```
Waku message hash computation (`meta` attribute not present):
```
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = 0x010203045445535405060708
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
message.meta = \<not-present\>
message.timestamp = 0x175789bfa23f8400
message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8
```
Waku message hash computation (`payload` length 0):
```
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
message.payload = []
message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
message.meta = 0x73757065722d736563726574
message.timestamp = 0x175789bfa23f8400
message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4
```
## Security Considerations
### Confidentiality, integrity, and authenticity
The level of confidentiality, integrity, and authenticity of the Waku message payload is discretionary.
Accordingly, the application layer shall utilize the encryption and signature schemes supported by Waku v2 to meet the application-specific privacy needs.
### Reliability of the `timestamp` attribute
The Waku message `timestamp` attribute is set by the sender.
Therefore, because message timestamps aren’t independently verified, this attribute is prone to exploitation and misuse.
It should not solely be relied upon for operations such as message ordering.
For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message to a high value so that it always shows up as the most recent message in a chat application.
Applications using Waku messages’ `timestamp` attribute are recommended to use additional methods for more robust message ordering.
An example of how to deal with message ordering against adversarial message timestamps can be found in the Status protocol, see [6/PAYLOADS](../6/waku1.md/#clock-vs-timestamp-and-message-ordering).
### Reliability of the `ephemeral` attribute
The Waku message `ephemeral` attribute is set by the sender.
Since there is currently no incentive mechanism for network participants to behave correctly, this attribute is inherently insecure.
A malicious actor can tamper with the value of a Waku message’s `ephemeral` attribute, and the receiver would not be able to verify the integrity of the message.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [6/WAKU1](/spec/6/)
- [Google Protocol buffers v3](https://developers.google.com/protocol-buffers/)
- [26/WAKU-PAYLOAD](../../application/26/payload.md)
- [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md))
- [6/PAYLOADS](https://specs.status.im/spec/6#clock-vs-timestamp-and-message-ordering)

66
waku/standards/core/15/bridge.md Normal file
View File

@@ -0,0 +1,66 @@
---
title: 15/WAKU-BRIDGE
name: Waku Bridge
status: draft
editor: Hanno Cornelius \<hanno@status.im\>
sidebar_position: 1
---
A bridge between Waku v1 and Waku v2.
## Bridge
A bridge requires supporting both Waku versions:
* Waku v1 - using devp2p RLPx protocol
* Waku v2 - using libp2p protocols
Packets received on the Waku v1 network SHOULD be published just once on the
Waku v2 network. More specifically, the bridge SHOULD publish
this through the Waku Relay (PubSub domain).
Publishing such packet will require the creation of a new `Message` with a
new `WakuMessage` as data field. The `data` and `topic` field from the Waku v1
`Envelope` MUST be copied to the `payload` and `contentTopic` fields of the
`WakuMessage`. Other fields such as nonce, expiry and ttl will be dropped as
they become obsolete in Waku v2.
Before this is done, the usual envelope verification still applies:
* Expiry & future time verification
* PoW verification
* Size verification
Bridging SHOULD occur through the `WakuRelay`, but it MAY also be done on other Waku
v2 protocols (e.g. `WakuFilter`). The latter is however not advised as it will
increase the complexity of the bridge and because of the
[Security Considerations](#security-considerations) explained further below.
Packets received on the Waku v2 network SHOULD be posted just once on the Waku
v1 network. The Waku v2 `WakuMessage` contains only the `payload` and
`contentTopic` fields. The bridge MUST create a new Waku v1 `Envelope` and
copy over the `payload` and `contentFilter` fields to the `data` and `topic`
fields. Next, before posting on the network, the bridge MUST set a new expiry
and ttl and do the PoW nonce calculation.
### Security Considerations
As mentioned above, a bridge will be posting new Waku v1 envelopes, which
requires doing the PoW nonce calculation.
This could be a DoS attack vector, as the PoW calculation will make it more
expensive to post the message compared to the original publishing on the Waku v2
network. Low PoW setting will lower this problem, but it is likely that it is
still more expensive.
For this reason, bridges SHOULD probably be run independently of other nodes, so
that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
traffic.
Bridging functionality SHOULD also be carefully implemented so that messages do
not bounce back and forth between the two networks. The bridge SHOULD properly
track messages with a seen filter so that no amplification can be achieved here.
## Copyright
Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).

638
waku/standards/core/16/rpc.md Normal file
View File

@@ -0,0 +1,638 @@
---
title: 16/WAKU2-RPC
name: Waku v2 RPC API
status: draft
editor: Hanno Cornelius \<hanno@status.im\>
sidebar_position: 1
---
## Introduction
This specification describes the JSON-RPC API that Waku v2 nodes MAY adhere to. Refer to the [Waku v2 specification](../10/waku2.md) for more information on Waku v2.
## Wire Protocol
### Transport
Nodes SHOULD expose an accessible [JSON-RPC](https://www.jsonrpc.org/specification) API. The JSON-RPC version SHOULD be `2.0`. Below is an example request:
```json
{
"jsonrpc":"2.0",
"method":"get_waku_v2_debug_info",
"params":[],
"id":1
}
```
#### Fields
| Field | Description |
| --------- | --------------------------------------------------- |
| `jsonrpc` | Contains the used JSON-RPC version (`Default: 2.0`) |
| `method` | Contains the JSON-RPC method that is being called |
| `params` | An array of parameters for the request |
| `id` | The request ID |
### Types
In this specification, the primitive types `Boolean`, `String`, `Number` and `Null`, as well as the structured types `Array` and `Object`, are to be interpreted according to the [JSON-RPC specification](https://www.jsonrpc.org/specification#conventions). It also adopts the same capitalisation conventions.
The following structured types are defined for use throughout the document:
### WakuMessage
Refer to [`Waku Message` specification](../14/message.md) for more information.
`WakuMessage` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ---: | :---: | :---: | --- |
| `payload` | `String` | mandatory | The message payload as a [base64 (with padding)](https://datatracker.ietf.org/doc/html/rfc4648) encoded data string |
| `contentTopic` | `String` | optional | Message content topic for optional content-based filtering |
| `version` | `Number` | optional | Message version. Used to indicate type of payload encryption. Default version is 0 (no payload encryption). |
| `timestamp` | `Number` | optional | The time at which the message is generated by its sender. This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. |
| `ephemeral` | `Boolean` | optional | This flag indicates the transient nature of the message. Indicates if the message is eligible to be stored by the `store` protocol, [13/WAKU2-STORE](../13/store.md). |
## Method naming
The JSON-RPC methods in this document are designed to be mappable to HTTP REST endpoints. Method names follow the pattern `\<method_type\>_waku_\<protocol_version\>_\<api\>_\<api_version\>_\<resource\>`
- `\<method_type\>`: prefix of the HTTP method type that most closely matches the JSON-RPC function. Supported `method_type` values are `get`, `post`, `put`, `delete` or `patch`.
- `\<protocol_version\>`: Waku version. Currently **v2**.
- `\<api\>`: one of the listed APIs below, e.g. `store`, `debug`, or `relay`.
- `\<api_version\>`: API definition version. Currently **v1** for all APIs.
- `\<resource\>`: the resource or resource path being addressed
The method `post_waku_v2_relay_v1_message`, for example, would map to the HTTP REST endpoint `POST /waku/v2/relay/v1/message`.
## Debug API
### Types
The following structured types are defined for use on the Debug API:
#### WakuInfo
`WakuInfo` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `listenAddresses` | `Array`[`String`] | mandatory | Listening addresses of the node |
| `enrUri` | `String` | optional | ENR URI of the node |
#### WakuInfo
### `get_waku_v2_debug_v1_info`
The `get_waku_v2_debug_v1_info` method retrieves information about a Waku v2 node
#### Parameters
none
#### Response
- [**`WakuInfo`**](#wakuinfo) - information about a Waku v2 node
### `get_waku_v2_debug_v1_version`
The `get_waku_v2_debug_v1_version` method retrieves the version of a Waku v2 node as a string.
The version SHOULD follow [semantic versioning](https://semver.org/).
In case the node's current build is based on a git commit between semantic versions,
the retrieved version string MAY contain the git commit hash alone or in combination with the latest semantic version.
#### Parameters
none
#### Response
- **`string`** - represents the version of a Waku v2 node
## Relay API
Refer to the [Waku Relay specification](../11/relay.md) for more information on the relaying of messages.
### `post_waku_v2_relay_v1_message`
The `post_waku_v2_relay_v1_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The `message` being relayed |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `post_waku_v2_relay_v1_subscriptions`
The `post_waku_v2_relay_v1_subscriptions` method subscribes a node to an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being subscribed to |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `delete_waku_v2_relay_v1_subscriptions`
The `delete_waku_v2_relay_v1_subscriptions` method unsubscribes a node from an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being unsubscribed from |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_relay_v1_messages`
The `get_waku_v2_relay_v1_messages` method returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
#### Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Relay Private API
The Private API provides functionality to encrypt/decrypt `WakuMessage` payloads using either symmetric or asymmetric cryptography. This allows backwards compatibility with [Waku v1 nodes](../6/waku1.md).
It is the API client's responsibility to keep track of the keys used for encrypted communication. Since keys must be cached by the client and provided to the node to encrypt/decrypt payloads, a Private API SHOULD NOT be exposed on non-local or untrusted nodes.
### Types
The following structured types are defined for use on the Private API:
#### KeyPair
`KeyPair` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `privateKey` | `String` | mandatory | Private key as hex encoded data string |
| `publicKey` | `String` | mandatory | Public key as hex encoded data string |
### `get_waku_v2_private_v1_symmetric_key`
Generates and returns a symmetric key that can be used for message encryption and decryption.
#### Parameters
none
#### Response
- **`String`** - A new symmetric key as hex encoded data string
### `get_waku_v2_private_v1_asymmetric_keypair`
Generates and returns a public/private key pair that can be used for asymmetric message encryption and decryption.
#### Parameters
none
#### Response
- **[`KeyPair`](#keypair)** - A new public/private key pair as hex encoded data strings
### `post_waku_v2_private_v1_symmetric_message`
The `post_waku_v2_private_v1_symmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
Before being relayed, the message payload is encrypted using the supplied symmetric key. The client MUST provide a symmetric key.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The (unencrypted) `message` being relayed |
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload encryption. This field MUST be included if symmetric key cryptography is selected |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `post_waku_v2_private_v1_asymmetric_message`
The `post_waku_v2_private_v1_asymmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
Before being relayed, the message payload is encrypted using the supplied public key. The client MUST provide a public key.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
| `message` | [`WakuMessage`](#wakumessage) | mandatory | The (unencrypted) `message` being relayed |
| `publicKey` | `String` | mandatory | The hex encoded public key to use for payload encryption. This field MUST be included if asymmetric key cryptography is selected |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_private_v1_symmetric_messages`
The `get_waku_v2_private_v1_symmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied symmetric key. The client MUST provide a symmetric key.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload decryption. This field MUST be included if symmetric key cryptography is selected |
#### Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_private_v1_asymmetric_messages`
The `get_waku_v2_private_v1_asymmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied private key. The client MUST provide a private key.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
| `privateKey` | `String` | mandatory | The hex encoded private key to use for payload decryption. This field MUST be included if asymmetric key cryptography is selected |
#### Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Store API
Refer to the [Waku Store specification](../13/store.md) for more information on message history retrieval.
### Types
The following structured types are defined for use on the Store API:
#### StoreResponse
`StoreResponse` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `messages` | `Array`[[`WakuMessage`](#wakumessage)] | mandatory | Array of retrieved historical messages |
| `pagingOptions` | [`PagingOptions`](#pagingOptions) | [conditional](#get_waku_v2_store_v1_messages) | Paging information from which to resume further historical queries |
#### PagingOptions
`PagingOptions` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: |:-----------------:| :---: |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `pageSize` | `Number` | mandatory | Number of messages to retrieve per page |
| `cursor` | [`Index`](#index) | optional | Message [`Index`](#index) from which to perform pagination. If not included and `forward` is set to `true`, paging will be performed from the beginning of the list. If not included and `forward` is set to `false`, paging will be performed from the end of the list. |
| `forward` | `Bool` | mandatory | `true` if paging forward, `false` if paging backward |
#### Index
`Index` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |-------------------------------------------------------------------------------------------|
| `digest` | `String` | mandatory | A hash for the message at this [`Index`](#index) |
| `receivedTime` | `Number` | mandatory | UNIX timestamp in nanoseconds at which the message at this [`Index`](#index) was received |
#### ContentFilter
`ContentFilter` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentTopic` | `String` | mandatory | The content topic of a [`WakuMessage`](#wakumessage) |
### `get_waku_v2_store_v1_messages`
The `get_waku_v2_store_v1_messages` method retrieves historical messages on specific content topics. This method MAY be called with [`PagingOptions`](#pagingoptions), to retrieve historical messages on a per-page basis. If the request included [`PagingOptions`](#pagingoptions), the node MUST return messages on a per-page basis and include [`PagingOptions`](#pagingoptions) in the response. These [`PagingOptions`](#pagingoptions) MUST contain a `cursor` pointing to the [`Index`](#index) from which a new page can be requested.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `pubsubTopic` | `String` | optional | The pubsub topic on which a [`WakuMessage`](#wakumessage) is published |
| `contentFilters` | `Array`[[`ContentFilter`](#contentfilter)] | optional | Array of content filters to query for historical messages |
| `startTime` | `Number`| optional | The inclusive lower bound on the [`timestamp`](../14/message.md/#message-attributes) of queried [`WakuMessage`s](#wakumessage). This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. |
| `endTime` | `Number` | optional | The inclusive upper bound on the [`timestamp`](../14/message.md/#message-attributes) of queried [`WakuMessage`s](#wakumessage). This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. |
| `pagingOptions` | [`PagingOptions`](#pagingoptions) | optional | Pagination information |
#### Response
- [**`StoreResponse`**](#storeresponse) - the response to a `query` for historical messages.
## Filter API
Refer to the [Waku Filter specification](../12/filter.md) for more information on content filtering.
### Types
The following structured types are defined for use on the Filter API:
#### ContentFilter
`ContentFilter` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentTopic` | `String` | mandatory | message content topic |
### `post_waku_v2_filter_v1_subscription`
The `post_waku_v2_filter_v1_subscription` method creates a subscription in a [light node](../12/filter.md/#rationale) for messages that matches a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentFilters` | `Array`[[`ContentFilter`](#contentfilter)] | mandatory | Array of content filters being subscribed to |
| `topic` | `String` | optional | Message topic |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `delete_waku_v2_filter_v1_subscription`
The `delete_waku_v2_filter_v1_subscription` method removes subscriptions in a [light node](../12/filter.md/#rationale) matching a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentFilters` | `Array`[[`ContentFilter`](#contentfilter)] | mandatory | Array of content filters being unsubscribed from |
| `topic` | `String` | optional | Message topic |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
### `get_waku_v2_filter_v1_messages`
The `get_waku_v2_filter_v1_messages` method returns a list of messages that were received on a subscribed content `topic` after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled content `topic`. If no message has yet been received on the polled content `topic`, the server SHOULD respond with an empty list. This method can be used to poll a content `topic` for new messages.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `contentTopic` | `String` | mandatory | The content topic to poll for the latest messages |
#### Response
- **`Array`[[`WakuMessage`](#wakumessage)]** - the latest `messages` on the polled content `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Admin API
The Admin API provides privileged accesses to the internal operations of a Waku v2 node.
### Types
The following structured types are defined for use on the Admin API:
#### WakuPeer
`WakuPeer` is an `Object` containing the following fields:
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `multiaddr` | `String` | mandatory | Multiaddress containing this peer's location and identity |
| `protocol` | `String` | mandatory | Protocol that this peer is registered for |
| `connected` | `bool` | mandatory | `true` if peer has active connection for this `protocol`, `false` if not |
### `get_waku_v2_admin_v1_peers`
The `get_waku_v2_admin_v1_peers` method returns an array of peers registered on this node. Since a Waku v2 node may open either continuous or ad hoc connections, depending on the negotiated protocol, these peers may have different connected states. The same peer MAY appear twice in the returned array, if it is registered for more than one protocol.
#### Parameters
none
#### Response
- **`Array`[[`WakuPeer`](#wakupeer)]** - Array of peers registered on this node
### `post_waku_v2_admin_v1_peers`
The `post_waku_v2_admin_v1_peers` method connects a node to a list of peers.
#### Parameters
| Field | Type | Inclusion | Description |
| ----: | :---: | :---: |----------- |
| `peers` | `Array`[`String`] | mandatory | Array of peer `multiaddrs` to connect to. Each `multiaddr` must contain the [location and identity addresses](https://docs.libp2p.io/concepts/addressing/) of a peer. |
#### Response
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Example usage
### Store API
#### `get_waku_v2_store_v1_messages`
This method is part of the `store` API and the specific resources to retrieve are (historical) `messages`. The protocol (`waku`) is on `v2`, whereas the Store API definition is on `v1`.
1. `get` *all* the historical messages for content topic **"/waku/2/default-content/proto"**; no paging required
#### Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":["", [{"contentTopic":"/waku/2/default-content/proto"}]]}' --header "Content-Type: application/json" http://localhost:8545```
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"method": "get_waku_v2_store_v1_messages",
"params": [
"",
[
{"contentTopic": "/waku/2/default-content/proto"}
]
]
}
```
#### Response
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"result": {
"messages": [
{
"payload": dGVzdDE,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
},
{
"payload": dGVzdDI,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
},
{
"payload": dGVzdDM,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
}
],
"pagingInfo": null
},
"error": null
}
```
---
2. `get` a single page of historical messages for content topic **"/waku/2/default-content/proto"**; 2 messages per page, backward direction. Since this is the initial query, no `cursor` is provided, so paging will be performed from the end of the list.
#### Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"method": "get_waku_v2_store_v1_messages",
"params": [
"",
[
{"contentTopic": "/waku/2/default-content/proto"}
],
{
"pageSize": 2,
"forward": false
}
]
}
```
#### Response
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"result": {
"messages": [
{
"payload": dGVzdDI,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
},
{
"payload": dGVzdDM,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
}
],
"pagingInfo": {
"pageSize": 2,
"cursor": {
"digest": "abcdef",
"receivedTime": 1605887187000000000
},
"forward": false
}
},
"error": null
}
```
---
3. `get` the next page of historical messages for content topic **"/waku/2/default-content/proto"**, using the cursor received above; 2 messages per page, backward direction.
#### Request
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[ "", [{"contentTopic":"/waku/2/default-content/proto"}],{"pageSize":2,"cursor":{"digest":"abcdef","receivedTime":1605887187000000000},"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"method": "get_waku_v2_store_v1_messages",
"params": [
"",
[
{"contentTopic": "/waku/2/default-content/proto"}
],
{
"pageSize": 2,
"cursor": {
"digest": "abcdef",
"receivedTime": 1605887187000000000
},
"forward": false
}
]
}
```
#### Response
```jsonrpc
{
"jsonrpc": "2.0",
"id": "id",
"result": {
"messages": [
{
"payload": dGVzdDE,
"contentTopic": "/waku/2/default-content/proto",
"version": 0
},
],
"pagingInfo": {
"pageSize": 2,
"cursor": {
"digest": "123abc",
"receivedTime": 1605866187000000000
},
"forward": false
}
},
"error": null
}
```
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [JSON-RPC specification](https://www.jsonrpc.org/specification)
1. [LibP2P Addressing](https://docs.libp2p.io/concepts/addressing/)
1. [LibP2P PubSub specification - topic descriptor](https://github.com/libp2p/specs/tree/master/pubsub#the-topic-descriptor)
1. [Waku v2 specification](../10/waku2.md)
1. [IETF RFC 4648 - The Base16, Base32, and Base64 Data Encodings](https://datatracker.ietf.org/doc/html/rfc4648)

43
waku/standards/core/17/images/rln-message-verification.msc Normal file
View File

@@ -0,0 +1,43 @@
# Sequence diagram for RLN Relay protocol (publishing,routing, and slashing)
msc {
hscale="1",
wordwraparcs=true;
a [label=" "],
b [label=" "],
c [label=" "],
d [label=" "],
e [label=" "];
a rbox a [label="Relay Node: Publisher"],
b rbox b [label="Relay Node: Router"],
c rbox c [label="Relay Node"],
d rbox d [label="Relay Node"],
e note e [label="Membership Contract"];
|||;
b box b [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "],
c box c [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "],
d box d [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "];
|||;
...,
a -> a [label="Keep track of epoch"],
b -> b [label="Keep track of epoch"],
c -> c [label="Keep track of epoch"],
d -> d [label="Keep track of epoch"];
a box a [label=" \n Message: the intended message \n \n epoch: the current epoch \n "];
a box a [label=" \n A(x) = sk + H(sk, epoch)x \n \n shareX = H(message), shareY = A(shareX) \n \n nullifier = H(H(sk,epoch)) \n "];
a box a [label=" \n zkProof: generate the proof using zkSNARK \n "];
|||;
a => b [label="Message, epoch, proofBundle:(shareX, shareY, nullifier, zkProof) \n "];
b box b [label="1. If the received epoch is far from the current epoch"];
b -x c [label="Do not relay"];
b box b [label=" \n 2. If verification of zkProof failed \n "];
b -x c [label="Do not relay"];
b box b [label=" \n 3. If identical nullifier exists in the nullifierMap, \n \n extract the publisher sk \n "];
b -x c [label="Do not relay"];
b => e [label="Slash the publisher: Unlbock the deposit associated with sk"];
e => b [label="x ETH"];
b box b [label=" \n 4. If none of 1-3 happens, update the nullifierMap \n "];
b => c [label="Relay"];
b => d [label="Relay"];
}

BIN
waku/standards/core/17/images/rln-message-verification.png Normal file
View File

Binary file not shown.

33
waku/standards/core/17/images/rln-relay-reg-with-tree-on-chain.msc Normal file
View File

@@ -0,0 +1,33 @@
# Sequence diagram for RLN Relay protocol (registration)
msc {
hscale = "2";
d [label = " "], a [label = " "],b [label = " "];
a rbox a [label="Relay Node A"],
b note b [label="Membership Contract"],
d rbox d [label = "Relay Node B"];
b abox b [ label=" \n Listening to the membership contract \n "] ;
a box a [ label=" \n Generate sk,pk \n "] ;
a=>b [ label = " \n Register(pk, x ETH) \n " ] ;
a box a [ label=" \n Listening to the membership contract \n "] ;
b box b [label=" \n Insert pk as a leaf to the tree \n \n index: The index of the inserted leaf \n \n root: The updated tree root \n \n authPath: The authentication path \n "];
|||;
b=>a [ label = "index, root, authPath"];
|||;
..., ---;
... [ label = "Other relay nodes register and the membership tree gets updatetd" ];
..., ---;
a=>b [ label = "getRoot()" ] ;
b box b [label=" \n root: Get the current root\n "];
b=>a [ label = "root"];
..., --- [ label = " " ];
a=>b [ label = "getAuthPath(index)" ] ;
b box b [label=" \n authPath: Calculate the authentication path of the leaf with the given index and based on the current tree\n "];
b=>a [ label = "authPath"];
}

25
waku/standards/core/17/images/rln-relay.msc Normal file
View File

@@ -0,0 +1,25 @@
# Sequence diagram for RLN Relay protocol (registration)
msc {
hscale = "1.3";
d [label = " "], a [label = " "],b [label = " "];
a rbox a [label="Relay Node A"],
b note b [label="Membership Contract"],
d rbox d [label = "Relay Node B"];
|||;
d abox d [ label=" \n Listening to the membership contract \n "] ;
a box a [ label=" \n Generate sk,pk \n "] ;
a=>b [ label = "Register(pk, x ETH)" ] ;
a abox a [ label=" \n Listening to the membership contract \n "] ;
b box b [label=" \n Insert pk to the list. \n Emit an event announcing the insertion of pk and its index in the list. \n "];
|||;
---;
b abox b [ label=" \n Block containing the insertion transaction is mined \n "] ;
b=>a [ label = "Insert(pk, index)"];
b=>d [ label = "Insert(pk, index)"];
}

BIN
waku/standards/core/17/images/rln-relay.png Normal file
View File

Binary file not shown.

246
waku/standards/core/17/rln-relay.md Normal file
View File

@@ -0,0 +1,246 @@
---
title: 17/WAKU2-RLN-RELAY
name: Waku v2 RLN Relay
status: draft
editor: Sanaz Taheri \<sanaz@status.im\>
contributors:
- Oskar Thorén \<oskarth@titanproxy.com\>
- Aaryamann Challani \<aaryamann@status.im\>
sidebar_position: 1
---
The `17/WAKU2-RLN-RELAY` protocol is an extension of `11/WAKU2-RELAY` which additionally provides spam protection using [Rate Limiting Nullifiers (RLN)](../../../../vac/32/rln-v1.md).
The security objective is to contain spam activity in a GossipSub network by enforcing a global messaging rate to all the peers.
Peers that violate the messaging rate are considered spammers and their message is considered spam.
Spammers are also financially punished and removed from the system.
\<!-- **Protocol identifier***: `/vac/waku/waku-rln-relay/2.0.0-alpha1` --\>
# Motivation
In open and anonymous p2p messaging networks, one big problem is spam resistance.
Existing solutions, such as Whisper’s proof of work are computationally expensive hence not suitable for resource-limited nodes.
Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.
We augment the [`11/WAKU2-RELAY`](/spec/11) protocol with a novel construct of [RLN](/spec/32) to enable an efficient economic spam prevention mechanism that can be run in resource-constrained environments.
# Flow
The messaging rate is defined by the `period` which indicates how many messages can be sent in a given period.
We define an `epoch` as $\lceil$ `unix_time` / `period` $\rceil$. For example, if `unix_time` is `1644810116` and we set `period` to `30`, then `epoch` is $\lceil$`(unix_time/period)`$\rceil$ `= 54827003`.
Note that `epoch` refers to epoch in RLN and not Unix epoch. This means a message can only be sent every period, where period is up to the application.
See see section [Recommended System Parameters](#recommended-system-parameters) for some recommended ways to set a sensible `period` value depending on the application.
Peers subscribed to a spam-protected `pubsubTopic` are only allowed to send one message per `epoch`.
The higher-level layers adopting `17/WAKU2-RLN-RELAY` MAY choose to enforce the messaging rate for `WakuMessages` with a specific `contentTopic` published on a `pubsubTopic`.
## Setup and Registration
Peers subscribed to a specific `pubsubTopic` form a [RLN group](/spec/32).
\<!-- link to the RLN group definition in the RLN RFC --\>
Peers MUST be registered to the RLN group to be able to publish messages.
Registration is moderated through a smart contract deployed on the Ethereum blockchain.
Each peer has an [RLN key pair](/spec/32) denoted by `sk` and `pk`.
The secret key `sk` is secret data and MUST be persisted securely by the peer.
The state of the membership contract contains the list of registered members' public identity keys i.e., `pk`s.
For the registration, a peer creates a transaction that invokes the registration function of the contract via which registers its `pk` in the group.
The transaction also transfers some amount of ether to the contract to be staked.
This amount is denoted by `staked_fund` and is a system parameter.
The peer who has the secret key `sk` associated with a registered `pk` would be able to withdraw a portion `reward_portion` of the staked fund by providing valid proof. \<!-- a secure way to prove the possession of a pk is yet under discussion, maybe via commit and reveal --\>
`reward_portion` is also a system parameter.
Note that `sk` is initially only known to its owning peer however, it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per `epoch`.
An overview of registration is illustrated in Figure 1.
![Figure 1: Registration.](./images/rln-relay.png)
## Publishing
To publish at a given `epoch`, the publishing peer proceeds based on the regular [`11/WAKU2-RELAY`](/spec/11) protocol.
However, to protect against spamming, each `WakuMessage` (which is wrapped inside the `data` field of a PubSub message) MUST carry a [`RateLimitProof`](##RateLimitProof) with the following fields.
Section [Payload](#payloads) covers the details about the type and encoding of these fields.
The `merkle_root` contains the root of the Merkle tree.
The `epoch` represents the current epoch.
The `nullifier` is an internal nullifier acting as a fingerprint that allows specifying whether two messages are published by the same peer during the same `epoch`.
The `nullifier` is a deterministic value derived from `sk` and `epoch` therefore any two messages issued by the same peer (i.e., using the same `sk`) for the same `epoch` are guaranteed to have identical `nullifier`s.
The `share_x` and `share_y` can be seen as partial disclosure of peer's `sk` for the intended `epoch`.
They are derived deterministically from peer's `sk` and current `epoch` using [Shamir secret sharing scheme](/spec/32).
If a peer discloses more than one such pair (`share_x`, `share_y`) for the same `epoch`, it would allow full disclosure of its `sk` and hence get access to its staked fund in the membership contract.
The `proof` field is a zero-knowledge proof signifying that:
1. The message owner is the current member of the group i.e., her/his identity commitment key `pk` is part of the membership group Merkle tree with the root `merkle_root`.
2. `share_x` and `share_y` are correctly computed.
3. The `nullifier` is constructed correctly.
For more details about the proof generation check [RLN](/spec/32)
The proof generation relies on the knowledge of two pieces of private information i.e., `sk` and `authPath`.
The `authPath` is a subset of Merkle tree nodes by which a peer can prove the inclusion of its `pk` in the group. \<!-- TODO refer to RLN RFC for authPath def --\>
The proof generation also requires a set of public inputs which are: the Merkle tree root `merkle_root`, the current `epoch`, and the message for which the proof is going to be generated.
In `17/WAKU2-RLN-RELAY`, the message is the concatenation of `WakuMessage`'s `payload` filed and its `contentTopic` i.e., `payload||contentTopic`.
## Group Synchronization
Proof generation relies on the knowledge of Merkle tree root `merkle_root` and `authPath` which both require access to the membership Merkle tree.
Getting access to the Merkle tree can be done in various ways.
One way is that all the peers construct the tree locally.
This can be done by listening to the registration and deletion events emitted by the membership contract.
Peers MUST update the local Merkle tree on a per-block basis.
This is discussed further in the [Merkle Root Validation](#merkle-root-validation) section.
Another approach for synchronizing the state of slashed `pk`s is to disseminate such information through a p2p GossipSub network to which all peers are subscribed.
This is in addition to sending the deletion transaction to the membership contract.
The benefit of an off-chain slashing is that it allows real-time removal of spammers as opposed to on-chain slashing in which peers get informed with a delay,
where the delay is due to mining the slashing transaction.
For the group synchronization, one important security consideration is that peers MUST make sure they always use the most recent Merkle tree root in their proof generation.
The reason is that using an old root can allow inference about the index of the user's `pk` in the membership tree hence compromising user privacy and breaking message unlinkability.
## Routing
Upon the receipt of a PubSub message via [`11/WAKU2-RELAY`](/spec/11) protocol, the routing peer parses the `data` field as a `WakuMessage` and gets access to the `RateLimitProof` field.
The peer then validates the `RateLimitProof` as explained next.
### Epoch Validation
If the `epoch` attached to the message is more than `max_epoch_gap` apart from the routing peer's current `epoch` then the message is discarded and considered invalid.
This is to prevent a newly registered peer from spamming the system by messaging for all the past epochs.
`max_epoch_gap` is a system parameter for which we provide some recommendations in section [Recommended System Parameters](#recommended-system-parameters).
### Merkle Root Validation
The routing peers MUST check whether the provided Merkle root in the `RateLimitProof` is valid.
It can do so by maintaining a local set of valid Merkle roots, which consist of `acceptable_root_window_size` past roots.
These roots refer to the final state of the Merkle tree after a whole block consisting of group changes is processed.
The Merkle roots are updated on a per-block basis instead of a per-event basis.
This is done because if Merkle roots are updated on a per-event basis, some peers could send messages with a root that refers to a Merkle tree state that might get invalidated while the message is still propagating in the network, due to many registrations happening during this time frame.
By updating roots on a per-block basis instead, we will have only one root update per-block processed, regardless on how many registrations happened in a block, and peers will be able to successfully propagate messages in a time frame corresponding to roughly the size of the roots window times the block mining time.
Atomic processing of the blocks are necessary so that even if the peer is unable to process one event, the previous roots remain valid, and can be used to generate valid RateLimitProof's.
This also allows peers which are not well connected to the network to be able to send messages, accounting for network delay.
This network delay is related to the nature of asynchronous network conditions, which means that peers see membership changes asynchronously, and therefore may have differing local Merkle trees.
See [Recommended System Parameters](#recommended-system-parameters) on choosing an appropriate `acceptable_root_window_size`.
### Proof Verification
The routing peers MUST check whether the zero-knowledge proof `proof` is valid.
It does so by running the zk verification algorithm as explained in [RLN](/spec/32).
If `proof` is invalid then the message is discarded.
### Spam detection
To enable local spam detection and slashing, routing peers MUST record the `nullifier`, `share_x`, and `share_y` of incoming messages which are not discarded i.e., not found spam or with invalid proof or epoch.
To spot spam messages, the peer checks whether a message with an identical `nullifier` has already been relayed.
1. If such a message exists and its `share_x` and `share_y` components are different from the incoming message, then slashing takes place.
That is, the peer uses the `share_x` and `share_y` of the new message and the `share'_x` and `share'_y` of the old record to reconstruct the `sk` of the message owner.
The `sk` then can be used to delete the spammer from the group and withdraw a portion `reward_portion` of its staked fund.
2. If the `share_x` and `share_y` fields of the previously relayed message are identical to the incoming message, then the message is a duplicate and shall be discarded.
3. If none is found, then the message gets relayed.
An overview of the routing procedure and slashing is provided in Figure 2.
\<!-- TODO: may consider [validator functions](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation) or [extended validators](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for the spam detection --\>
![Figure 2: Publishing, Routing and Slashing workflow.](./images/rln-message-verification.png)
-------
# Payloads
Payloads are protobuf messages implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
Nodes MAY extend the [14/WAKU2-MESSAGE](/spec/14) with a `rate_limit_proof` field to indicate that their message is not spam.
```diff
syntax = "proto3";
message RateLimitProof {
bytes proof = 1;
bytes merkle_root = 2;
bytes epoch = 3;
bytes share_x = 4;
bytes share_y = 5;
bytes nullifier = 6;
}
message WakuMessage {
bytes payload = 1;
string content_topic = 2;
optional uint32 version = 3;
optional sint64 timestamp = 10;
optional bool ephemeral = 31;
+ optional bytes rate_limit_proof = 21;
}
```
## WakuMessage
`rate_limit_proof` holds the information required to prove that the message owner has not exceeded the message rate limit.
## RateLimitProof
Below is the description of the fields of `RateLimitProof` and their types.
| Parameter | Type | Description |
| ----: | ----------- | ----------- |
| `proof` | array of 256 bytes | the zkSNARK proof as explained in the [Publishing process](##Publishing) |
| `merkle_root` | array of 32 bytes in little-endian order | the root of membership group Merkle tree at the time of publishing the message |
| `share_x` and `share_y`| array of 32 bytes each | Shamir secret shares of the user's secret identity key `sk` . `share_x` is the Poseidon hash of the `WakuMessage`'s `payload` concatenated with its `contentTopic` . `share_y` is calculated using [Shamir secret sharing scheme](/spec/32) | \<!-- todo specify the poseidon hash setting --\>
| `nullifier` | array of 32 bytes | internal nullifier derived from `epoch` and peer's `sk` as explained in [RLN construct](/spec/32)|
# Recommended System Parameters
The system parameters are summarized in the following table, and the recommended values for a subset of them are presented next.
| Parameter | Description |
| ----: |----------- |
| `period` | the length of `epoch` in seconds |
| `staked_fund` | the amount of wei to be staked by peers at the registration |
| `reward_portion` | the percentage of `staked_fund` to be rewarded to the slashers |
| `max_epoch_gap` | the maximum allowed gap between the `epoch` of a routing peer and the incoming message |
| `acceptable_root_window_size` | The maximum number of past Merkle roots to store |
## Epoch Length
A sensible value for the `period` depends on the application for which the spam protection is going to be used.
For example, while the `period` of `1` second i.e., messaging rate of `1` per second, might be acceptable for a chat application, might be too low for communication among Ethereum network validators.
One should look at the desired throughput of the application to decide on a proper `period` value.
In the proof of concept implementation of `17/WAKU2-RLN-RELAY` protocol which is available in [nim-waku](https://github.com/status-im/nim-waku), the `period` is set to `1` second.
Nevertheless, this value is also subject to change depending on user experience.
## Maximum Epoch Gap
We discussed in the [Routing](#routing) section that the gap between the epoch observed by the routing peer and the one attached to the incoming message should not exceed a threshold denoted by `max_epoch_gap` .
The value of `max_epoch_gap` can be measured based on the following factors.
- Network transmission delay `Network_Delay`: the maximum time that it takes for a message to be fully disseminated in the GossipSub network.
- Clock asynchrony `Clock_Asynchrony`: The maximum difference between the Unix epoch clocks perceived by network peers which can be due to clock drifts.
With a reasonable approximation of the preceding values, one can set `max_epoch_gap` as
`max_epoch_gap` $= \lceil \frac{\text{Network Delay} + \text{Clock Asynchrony}}{\text{Epoch Length}}\rceil$ where `period` is the length of the `epoch` in seconds.
`Network_Delay` and `Clock_Asynchrony` MUST have the same resolution as `period` .
By this formulation, `max_epoch_gap` indeed measures the maximum number of `epoch`s that can elapse since a message gets routed from its origin to all the other peers in the network.
`acceptable_root_window_size` depends upon the underlying chain's average blocktime, `block_time`
The lower bound for the `acceptable_root_window_size` SHOULD be set as $acceptable_root_window_size=(Network_Delay)/block_time$
`Network_Delay` MUST have the same resolution as `block_time`.
By this formulation, `acceptable_root_window_size` will provide a lower bound of how many roots can be acceptable by a routing peer.
The `acceptable_root_window_size` should indicate how many blocks may have been mined during the time it takes for a peer to receive a message.
This formula represents a lower bound of the number of acceptable roots.
# Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
# References
1. [RLN documentation](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view)
2. [Public inputs to the RLN circuit](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Public-Inputs)
3. [Shamir secret sharing scheme used in RLN](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Linear-Equation-amp-SSS)
4. [RLN internal nullifier](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Nullifiers)

66
waku/standards/core/19/lightpush.md Normal file
View File

@@ -0,0 +1,66 @@
---
title: 19/WAKU2-LIGHTPUSH
name: Waku v2 Light Push
status: draft
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Daniel Kaiser \<danielkaiser@status.im\>
sidebar_position: 1
---
**Protocol identifier**: `/vac/waku/lightpush/2.0.0-beta1`
## Motivation and Goals
Light nodes with short connection windows and limited bandwidth wish to publish messages into the Waku network.
Additionally, there is sometimes a need for confirmation that a message has been received "by the network"
(here, at least one node).
`19/WAKU2-LIGHTPUSH` is a request/response protocol for this.
## Payloads
```protobuf
syntax = "proto3";
message PushRequest {
string pubsub_topic = 1;
WakuMessage message = 2;
}
message PushResponse {
bool is_success = 1;
// Error messages, etc
string info = 2;
}
message PushRPC {
string request_id = 1;
PushRequest request = 2;
PushResponse response = 3;
}
```
### Message Relaying
Nodes that respond to `PushRequests` MUST either
relay the encapsulated message via [11/WAKU2-RELAY](../11/relay.md) protocol on the specified `pubsub_topic`,
or forward the `PushRequest` via 19/LIGHTPUSH on a [44/WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md) stem.
If they are unable to do so for some reason, they SHOULD return an error code in `PushResponse`.
## Security Considerations
Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions.
This can be done by rate limiting via [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/).
Note that the above is currently not fully implemented.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
* [11/WAKU2-RELAY](../11/relay.md)
* [44/WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md)
* [17/WAKU2-RLN-RELAY](../17/rln-relay.md)

176
waku/standards/core/33/discv5.md Normal file
View File

@@ -0,0 +1,176 @@
---
title: 33/WAKU2-DISCV5
name: Waku v2 Discv5 Ambient Peer Discovery
status: draft
editor: Daniel Kaiser \<danielkaiser@status.im\>
contributors:
sidebar_position: 1
---
## Abstract
`33/WAKU2-DISCV5` specifies a modified version of [Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) as a means for ambient node discovery.
[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network for establishing a decentralized network of interconnected Waku2 nodes.
In its current version, the `33/WAKU2-DISCV5` discovery network is isolated from the Ethereum Discovery v5 network.
Isolation improves discovery efficiency, which is especially significant with a low number of Waku nodes compared to the total number of Ethereum nodes.
## Disclaimer
This version of `33/WAKU2-DISCV5` has a focus on timely deployment of an efficient discovery method for [10/WAKU2](../10/waku2.md).
Establishing a separate discovery network is in line with this focus.
However, we are aware of potential resilience problems (see section on security considerations) and are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
and researching hybrid approaches.
## Background and Rationale
[11/WAKU2-RELAY](../11/relay.md) assumes the existence of a network of Waku2 nodes.
For establishing and growing this network, new nodes trying to join the Waku2 network need a means of discovering nodes within the network.
[10/WAKU2](../10/waku2.md) supports the following discovery methods in order of increasing decentralization
* hard coded bootstrap nodes
* [`DNS discovery`](https://rfc.vac.dev/spec/10/#discovery-domain) (based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459))
* `peer-exchange` (work in progress)
* `33/WAKU2-DISCV5` (specified in this document)
The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md) is discovering Waku2 nodes in a decentralized way.
The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network, which allows avoiding hotspots and allows merging the network after a split.
While the other methods provide either a fixed or local set of nodes, `33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.
### Separate Discovery Network
#### w.r.t. Waku2 Relay Network
`33/WAKU2-DISCV5` spans an overlay network separate from the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) network [11/WAKU2-RELAY](../11/relay.md) builds on.
Because it is a P2P network on its own, it also depends on bootstrap nodes.
Having a separate discovery network reduces load on the bootstrap nodes, because the actual work is done by randomly discovered nodes.
This also increases decentralization.
#### w.r.t. Ethereum Discovery v5
`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.
Another simple solution would be taking part in the Ethereum Discovery network, and filtering Waku nodes based on whether they support [31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md).
This solution is more resilient towards eclipse attacks.
However, this discovery method is very inefficient for small percentages of Waku nodes (see [estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)).
It boils down to random walk discovery and does not offer a O(log(n)) hop bound.
The rarer the requested property (in this case Waku), the longer a random walk will take until finding an appropriate node, which leads to a needle-in-the-haystack problem.
Using a dedicated Waku2 discovery network, nodes can query this discovery network for a random set of nodes
and all (well-behaving) returned nodes can serve as bootstrap nodes for other Waku2 protocols.
A more sophisticated solution would be using [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement).
However, in its current state it also has efficiency problems for small percentages of Waku nodes and is still in the design phase ([see here](https://github.com/ethereum/devp2p/issues/199)).
Currently, the Ethereum discv5 network is very efficient in finding other discv5 nodes,
but it is not so efficient for finding discv5 nodes that have a specific property or offer specific services, e.g. Waku.
As part of our [discv5 roadmap](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121), we consider two ideas for future versions of `33/WAKU2-DISCV5`
* [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement) with adjustments (ideally upstream)
* a hybrid solution that uses both a separate discv5 network and a Waku-ENR-filtered Ethereum discv5 network
## Semantics
`33/WAKU2-DISCV5` fully inherits the [discv5 semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
Before announcing their address via Waku2 discv5, nodes SHOULD check if this address is publicly reachable.
Nodes MAY use the [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md) to perform that check.
Nodes SHOULD only announce publicly reachable addresses via Waku2 discv5,
to avoid cluttering peer lists with nodes that are not reachable.
## Wire Format Specification
`33/WAKU2-DISCV5` inherits the [discv5 wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md) except for the following differences
## WAKU2-Specific `protocol-id`
Ethereum discv5:
\<pre\>
\<code\>
header = static-header || authdata
static-header = protocol-id || version || flag || nonce || authdata-size
protocol-id = \<b\>"discv5"\</b\>
version = 0x0001
authdata-size = uint16 -- byte length of authdata
flag = uint8 -- packet type identifier
nonce = uint96 -- nonce of message
\</code\>
\</pre\>
`33/WAKU2-DISCV5`:
\<pre\>
kcode\>
header = static-header || authdata
static-header = protocol-id || version || flag || nonce || authdata-size
protocol-id = \<b\>"d5waku"\</b\>
version = 0x0001
authdata-size = uint16 -- byte length of authdata
flag = uint8 -- packet type identifier
nonce = uint96 -- nonce of message
\</code\>
\</pre\>
## Suggestions for Implementations
Existing discv5 implementations
* can be augmented to make the `protocol-id` selectable using a compile-time flag as in [this feature branch](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim#L34) of nim-eth/discv5.
* can be forked followed by changing the `protocol-id` string as in [go-waku](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go#L135-L137).
## Security Considerations
### Sybil attack
Implementations should limit the number of bucket entries that have the same network parameters (IP address / port) to mitigate Sybil attacks.
### Eclipse attack
Eclipse attacks aim to eclipse certain regions in a DHT.
Malicious nodes provide false routing information for certain target regions.
The larger the desired eclipsed region, the more resources (i.e. controlled nodes) the attacker needs.
This introduces an efficiency versus resilience tradeoff.
Discovery is more efficient if information about target objects (e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address.
If nodes providing specific information are closer to each other, they cover a smaller range in the DHT and are easier to eclipse.
Sybil attacks greatly increase the power of eclipse attacks, because they significantly reduce resources necessary to mount a successful eclipse attack.
## Security Implications of a Separate Discovery Network
A dedicated Waku discovery network is more likely to be subject to successful eclipse attacks (and to DoS attacks in general).
This is because eclipsing in a smaller network requires less resources for the attacker.
DoS attacks render the whole network unusable if the percentage of attacker nodes is sufficient.
Using random walk discovery would mitigate eclipse attacks targeted at specific capabilities, e.g. Waku.
However, this is because eclipse attacks aim at the DHT overlay structure, which is not used by random walks.
So, this mitigation would come at the cost of giving up overlay routing efficiency.
The efficiency loss is especially severe with a relatively small number of Waku nodes.
Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.
## References
1. [10/WAKU2](../10/waku2.md)
1. [11/WAKU2-RELAY](../11/relay.md)
1. [`31/WAKU2-ENR`](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
1. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
1. [`discv5` semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
1. [`discv5` wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
1. [`discv5` topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
1. [Waku DNS discovery](https://rfc.vac.dev/spec/10/#discovery-domain)
1. [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)
1. [`EIP-1459`](https://eips.ethereum.org/EIPS/eip-1459)
1. [`GossipSub`](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
1. [Waku discv5 roadmap discussion](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121)
1. [discovery efficiency estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
1. [implementation: Nim](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim)
1. [implementation: Go](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go)
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

1333
waku/standards/core/36/bindings-api.md Normal file
View File

File diff suppressed because it is too large Load Diff

635
waku/standards/legacy/6/waku1.md Normal file
View File

@@ -0,0 +1,635 @@
---
title: 6/WAKU1
name: Waku v1
status: stable
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Adam Babik \<adam@status.im\>
- Andrea Maria Piana \<andreap@status.im\>
- Dean Eigenmann \<dean@status.im\>
- Kim De Mey \<kimdemey@status.im\>
sidebar_position: 1
---
This specification describes the format of Waku packets within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic envelopes (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
## Motivation
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku packets in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
## Definitions
| Term | Definition |
| --------------- | ----------------------------------------------------------------------------------------|
| **Batch Ack** | An abbreviated term for Batch Acknowledgment |
| **Light node** | A Waku node that does not forward any envelopes through the Messages packet. |
| **Envelope** | Messages sent and received by Waku nodes. Described in [ABNF spec `waku-envelope`](#abnf-specification) |
| **Node** | Some process that is able to communicate for Waku. |
## Underlying Transports and Prerequisites
### Use of DevP2P
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
This protocol needs to advertise the `waku/1` [capability](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
### Gossip based routing
In Whisper, envelopes are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Envelopes are eligible for retransmission until their TTL expires. A node SHOULD relay envelopes to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [8/WAKU-MAIL](../../application/8/mail.md) response. A node SHOULD NOT send an envelope to a peer that it has already sent before.
### Maximum Packet Size
Nodes SHOULD limit the maximum size of both packets and envelopes. If a packet or envelope exceeds its limit, it MUST be dropped.
- **RLPx Packet Size** - This size MUST be checked before a message is decoded.
- **Waku Envelope Size** - Each envelope contained in an RLPx packet MUST then separately be checked against the maximum envelope size.
Clients MAY use their own maximum packet and envelope sizes. The default values are `1.5mb` for the RLPx Packet and `1mb` for a Waku envelope.
## Wire Specification
### Use of RLPx transport protocol
All Waku packets are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
### ABNF specification
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
```abnf
; Packet codes 0 - 127 are reserved for Waku protocol
packet-code = 1*3DIGIT
; rate limits per packet
packet-limit-ip = 1*DIGIT
packet-limit-peerid = 1*DIGIT
packet-limit-topic = 1*DIGIT
; rate limits by size in bytes
bytes-limit-ip = 1*DIGIT
bytes-limit-peerid = 1*DIGIT
bytes-limit-topic = 1*DIGIT
packet-rate-limits = "[" packet-limit-ip packet-limit-peerid packet-limit-topic "]"
bytes-rate-limits = "[" bytes-limit-ip bytes-limit-peerid bytes-limit-topic "]"
pow-requirement-key = 0
bloom-filter-key = 1
light-node-key = 2
confirmations-enabled-key = 3
packet-rate-limits-key = 4
topic-interest-key = 5
bytes-rate-limits-key = 6
status-options = "["
[ pow-requirement-key pow-requirement ]
[ bloom-filter-key bloom-filter ]
[ light-node-key light-node ]
[ confirmations-enabled-key confirmations-enabled ]
[ packet-rate-limits-key packet-rate-limits ]
[ topic-interest-key topic-interest ]
[ bytes-limits-key bytes-rate-limits ]
"]"
status = status-options
status-update = status-options
confirmations-enabled = BIT
light-node = BIT
; pow is "a single floating point value of PoW.
; This value is the IEEE 754 binary representation
; of a 64-bit floating point number packed as a uint64.
; Values of qNAN, sNAN, INF and -INF are not allowed.
; Negative values are also not allowed."
pow = 1*DIGIT "." 1*DIGIT
pow-requirement = pow
; bloom filter is "a byte array"
bloom-filter = *OCTET
waku-envelope = "[" expiry ttl topic data nonce "]"
; List of topics interested in
topic-interest = "[" *10000topic "]"
; 4 bytes (UNIX time in seconds)
expiry = 4OCTET
; 4 bytes (time-to-live in seconds)
ttl = 4OCTET
; 4 bytes of arbitrary data
topic = 4OCTET
; byte array of arbitrary size
; (contains encrypted payload)
data = *OCTET
; 8 bytes of arbitrary data
; (used for PoW calculation)
nonce = 8OCTET
messages = 1*waku-envelope
; version of the confirmation packet
version = 1*DIGIT
; keccak256 hash of the envelopes batch data (raw bytes)
; for which the confirmation is sent
hash = *OCTET
hasherror = *OCTET
; error code
code = 1*DIGIT
; a descriptive error message
description = *ALPHA
error = "[" hasherror code description "]"
errors = *error
response = "[" hash errors "]"
confirmation = "[" version response "]"
; message confirmation packet types
batch-ack = confirmation
message-response = confirmation
; mail server / client specific
p2p-request = waku-envelope
p2p-message = 1*waku-envelope
p2p-request-complete = *OCTET
; packet-format needs to be paired with its
; corresponding packet-format
packet-format = "[" packet-code packet-format "]"
required-packet = 0 status /
1 messages /
22 status-update /
optional-packet = 11 batch-ack /
12 message-response /
126 p2p-request-complete /
126 p2p-request /
127 p2p-message
packet = "[" required-packet [ optional-packet ] "]"
```
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
### Packet Codes
The packet codes reserved for Waku protocol: 0 - 127.
Packets with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
| Name | Int Value |
| -------------------------- | ------------- |
| Status | 0 |
| Messages | 1 |
| Status Update | 22 |
The following message codes are optional, but they are reserved for specific purpose.
| Name | Int Value | Comment |
|----------------------------|-----------|---------|
| Batch Ack | 11 | |
| Message Response | 12 | |
| P2P Request Complete | 125 | |
| P2P Request | 126 | |
| P2P Message | 127 | |
### Packet usage
#### Status
The Status packet serves as a Waku handshake and peers MUST exchange this
packet upon connection. It MUST be sent after the RLPx handshake and prior to
any other Waku packets.
A Waku node MUST await the Status packet from a peer before engaging in other Waku protocol activity with that peer.
When a node does not receive the Status packet from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status packet, the node SHOULD validate the packet
received and validated the Status packet. Note that its peer might not be in
the same state.
When a node is receiving other Waku packets from a peer before a Status
packet is received, the node MUST ignore these packets and SHOULD disconnect from that peer. Status packets received after the handshake is completed MUST also be ignored.
The Status packet MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
#### Messages
This packet is used for sending the standard Waku envelopes.
#### Status Update
The Status Update packet is used to communicate an update of the settings of the node.
The format is the same as the Status packet, all fields are optional.
If none of the options are specified the packet MUST be ignored and considered a noop.
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
be transmitted.
##### PoW Requirement Field
When PoW Requirement is updated, peers MUST NOT deliver envelopes with PoW lower than the PoW Requirement specified.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by envelope size and TTL:
PoW = (2**BestBit) / (size * TTL)
PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
##### Bloom Filter Field
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
LET D[*] = 0
FOREACH i IN { 0, 1, 2 } DO
LET n = S[i]
IF S[3] & (2 ** i) THEN n += 256
D[n] = 1
END FOR
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
If both topic interest and bloom filter are specified, topic interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current topic interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
##### Topic Interest Field
Topic interest is used to share a node's interest in envelopes with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only topic interest is specified, the current bloom filter MUST be discarded and only the updated topic interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node is not currently interested in receiving any envelope.
##### Rate Limits Field
Rate limits is used to inform other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the `status-update` packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
Two rate limits strategies are applied:
1) Number of packets per second
2) Size of packets (in bytes) per second
Both strategies SHOULD be applied per IP address, peer id and topic.
The size limit SHOULD be greater or equal than the maximum packet size.
##### Light Node Field
When the node's `light-node` field is set to true, the node SHOULD NOT forward Envelopes from its peers.
A node connected to a peer with the `light-node` field set to true MUST NOT depend on the peer for forwarding Envelopes.
##### Confirmations Enabled Field
When the node's `confirmations-enabled` field is set to true, the node SHOULD send [message confirmations](#batch-ack-and-message-response) to its peers.
#### Batch Ack and Message Response
Message confirmations tell a node that an envelope originating from it has been received by its peers, allowing a node to know whether an envelope has or has not been received.
A node MAY send a message confirmation for any batch of envelopes received with a Messages packet (`0x01`).
A message confirmation is sent using Batch Ack packet (`0x0B`) or Message Response packet (`0x0C`). The message confirmation is specified in the [ABNF specification](#abnf-specification).
The current `version` in the `confirmation` is `1`.
The supported error codes:
- `1`: time sync error which happens when an envelope is too old or was created in the future (typically because of an unsynchronized clock of a node).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent envelope, a corresponding confirmation is broadcast by one or more peers.
#### P2P Request
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting historic (expired) envelopes from the [Waku Mail Server](../../application/8/mail.md).
#### P2P Message
This packet is used for sending the peer-to-peer envelopes, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of historic (expired) envelopes, which is otherwise not allowed.
#### P2P Request Complete
This packet is used to indicate that all envelopes, requested earlier with a P2P Request packet (`0x7E`), have been sent via one or more P2P Message packets (`0x7F`).
The content of the packet is explained in the [Waku Mail Server](../../application/8/mail.md) specification.
### Payload Encryption
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
### Packet code Rationale
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
Packet code `0x22` is used to dynamically change the settings of a node.
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without the P2P Message packet it would be impossible to deliver the historic envelopes, since they will be recognized as expired, and the peer will be disconnected for violating the Waku protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
## Additional capabilities
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on [specification](../../application/8/mail.md).
### Light node
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming envelopes, they MUST only send their own envelopes. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in envelopes being dropped between the two.
Light nodes are identified by the `light_node` value in the Status packet.
### Accounting for resources (experimental)
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
| peer | sent | received |
|-------|------|----------|
| peer1 | 0 | 123 |
| peer2 | 10 | 40 |
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
## Upgradability and Compatibility
### General principles and policy
The currently advertised capability is `waku/1`. This needs to be advertised in the `hello` `ÐΞVp2p` [packet](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
If a node supports multiple versions of `waku`, those needs to be explicitly advertised. For example if both `waku/0` and `waku/1` are supported, both `waku/0` and `waku/1` MUST be advertised.
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
1. Waku aims to be compatible with previous and future versions.
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
3. To achieve this, we employ the following two general strategies:
- a) Accretion (including protocol negotiation) over changing data
- b) When we want to change things, we give it a new name (for example, a version number).
Examples:
- We enable bridging between `shh/6` and `waku/1` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
- When we we want to provide a new set of packets that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
### Backwards Compatibility
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
### Waku-Whisper bridging
`waku/1` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
**Roles:**
- Waku client A, only Waku capability
- Whisper client B, only Whisper capability
- WakuWhisper bridge C, both Waku and Whisper capability
**Flow:**
1. A posts envelope; B posts envelope.
2. C picks up envelope from A and B and relays them both to Waku and Whisper.
3. A receives envelope on Waku; B on Whisper.
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a envelope between C1 and C2. I.e. Whisper(\<\>Waku\<\>Whisper)\<\>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
### Forward Compatibility
It is desirable to have a strategy for maintaining forward compatibility between `waku/1` and future version of waku. Here we outline some concerns and strategy for this.
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished is by negotiating the versions of subprotocols, within the `hello` packet nodes transmit their capabilities along with a version. The highest common version should then be used.
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
## Appendix A: Security considerations
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](../../application/8/mail.md#security-considerations) can be found in their respective specifications.
### Scalability and UX
#### Bandwidth usage:
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
#### Gossip-based routing:
Use of gossip-based routing doesn't necessarily scale. It means each node can see an envelope multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
#### Lack of incentives:
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
### Privacy
#### Light node privacy:
The main privacy concern with a light node is that it has to reveal its topic interests (in addition to its IP/ID) to its directed peers. This is because when a light node publishes an envelope, its directed peers will know that the light node owns that envelope (as light nodes do not relay other envelopes). Therefore, the directed peers of a light node can make assumptions about what envelopes (topics) the light node is interested in.
#### Mailserver client privacy:
A mailserver client fetches archival envelopes from a mailserver through a direct connection.
In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs to their topic interests and build a profile for each client over time.
As such, the mailserver client has to trust the mailserver with this level of information.
#### Bloom filter privacy:
By having a bloom filter where only the topics you are interested in are set, you reveal which envelopes you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
#### Privacy guarantees not rigorous:
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
#### Topic hygiene:
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
### Spam resistance
**PoW bad for heterogeneous devices:**
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
### Censorship resistance
**Devp2p TCP port blockable:**
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
## Appendix B: Implementation Notes
### Implementation Matrix
| Client | Spec supported | Details |
|--------|----------------|---------|
| **Status-go** | 0.5 | [details](https://github.com/status-im/status-go/blob/develop/WAKU.md) |
| **Nim-waku** | 1.0 | [details](https://github.com/status-im/nim-waku/blob/master/README.md) |
### Recommendations for clients
Notes useful for implementing Waku mode.
1. Avoid duplicate envelopes
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
2. Topic specific recommendations
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
### Node discovery
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
Known static nodes MAY also be used.
## Changelog
### [Initial Release](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)
- Add section on P2P Request Complete packet and update packet code table.
- Correct the header hierarchy for the status-options fields.
- Consistent use of the words packet, message and envelope.
- Added section on max packet size
- Complete the ABNF specification and minor ABNF fixes.
### Version 1.1
Released [June 09, 2020](https://github.com/vacp2p/specs/commit/33b8d7304c9ebece90ea94e601f11080a8ac2c4d)
- Add rate limit per bytes
### Version 1.0
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
- Removed `version` from handshake
- Changed `RLP` keys from 48,49.. to 0,1..
- Upgraded to `waku/1`
### Version 0.6
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
- Mark spec as Deprecated mode in terms of its lifecycle.
### Version 0.5
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
### Version 0.4
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
- Simplify implementation matrix with latest state
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
- Increased `topic-interest` capacity from 1000 to 10000
### Version 0.3
Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954ab4c315e1b8d210ac7631b6d1428).
- Recommend DNS based node discovery over other Discovery methods.
- Mark spec as Draft mode in terms of its lifecycle.
- Simplify Changelog and misc formatting.
- Handshake/Status packet not compatible with shh/6 nodes; specifying options as association list.
- Include topic-interest in Status handshake.
- Upgradability policy.
- `topic-interest` packet code.
### Version 0.2
Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/waku.md).
- General style improvements.
- Fix ABNF grammar.
- Mailserver requesting/receiving.
- New packet codes: topic-interest (experimental), rate limits (experimental).
- More details on handshake modifications.
- Accounting for resources mode (experimental)
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
- More details on handshake and parameters.
- Describe rate limits in more detail.
- More details on mailserver and mail client API.
- Accounting for resources mode (very experimental).
- Clarify differences with Whisper.
### Version 0.1
Initial version. Released [November 21, 2019](https://github.com/vacp2p/specs/blob/b59b9247f2ac1bf45c75bd3227a2e5dd87b6d7b0/waku.md).
### Differences between shh/6 and waku/1
Summary of main differences between this spec and Whisper v6, as described in [EIP-627](https://eips.ethereum.org/EIPS/eip-627):
- RLPx subprotocol is changed from `shh/6` to `waku/1`.
- Light node capability is added.
- Optional rate limiting is added.
- Status packet has following additional parameters: light-node,
confirmations-enabled and rate-limits
- Mail Server and Mail Client functionality is now part of the specification.
- P2P Message packet contains a list of envelopes instead of a single envelope.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## Footnotes
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.

64
waku/standards/legacy/7/data.md Normal file
View File

@@ -0,0 +1,64 @@
---
title: 7/WAKU-DATA
name: Waku Envelope data field
status: stable
editor: Oskar Thorén \<oskarth@titanproxy.com\>
contributors:
- Dean Eigenmann \<dean@status.im\>
- Kim De Mey \<kimdemey@status.im\>
sidebar_position: 1
---
This specification describes the encryption, decryption and signing of the content in the [data field used in Waku](../../standards/core/6/waku1.md/#abnf-specification).
## Specification
The `data` field is used within the `waku envelope`, the field MUST contain the encrypted payload of the envelope.
The fields that are concatenated and encrypted as part of the `data` field are:
- flags
- auxiliary field
- payload
- padding
- signature
In case of symmetric encryption, a `salt` (a.k.a. AES Nonce, 12 bytes) field MUST be appended.
### ABNF
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
```abnf
; 1 byte; first two bits contain the size of auxiliary field,
; third bit indicates whether the signature is present.
flags = 1OCTET
; contains the size of payload.
auxiliary-field = 4*OCTET
; byte array of arbitrary size (may be zero)
payload = *OCTET
; byte array of arbitrary size (may be zero).
padding = *OCTET
; 65 bytes, if present.
signature = 65OCTET
; 2 bytes, if present (in case of symmetric encryption).
salt = 2OCTET
data = flags auxiliary-field payload padding [signature] [salt]
```
### Signature
Those unable to decrypt the envelope data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the `R`, `S` and `V` parameters of the SECP-256k1 ECDSA signature, in that order. `R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned. `V` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
### Padding
The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

123
waku/standards/legacy/8/mail.md Normal file
View File

@@ -0,0 +1,123 @@
---
title: 8/WAKU-MAIL
name: Waku Mailserver
status: stable
editor: Andrea Maria Piana \<andreap@status.im\>
contributors:
- Adam Babik \<adam@status.im\>
- Dean Eigenmann \<dean@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
sidebar_position: 1
---
## Abstract
In this specification, we describe Mailservers. These are nodes responsible for archiving envelopes and delivering them to peers on-demand.
## Specification
A node which wants to provide mailserver functionality MUST store envelopes from incoming Messages packets (Waku packet-code `0x01`). The envelopes can be stored in any format, however they MUST be serialized and deserialized to the Waku envelope format.
A mailserver SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
### Requesting Historic Envelopes
In order to request historic envelopes, a node MUST send a packet P2P Request (`0x7e`) to a peer providing mailserver functionality. This packet requires one argument which MUST be a Waku envelope.
In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
```abnf
; UNIX time in seconds; oldest requested envelope's creation time
lower = 4OCTET
; UNIX time in seconds; newest requested envelope's creation time
upper = 4OCTET
; array of Waku topics encoded in a bloom filter to filter envelopes
bloom = 64OCTET
; unsigned integer limiting the number of returned envelopes
limit = 4OCTET
; array of a cursor returned from the previous request (optional)
cursor = *OCTET
; List of topics interested in
topics = "[" *1000topic "]"
; 4 bytes of arbitrary data
topic = 4OCTET
payload-without-topic = "[" lower upper bloom limit [ cursor ] "]"
payload-with-topic = "[" lower upper bloom limit cursor [ topics ] "]"
payload = payload-with-topic | payload-without-topic
```
The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit` so that the requester can send another request using the obtained `Cursor` value. What exactly is in the `Cursor` is up to the implementation. The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request to another mailserver because the format or the result MAY be different.
The envelope MUST be encrypted with a symmetric key agreed between the requester and Mailserver.
If `Topics` is used the `Cursor` field MUST be specified for the argument order to be unambiguous. However, it MAY be set to `null`. `Topics` is used to specify which topics a node is interested in. If `Topics` is not empty, a mailserver MUST only send envelopes that belong to a topic from `Topics` list and `Bloom` value MUST be ignored.
### Receiving Historic Envelopes
Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes. A Mailserver MUST limit the amount of messages sent, either by the `Limit` specified in the request or limited to the maximum [RLPx packet size](./waku#maximum-packet-size), whichever limit comes first.
In order to receive historic envelopes from a mailserver, a node MUST trust the selected mailserver, that is allow to receive expired packets with the P2P Message code. By default, such packets are discarded.
Received envelopes MUST be passed through the Whisper envelope pipelines so that they are picked up by registered filters and passed to subscribers.
For a requester, to know that all envelopes have been sent by mailserver, it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by a list with:
```abnf
; array with a Keccak-256 hash of the envelope containing the original request.
request-id = 32OCTET
; array with a Keccak-256 hash of the last sent envelope for the request.
last-envelope-hash = 32OCTET
; array of a cursor returned from the previous request (optional)
cursor = *OCTET
payload = "[" request-id last-envelope-hash [ cursor ] "]"
```
If `Cursor` is not empty, it means that not all envelopes were sent due to the set `Limit` in the request. One or more consecutive requests MAY be sent with `Cursor` field filled in in order to receive the rest of the envelopes.
### Security considerations
There are several security considerations to take into account when running or interacting with Mailservers. Chief among them are: scalability, DDoS-resistance and privacy.
**Mailserver High Availability requirement:**
A mailserver has to be online to receive envelopes for other nodes, this puts a high availability requirement on it.
**Mailserver client privacy:**
A mailserver client fetches archival envelopes from a mailserver through a direct connection.
In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver.
The collection of such information allows the mailserver to link clients' IP/IDs to their topic interests and build a profile for each client over time.
As such, the mailserver client has to trust the mailserver with this level of information.
A similar concern exists for the light nodes and their direct peers which is discussed in the security considerations of [6/WAKU1](/spec/7).
**Mailserver trusted connection:**
A mailserver has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning mailserver can overwhelm an individual node.
## Changelog
| Version | Comment |
| :--------------------------------------------------------------------------------------------: | ------- |
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8) | marked stable as it is in use. |
| 0.2.0 | Add topic interest to reduce bandwidth usage |
| [0.1.0](https://github.com/vacp2p/specs/blob/06d4c736c920526472a507e5d842212843a112ed/wms.md) | Initial Release |
### Difference between wms 0.1 and wms 0.2
- `topics` option
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

409
waku/standards/legacy/9/rpc.md Normal file
View File

@@ -0,0 +1,409 @@
---
title: 9/WAKU-RPC
name: Waku RPC API
status: stable
editor: Andrea Maria Piana \<andreap@status.im\>
contributors:
- Dean Eigenmann \<dean@status.im\>
- Oskar Thorén \<oskarth@titanproxy.com\>
sidebar_position: 1
---
This specification describes the RPC API that Waku nodes MAY adhere to. The unified API allows clients to easily
be able to connect to any node implementation. The API described is privileged as a node stores the keys of clients.
## Introduction
This API is based off the [Whisper V6 RPC API](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API).
## Wire Protocol
### Transport
Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API that can be accessed. The JSON RPC version SHOULD be `2.0`. Below is an example request:
```json
{
"jsonrpc":"2.0",
"method":"waku_version",
"params":[],
"id":1
}
```
#### Fields
| Field | Description |
| --------- | --------------------------------------------------- |
| `jsonrpc` | Contains the used JSON RPC version (`Default: 2.0`) |
| `method` | Contains the JSON RPC method that is being called |
| `params` | An array of parameters for the request |
| `id` | The request ID |
### Objects
In this section you will find objects used throughout the JSON RPC API.
#### Message
The message object represents a Waku message. Below you will find the description of the attributes contained in the message object. A message is the decrypted payload and padding of an [envelope](/spec/7) along with all of its metadata and other extra information such as the hash.
| Field | Type | Description |
| ----: | :--: | ----------- |
| `sig` | string | Public Key that signed this message |
| `recipientPublicKey` | string | The recipients public key |
| `ttl` | number | Time-to-live in seconds |
| `timestamp` | number | Unix timestamp of the message generation |
| `topic` | string | 4 bytes, the message topic |
| `payload` | string | Decrypted payload |
| `padding` | string | Optional padding, byte array of arbitrary length |
| `pow` | number | The proof of work value |
| `hash` | string | Hash of the enveloped message |
#### Filter
The filter object represents filters that can be applied to retrieve messages. Below you will find the description of the attributes contained in the filter object.
| Field | Type | Description |
| ----: | :--: | ----------- |
| `symKeyID` | string | ID of the symmetric key for message decryption |
| `privateKeyID` | string | ID of private (asymmetric) key for message decryption |
| `sig` | string | Public key of the signature |
| `minPow` | number | Minimal PoW requirement for incoming messages |
| `topics` | array | Array of possible topics, this can also contain partial topics |
| `allowP2P` | boolean | Indicates if this filter allows processing of direct peer-to-peer messages |
All fields are optional, however `symKeyID` or `privateKeyID` must be present, it cannot be both. Additionally, the `topics` field is only optional when an asymmetric key is used.
### Methods
#### `waku_version`
The `waku_version` method returns the current version number.
##### Parameters
none
##### Response
- **string** - The version number.
#### `waku_info`
The `waku_info` method returns information about a Waku node.
##### Parameters
none
##### Response
The response is an `Object` containing the following fields:
- **`minPow` [number]** - The current PoW requirement.
- **`maxEnvelopeSize` [float]** - The current maximum envelope size in bytes.
- **`memory` [number]** - The memory size of the floating messages in bytes.
- **`envelopes` [number]** - The number of floating envelopes.
#### `waku_setMaxEnvelopeSize`
Sets the maximum envelope size allowed by this node. Any envelopes larger than this size both incoming and outgoing will be rejected. The envelope size can never exceed the underlying envelope size of `10mb`.
##### Parameters
- **number** - The message size in bytes.
##### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_setMinPoW`
Sets the minimal PoW required by this node.
##### Parameters
- **number** - The new PoW requirement.
##### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_markTrustedPeer`
Marks a specific peer as trusted allowing it to send expired messages.
##### Parameters
- **string** - `enode` of the peer.
##### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_newKeyPair`
Generates a keypair used for message encryption and decryption.
##### Parameters
none
##### Response
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_addPrivateKey`
Stores a key and returns its ID.
##### Parameters
- **string** - Private key as hex bytes.
##### Response
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_deleteKeyPair`
Deletes a specific key if it exists.
##### Parameters
- **string** - ID of the Key pair.
##### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_hasKeyPair`
Checks if the node has a private key of a key pair matching the given ID.
##### Parameters
- **string** - ID of the Key pair.
##### Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getPublicKey`
Returns the public key for an ID.
##### Parameters
- **string** - ID of the Key.
##### Response
- **string** - The public key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getPrivateKey`
Returns the private key for an ID.
##### Parameters
- **string** - ID of the Key.
##### Response
- **string** - The private key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_newSymKey`
Generates a random symmetric key and stores it under an ID. This key can be used to encrypt and decrypt messages where the key is known to both parties.
##### Parameters
none
##### Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_addSymKey`
Stores the key and returns its ID.
##### Parameters
- **string** - The raw key for symmetric encryption hex encoded.
##### Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_generateSymKeyFromPassword`
Generates the key from a password and stores it.
##### Parameters
- **string** - The password.
##### Response
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_hasSymKey`
Returns whether there is a key associated with the ID.
##### Parameters
- **string** - ID of the Key.
##### Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getSymKey`
Returns the symmetric key associated with an ID.
##### Parameters
- **string** - ID of the Key.
##### Response
- **string** - Raw key on success or an [error](https://www.jsonrpc.org/specification#error_object) of failure.
#### `waku_deleteSymKey`
Deletes the key associated with an ID.
##### Parameters
- **string** - ID of the Key.
##### Response
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_subscribe`
Creates and registers a new subscription to receive notifications for inbound Waku messages.
##### Parameters
The parameters for this request is an array containing the following fields:
1. **string** - The ID of the function call, in case of Waku this must contain the value "messages".
2. **object** - The [message filter](#filter).
##### Response
- **string** - ID of the subscription or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
###### Notifications
Notifications received by the client contain a [message](#message) matching the filter. Below is an example notification:
```json
{
"jsonrpc": "2.0",
"method": "waku_subscription",
"params": {
"subscription": "02c1f5c953804acee3b68eda6c0afe3f1b4e0bec73c7445e10d45da333616412",
"result": {
"sig": "0x0498ac1951b9078a0549c93c3f6088ec7c790032b17580dc3c0c9e900899a48d89eaa27471e3071d2de6a1f48716ecad8b88ee022f4321a7c29b6ffcbee65624ff",
"recipientPublicKey": null,
"ttl": 10,
"timestamp": 1498577270,
"topic": "0xffaadd11",
"payload": "0xffffffdddddd1122",
"padding": "0x35d017b66b124dd4c67623ed0e3f23ba68e3428aa500f77aceb0dbf4b63f69ccfc7ae11e39671d7c94f1ed170193aa3e327158dffdd7abb888b3a3cc48f718773dc0a9dcf1a3680d00fe17ecd4e8d5db31eb9a3c8e6e329d181ecb6ab29eb7a2d9889b49201d9923e6fd99f03807b730780a58924870f541a8a97c87533b1362646e5f573dc48382ef1e70fa19788613c9d2334df3b613f6e024cd7aadc67f681fda6b7a84efdea40cb907371cd3735f9326d02854",
"pow": 0.6714754098360656,
"hash": "0x17f8066f0540210cf67ef400a8a55bcb32a494a47f91a0d26611c5c1d66f8c57"
}
}
}
```
#### `waku_unsubscribe`
Cancels and removes an existing subscription. The node MUST stop sending the client notifications.
##### Parameters
- **string** - The subscription ID.
##### Response
- **bool** - `true` or `false`
#### `waku_newMessageFilter`
Creates a new message filter within the node. This filter can be used to poll for new messages that match the criteria.
##### Parameters
The request must contain a [message filter](#filter) as its parameter.
##### Response
- **string** - The ID of the filter.
#### `waku_deleteMessageFilter`
Removes a message filter from the node.
##### Parameters
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
##### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_getFilterMessages`
Retrieves messages that match a filter criteria and were received after the last time this function was called.
##### Parameters
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
##### Response
The response contains an array of [messages](#messages) or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
#### `waku_post`
The `waku_post` method creates a waku envelope and propagates it to the network.
##### Parameters
The parameters is an `Object` containing the following fields:
- **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
- **`pubKey` [string]** `optional` - The public key for message encryption.
- **`sig` [string]** `optional` - The ID of the signing key.
- **`ttl` [number]** - The time-to-live in seconds.
- **`topic` [string]** - 4 bytes message topic.
- **`payload` [string]** - The payload to be encrypted.
- **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
- **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
- **`powTarget` [number]** - Minimal PoW target required for this message.
- **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
*Either the **`symKeyID`** or the **`pubKey`** need to be present. It can not be both.*
#### Response
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
## Changelog
| Version | Comment |
| :--------------------------------------------------------------------------------------:| ---------------- |
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. |
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).