Update markdown-lint.yml

Included preliminary notes related to the onchain component.

---------

Co-authored-by: kaiserd <1684595+kaiserd@users.noreply.github.com>

.github/workflows/.markdownlint.json

@@ -0,0 +1,7 @@
+{
+    "default": true,
+    "MD013": {
+      "tables": false,
+      "code_blocks" : false
+    }
+}

.github/workflows/markdown-lint.yml

@@ -1,22 +1,23 @@
-name: markdown-linting
-
-on:
-  push:
-    branches: 
-      - '**'
-  pull_request:
-    branches: 
-      - '**'
-
-jobs:
-  lint:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v4
-        with:
-          ref: ${{ github.event.pull_request.base.ref }}
-      - name: Markdown Linter
-        uses: DavidAnson/markdownlint-cli2-action@v15
-        with:
-          globs: '/*.md'
+name: markdown-linting
+
+on:
+
+  push:
+    branches: 
+      - '**'
+  pull_request:
+    branches: 
+      - '**'
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+      
+      - name: Markdown Linter
+        uses: DavidAnson/markdownlint-cli2-action@v15
+        with:
+          config: .github/workflows/.markdownlint.json
+          globs: '**/*.md'

README.md

@@ -1,39 +1,47 @@
-# Vac Request For Comments(RFC)
-
-*NOTE*: This repo is WIP. We are currently restructuring the RFC process.
-
-This repository contains specifications from the [Waku](https://waku.org/), [Nomos](https://nomos.tech/), 
-[Codex](https://codex.storage/), and [Status](https://status.app/) projects that are part of the [IFT portfolio](https://free.technology/).
-[Vac](https://vac.dev) is an [IFT service](https://free.technology/services) that will manage the RFC, [Request for Comments](https://en.wikipedia.org/wiki/Request_for_Comments), process within this repository. 
-
-## New RFC Process
-
-This repository replaces the previous `rfc.vac.dev` resource.
-Each project will maintain initial specifications in separate repositories, 
-which may be considered as a **raw** specification.
-All [Vac](https://vac.dev) **raw** specifications and discussions will live in the Vac subdirectory.
-When projects have reached some level of maturity for a specification living in their repository,
-the process of updating the status to **draft** may begin in this repository.
-Specifications will adhere to [1/COSS](./vac/1/coss.md) before obtaining **draft** status.
-
-Implementations should follow specifications as described,
-and all contributions will be discussed before the **stable** status is obtained.
-The goal of this RFC process will to engage all interseted parities and 
-reach a rough consensus for techcinal specifications.
-
-## Contributing
-
-Please see [1/COSS](./vac/1/coss.md) for general guidelines and specification lifecycle.
-
-Feel free to join the [Vac discord](https://discord.gg/Vy54fEWuqC). 
-
-Here's the project board used by core contributors and maintainers: [Projects](https://github.com/orgs/vacp2p/projects/5)
-
-## IFT Projects' Raw Specifications
-
-The repository for each project **raw** specifications:
-- [Vac Raw Specifications](./vac/raw)
-- [Status Raw Specifications](./status/raw)
-- [Waku Raw Specificiations](https://github.com/waku-org/specs/tree/master)
-- [Codex Raw Specifications]()
-- [Nomos Raw Specifications](https://github.com/logos-co/nomos-specs)
+# Vac Request For Comments(RFC)
+
+*NOTE*: This repo is WIP. We are currently restructuring the RFC process.
+
+This repository contains specifications from the [Waku](https://waku.org/), [Nomos](https://nomos.tech/),
+[Codex](https://codex.storage/), and
+[Status](https://status.app/) projects thatare part of the [IFT portfolio](https://free.technology/).
+[Vac](https://vac.dev) is an
+[IFT service](https://free.technology/services) that will manage the RFC,
+[Request for Comments](https://en.wikipedia.org/wiki/Request_for_Comments),
+process within this repository.
+
+## New RFC Process
+
+This repository replaces the previous `rfc.vac.dev` resource.
+Each project will maintain initial specifications in separate repositories,
+which may be considered as a **raw** specification.
+All [Vac](https://vac.dev) **raw** specifications and
+discussions will live in the Vac subdirectory.
+When projects have reached some level of maturity
+for a specification living in their repository,
+the process of updating the status to **draft** may begin in this repository.
+Specifications will adhere to
+[1/COSS](./vac/1/coss.md) before obtaining **draft** status.
+
+Implementations should follow specifications as described,
+and all contributions will be discussed before the **stable** status is obtained.
+The goal of this RFC process will to engage all interseted parities and
+reach a rough consensus for techcinal specifications.
+
+## Contributing
+
+Please see [1/COSS](./vac/1/coss.md) for general guidelines and specification lifecycle.
+
+Feel free to join the [Vac discord](https://discord.gg/Vy54fEWuqC).
+
+Here's the project board used by core contributors and maintainers: [Projects](https://github.com/orgs/vacp2p/projects/5)
+
+## IFT Projects' Raw Specifications
+
+The repository for each project **raw** specifications:
+
+- [Vac Raw Specifications](./vac/raw)
+- [Status Raw Specifications](./status/raw)
+- [Waku Raw Specificiations](https://github.com/waku-org/specs/tree/master)
+- [Codex Raw Specifications](none)
+- [Nomos Raw Specifications](https://github.com/logos-co/nomos-specs)

codex/README.md

@@ -1,3 +1,5 @@
-# Codex RFCs
-
-Codex specifications related to a decentralised data storage platform.
+# Codex RFCs
+
+Specifications related the Codex decentralised data storage platform.
+Visit [Codex specs](https://github.com/codex-storage/codex-spec)
+to view the new Codex specifications currently under discussion.

nomos/README.md

@@ -1,3 +1,6 @@
-# Nomos Request For Comments(RFC)
-
-Nomos is building secure, flexible, and scalable infrastructure for developers creating applications for the network state.
+# Nomos RFCs
+
+Nomos is building a secure, flexible, and
+scalable infrastructure for developers creating applications for the network state.
+To learn more about Nomos current protocols under discussion,
+head over to [Nomos Specs](https://github.com/logos-co/nomos-specs).

status/24/curation.md

@@ -1,83 +1,108 @@
----
-slug: 24
-title: 24/STATUS-CURATION
-name: Status Community Directory Curation Voting using Waku v2
-status: draft
-tags: waku-application
-description: A voting protocol for SNT holders to submit votes to a smart contract. Voting is immutable, which helps avoid sabotage from malicious peers.
-editor: Szymon Szlachtowicz <szymon.s@ethworks.io>
----
-
-## Abstract
-This specification is a voting protocol for peers to submit votes to a smart contract. Voting is immutable, 
-this will help avoid sabotage from malicious peers.
-
-## Motivation
-
-In open p2p protocol there is an issue with voting off-chain as there is much room for malicious peers to only include votes that support their case when submitting votes to chain.
-
-Proposed solution is to aggregate votes over waku and allow users to submit votes to smart contract that aren't already submitted.
-
-### Smart contract
-
-Voting should be finalized on chain so that the finished vote is immutable.
-Because of that, smart contract needs to be deployed.
-When votes are submitted smart contract has to verify what votes are properly signed and that sender has correct amount of SNT.
-When Vote is verified the amount of SNT voted on specific topic by specific sender is saved on chain.
-
-### Double voting
-
-Smart contract should also keep a list of all signatures so that no one can send the same vote twice.
-Another possibility is to allow each sender to only vote once.
-
-### Initializing Vote
-
-When someone wants to initialize vote he has to send a transaction to smart contract that will create a new voting session.
-When initializing a user has to specify type of vote (Addition, Deletion), amount of his initial SNT to submit and public key of community under vote.
-Smart contract will return a ID which is identifier of voting session.
-Also there will be function on Smart Contract that when given community public key it will return voting session ID or undefined if community isn't under vote.
-
-## Voting
-
-### Sending votes
-
-Sending votes is simple every peer is able to send a message to Waku topic specific to given application: 
-```
-/status-community-directory-curation-vote/1/{voting-session-id}/json
-```
-
-vote object that is sent over waku should contain information about: 
-
-```ts
-type Vote = {
-    sender: string // address of the sender
-    vote: string // vote sent eg. 'yes' 'no'
-    sntAmount: BigNumber //number of snt cast on vote
-    sign: string // cryptographic signature of a transaction (signed fields: sender,vote,sntAmount,nonce,sessionID)
-    nonce: number // number of votes cast from this address on current vote (only if we allow multiple votes from the same sender)
-    sessionID: number // ID of voting session
-}
-```
-
-### Aggregating votes
-
-Every peer that is opening specific voting session will listen to votes sent over p2p network, and aggregate them for a single transaction to chain.
-
-### Submitting to chain
-
-Every peer that has aggregated at least one vote will be able to send them to smart contract.
-When someone votes he will aggregate his own vote and will be able to immediately send it.
-
-Peer doesn't need to vote to be able to submit the votes to the chain.
-
-Smart contract needs to verify that all votes are valid (eg. all senders had enough SNT, all votes are correctly signed) and that votes aren't duplicated on smart contract.
-
-### Finalizing 
-
-Once the vote deadline has expired, the smart contract will not accept votes anymore.
-Also directory will be updated according to vote results (community added to directory, removed etc.)
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 24
+title: 24/STATUS-CURATION
+name: Status Community Directory Curation Voting using Waku v2
+status: draft
+tags: waku-application
+description: A voting protocol for SNT holders to submit votes to a smart contract. Voting is immutable, which helps avoid sabotage from malicious peers.
+editor: Szymon Szlachtowicz <szymon.s@ethworks.io>
+---
+
+## Abstract
+
+This specification is a voting protocol for peers to submit votes to a smart contract.
+Voting is immutable,
+this will help avoid sabotage from malicious peers.
+
+## Motivation
+
+In open p2p protocol there is an issue with voting off-chain
+as there is much room for malicious peers to only include votes that support
+their case when submitting votes to chain.
+
+Proposed solution is to aggregate votes over waku and
+allow users to submit votes to smart contract that aren't already submitted.
+
+### Smart contract
+
+Voting should be finalized on chain so that the finished vote is immutable.
+Because of that, smart contract needs to be deployed.
+When votes are submitted
+smart contract has to verify what votes are properly signed and
+that sender has correct amount of SNT.
+When Vote is verified
+the amount of SNT voted on specific topic by specific sender is saved on chain.
+
+### Double voting
+
+Smart contract should also keep a list of all signatures so
+that no one can send the same vote twice.
+Another possibility is to allow each sender to only vote once.
+
+### Initializing Vote
+
+When someone wants to initialize vote
+he has to send a transaction to smart contract that will create a new voting session.
+When initializing a user has to specify type of vote (Addition, Deletion),
+amount of his initial SNT to submit and public key of community under vote.
+Smart contract will return a ID which is identifier of voting session.
+Also there will be function on Smart Contract that
+when given community public key it will return voting session ID or
+undefined if community isn't under vote.
+
+## Voting
+
+### Sending votes
+
+Sending votes is simple every peer is able to send a message to Waku topic
+specific to given application:
+
+```json
+
+/status-community-directory-curation-vote/1/{voting-session-id}/json
+
+```
+
+vote object that is sent over waku should contain information about:
+
+```ts
+type Vote = {
+    sender: string // address of the sender
+    vote: string // vote sent eg. 'yes' 'no'
+    sntAmount: BigNumber //number of snt cast on vote
+    sign: string // cryptographic signature of a transaction (signed fields: sender,vote,sntAmount,nonce,sessionID)
+    nonce: number // number of votes cast from this address on current vote 
+    // (only if we allow multiple votes from the same sender)
+    sessionID: number // ID of voting session
+}
+```
+
+### Aggregating votes
+
+Every peer that is opening specific voting session
+will listen to votes sent over p2p network, and
+aggregate them for a single transaction to chain.
+
+### Submitting to chain
+
+Every peer that has aggregated at least one vote
+will be able to send them to smart contract.
+When someone votes he will aggregate his own vote and
+will be able to immediately send it.
+
+Peer doesn't need to vote to be able to submit the votes to the chain.
+
+Smart contract needs to verify that all votes are valid
+(eg. all senders had enough SNT, all votes are correctly signed) and
+that votes aren't duplicated on smart contract.
+
+### Finalizing
+
+Once the vote deadline has expired, the smart contract will not accept votes anymore.
+Also directory will be updated according to vote results
+(community added to directory, removed etc.)
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).

status/28/featuring.md

@@ -1,57 +1,67 @@
----
-slug: 28
-title: 28/STATUS-FEATURING
-name: Status community featuring using waku v2
-status: draft
-tags: waku-application
-description: To gain new members, current SNT holders can vote to feature an active Status community to the larger Status audience.
-editor: Szymon Szlachtowicz <szymon.s@ethworks.io>
----
-
-## Abstract
-This specification describes a voting method to feature different active Status Communities.
-
-## Overview
-
-When there is a active community that is seeking new members, current users of community should be able to feature their community so that it will be accessible to larger audience.
-Status community curation DApp should provide such a tool.
-
-Rules of featuring:
-    - Given community can't be featured twice in a row.
-    - Only one vote per user per community (single user can vote on multiple communities)
-    - Voting will be done off-chain
-    - If community hasn't been featured votes for given community are still valid for the next 4 weeks
-
-Since voting for featuring is similar to polling solutions proposed in this spec could be also used for different applications.
-
-### Voting
-
-Voting for featuring will be done through waku v2. 
-
-Payload of waku message will be :
-```ts
-type FeatureVote = {
-    voter: string // address of a voter
-    sntAmount: BigNumber // amount of snt voted on featuring
-    communityPK: string // public key of community
-    timestamp: number // timestamp of message, must match timestamp of wakuMessage
-    sign: string // cryptographic signature of a transaction (signed fields: voterAddress,sntAmount,communityPK,timestamp)
-}
-```
-
-timestamp is necessary so that votes can't be reused after 4 week period
-
-### Counting Votes
-
-Votes will be counted by the DApp itself.
-DApp will aggregate all the votes in the last 4 weeks and calculate which communities should be displayed in the Featured tab of DApp.
-
-Rules of counting:
-    - When multiple votes from the same address on the same community are encountered only the vote with highest timestamp is considered valid.
-    - If a community has been featured in a previous week it can't be featured in current week.
-    - In a current week top 5 (or 10) communities with highest amount of SNT votes up to previous Sunday 23:59:59 UTC are considered featured.
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 28
+title: 28/STATUS-FEATURING
+name: Status community featuring using waku v2
+status: draft
+tags: waku-application
+description: To gain new members, current SNT holders can vote to feature an active Status community to the larger Status audience.
+editor: Szymon Szlachtowicz <szymon.s@ethworks.io>
+---
+
+## Abstract
+
+This specification describes a voting method to feature different active Status Communities.
+
+## Overview
+
+When there is a active community that is seeking new members,
+current users of community should be able to feature their community so
+that it will be accessible to larger audience.
+Status community curation DApp should provide such a tool.
+
+Rules of featuring:
+    - Given community can't be featured twice in a row.
+    - Only one vote per user per community (single user can vote on multiple communities)
+    - Voting will be done off-chain
+    - If community hasn't been featured
+    votes for given community are still valid for the next 4 weeks
+
+Since voting for featuring is similar to polling solutions proposed
+in this spec could be also used for different applications.
+
+### Voting
+
+Voting for featuring will be done through waku v2.
+
+Payload of waku message will be :
+
+```ts
+type FeatureVote = {
+    voter: string // address of a voter
+    sntAmount: BigNumber // amount of snt voted on featuring
+    communityPK: string // public key of community
+    timestamp: number // timestamp of message, must match timestamp of wakuMessage
+    sign: string // cryptographic signature of a transaction (signed fields: voterAddress,sntAmount,communityPK,timestamp)
+}
+```
+
+timestamp is necessary so that votes can't be reused after 4 week period
+
+### Counting Votes
+
+Votes will be counted by the DApp itself.
+DApp will aggregate all the votes in the last 4 weeks and
+calculate which communities should be displayed in the Featured tab of DApp.
+
+Rules of counting:
+    - When multiple votes from the same address on the same community are encountered
+    only the vote with highest timestamp is considered valid.
+    - If a community has been featured in a previous week
+    it can't be featured in current week.
+    - In a current week top 5 (or 10) communities with highest amount of SNT votes
+    up to previous Sunday 23:59:59 UTC are considered featured.
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).

status/55/1to1-chat.md

@@ -1,221 +1,273 @@
----
-slug: 55
-title: 55/STATUS-1TO1-CHAT
-name: Status 1-to-1 Chat
-status: draft
-category: Standards Track
-tags: waku-application
-description: A chat protocol to send public and private messages to a single recipient by the Status app.
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-- Andrea Piana <andreap@status.im>
-- Pedro Pombeiro <pedro@status.im>
-- Corey Petty <corey@status.im>
-- Oskar Thorén <oskarth@titanproxy.com>
-- Dean Eigenmann <dean@status.im>
----
-
-## Abstract
-
-This specification describes how the Status 1-to-1 chat protocol is implemented on top of the Waku v2 protocol. 
-This protocol can be used to send messages to a single recipient.
-
-## Terminology
-
-- **Participant**: A participant is a user that is able to send and receive messages.
-- **1-to-1 chat**: A chat between two participants.
-- **Public chat**: A chat where any participant can join and read messages.
-- **Private chat**: A chat where only invited participants can join and read messages.
-- **Group chat**: A chat where multiple select participants can join and read messages.
-- **Group admin**: A participant that is able to add/remove participants from a group chat.
-
-## Background
-
-This document describes how 2 peers communicate with each other to send messages in a 1-to-1 chat, with privacy and authenticity guarantees.
-
-## Specification
-
-### Overview
-
-This protocol MAY use any key-exchange mechanism previously discussed -
-
-1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) 
-2. [WAKU2-NOISE](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)
-
-This protocol can provide end-to-end encryption to give peers a strong degree of privacy and security. 
-Public chat messages are publicly readable by anyone since there's no permission model for who is participating in a public chat.
-
-## Flow
-
-### Negotiation of a 1:1 chat
-
-There are two phases in the initial negotiation of a 1:1 chat:
-1. **Identity verification** (e.g., face-to-face contact exchange through QR code, Identicon matching).
-A QR code serves two purposes simultaneously - identity verification and initial key material retrieval;
-1. **Asynchronous initial key exchange**
-
-For more information on account generation and trust establishment, see [65/ACCOUNT-ADDRESS](../65/account-address.md)
-
-### Post Negotiation
-
-After the peers have shared their public key material, a 1:1 chat can be established using the methods described in the key-exchange protocols mentioned above.
-
-### Session management
-
-The 1:1 chat is made robust by having sessions between peers.
-It is handled by the key-exchange protocol used. For example,
-
-1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md), the session management is described in [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
-
-2. [WAKU2-NOISE](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md), the session management is described in [WAKU2-NOISE-SESSIONS](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise-sessions/noise-sessions.md)
-
-## Negotiation of a 1:1 chat amongst multiple participants (group chat)
-
-A small, private group chat can be constructed by having multiple participants negotiate a 1:1 chat amongst each other.
-Each participant MUST maintain a session with all other participants in the group chat.
-This allows for a group chat to be created with a small number of participants.
-
-However, this method does not scale as the number of participants increases, for the following reasons -
-1. The number of messages sent over the network increases with the number of participants.
-2. Handling the X3DH key exchange for each participant is computationally expensive.
-
-The above issues are addressed in [56/STATUS-COMMUNITIES](../56/communities.md), with other trade-offs.
-
-### Flow
-
-The following flow describes how a group chat is created and maintained.
-
-#### Membership Update Flow
-
-Membership updates have the following wire format:
-
-```protobuf
-message MembershipUpdateMessage {
-  // The chat id of the private group chat
-  // derived in the following way:
-  // chat_id = hex(chat_creator_public_key) + "-" + random_uuid
-  // This chat_id MUST be validated by all participants
-  string chat_id = 1;
-  // A list of events for this group chat, first 65 bytes are the signature, then is a 
-  // protobuf encoded MembershipUpdateEvent
-  repeated bytes events = 2;
-  oneof chat_entity {
-    // An optional chat message
-    ChatMessage message = 3;
-    // An optional reaction to a message
-    EmojiReaction emoji_reaction = 4; 
-  }
-}
-```
-
-Note that in `events`, the first element is the signature, and all other elements after are encoded `MembershipUpdateEvent`'s.
-
-where `MembershipUpdateEvent` is defined as follows:
-
-```protobuf
-message MembershipUpdateEvent {
-  // Lamport timestamp of the event
-  uint64 clock = 1;
-  // Optional list of public keys of the targets of the action
-  repeated string members = 2;
-  // Name of the chat for the CHAT_CREATED/NAME_CHANGED event types
-  string name = 3;
-  // The type of the event
-  EventType type = 4;
-  // Color of the chat for the CHAT_CREATED/COLOR_CHANGED event types
-  string color = 5;
-  // Chat image
-  bytes image = 6;
-
-  enum EventType {
-    UNKNOWN = 0;
-    CHAT_CREATED = 1; // See [CHAT_CREATED](#chat-created)
-    NAME_CHANGED = 2; // See [NAME_CHANGED](#name-changed)
-    MEMBERS_ADDED = 3; // See [MEMBERS_ADDED](#members-added)
-    MEMBER_JOINED = 4; // See [MEMBER_JOINED](#member-joined)
-    MEMBER_REMOVED = 5; // See [MEMBER_REMOVED](#member-removed)
-    ADMINS_ADDED = 6; // See [ADMINS_ADDED](#admins-added)
-    ADMIN_REMOVED = 7; // See [ADMIN_REMOVED](#admin-removed)
-    COLOR_CHANGED = 8; // See [COLOR_CHANGED](#color-changed)
-    IMAGE_CHANGED = 9; // See [IMAGE_CHANGED](#image-changed)
-  }
-}
-```
-<!-- Note: I don't like defining wire formats which are out of the scope of the rfc this way. Should explore alternatives -->
-Note that the definitions for `ChatMessage` and `EmojiReaction` can be found in [chat_message.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/chat_message.proto#L1) and [emoji_reaction.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/emoji_reaction.proto).
-
-##### Chat Created
-
-When creating a group chat, this is the first event that MUST be sent. 
-Any event with a clock value lower than this MUST be discarded. 
-Upon receiving this event a client MUST validate the `chat_id` provided with the update and create a chat with identified by `chat_id`.
-
-By default, the creator of the group chat is the only group admin.
-
-##### Name Changed
-
-To change the name of the group chat, group admins MUST use a `NAME_CHANGED` event.
-Upon receiving this event a client MUST validate the `chat_id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. 
-If the event is valid the chat name SHOULD be changed according to the provided message.
-
-##### Members Added
-
-To add members to the chat, group admins MUST use a `MEMBERS_ADDED` event. 
-Upon receiving this event a participant MUST validate the `chat_id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. 
-If the event is valid, a participant MUST update the list of members of the chat who have not joined, adding the members received. 
-
-##### Member Joined
-
-To signal the intent to start receiving messages from a given chat, new participants MUST use a `MEMBER_JOINED` event.
-Upon receiving this event a participant MUST validate the `chat_id` provided with the updates. 
-If the event is valid a participant MUST add the new participant to the list of participants stored locally. 
-Any message sent to the group chat MUST now include the new participant.
-
-##### Member Removed
-
-There are two ways in which a member MAY be removed from a group chat:
-- A member MAY leave the chat by sending a `MEMBER_REMOVED` event, with the `members` field containing their own public key.
-- An admin MAY remove a member by sending a `MEMBER_REMOVED` event, with the `members` field containing the public key of the member to be removed.
-
-Each participant MUST validate the `chat_id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
-If the event is valid, a participant MUST update the local list of members accordingly.
-
-##### Admins Added
-
-To promote participants to group admin, group admins MUST use an `ADMINS_ADDED` event.
-Upon receiving this event, a participant MUST validate the `chat_id` provided with the updates, MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. 
-If the event is valid, a participant MUST update the list of admins of the chat accordingly.
-
-##### Admin Removed
-
-Group admins MUST NOT be able to remove other group admins.
-An admin MAY remove themselves by sending an `ADMIN_REMOVED` event, with the `members` field containing their own public key.
-Each participant MUST validate the `chat_id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
-If the event is valid, a participant MUST update the list of admins of the chat accordingly.
-
-##### Color Changed
-
-To change the text color of the group chat name, group admins MUST use a `COLOR_CHANGED` event.
-
-##### Image Changed
-
-To change the display image of the group chat, group admins MUST use an `IMAGE_CHANGED` event.
-
-## Security Considerations
-
-1. Inherits the security considerations of the key-exchange mechanism used, e.g., [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) or [WAKU2-NOISE](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
-2. [35/WAKU2-NOISE](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)
-3. [65/STATUS-ACCOUNT](../65/account-address.md)
-4. [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
-5. [37/WAKU2-NOISE-SESSIONS](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise-sessions/noise-sessions.md)
-6. [56/STATUS-COMMUNITIES](../56/communities.md)
-7. [chat_message.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/chat_message.proto#L1)
-8. [emoji_reaction.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/emoji_reaction.proto)
+---
+slug: 55
+title: 55/STATUS-1TO1-CHAT
+name: Status 1-to-1 Chat
+status: draft
+category: Standards Track
+tags: waku-application
+description: A chat protocol to send public and private messages to a single recipient by the Status app.
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Andrea Piana <andreap@status.im>
+- Pedro Pombeiro <pedro@status.im>
+- Corey Petty <corey@status.im>
+- Oskar Thorén <oskarth@titanproxy.com>
+- Dean Eigenmann <dean@status.im>
+---
+
+## Abstract
+
+This specification describes how the Status 1-to-1 chat protocol is implemented
+on top of the Waku v2 protocol.
+This protocol can be used to send messages to a single recipient.
+
+## Terminology
+
+- **Participant**: A participant is a user that is able to send and receive messages.
+- **1-to-1 chat**: A chat between two participants.
+- **Public chat**: A chat where any participant can join and read messages.
+- **Private chat**: A chat where only invited participants can join and read messages.
+- **Group chat**: A chat where multiple select participants can join and read messages.
+- **Group admin**: A participant that is able to
+add/remove participants from a group chat.
+
+## Background
+
+This document describes how 2 peers communicate with each other
+to send messages in a 1-to-1 chat, with privacy and authenticity guarantees.
+
+## Specification
+
+### Overview
+
+This protocol MAY use any key-exchange mechanism previously discussed -
+
+1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
+2. [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md)
+
+This protocol can provide end-to-end encryption
+to give peers a strong degree of privacy and security.
+Public chat messages are publicly readable by anyone since
+there's no permission model for who is participating in a public chat.
+
+## Chat Flow
+
+### Negotiation of a 1:1 chat
+
+There are two phases in the initial negotiation of a 1:1 chat:
+
+1. **Identity verification**
+(e.g., face-to-face contact exchange through QR code, Identicon matching).
+A QR code serves two purposes simultaneously -
+identity verification and initial key material retrieval;
+1. **Asynchronous initial key exchange**
+
+For more information on account generation and trust establishment, see [65/ACCOUNT-ADDRESS](../65/account-address.md)
+
+### Post Negotiation
+
+After the peers have shared their public key material,
+a 1:1 chat can be established using the methods described in the
+key-exchange protocols mentioned above.
+
+### Session management
+
+The 1:1 chat is made robust by having sessions between peers.
+It is handled by the key-exchange protocol used. For example,
+
+1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md),
+the session management is described in [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
+
+2. [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md),
+the session management is described in [WAKU2-NOISE-SESSIONS](https://github.com/waku-org/specs/blob/master/standards/application/noise-sessions.md)
+
+## Negotiation of a 1:1 chat amongst multiple participants (group chat)
+
+A small, private group chat can be constructed by having multiple participants
+negotiate a 1:1 chat amongst each other.
+Each participant MUST
+maintain a session with all other participants in the group chat.
+This allows for a group chat to be created with a small number of participants.
+
+However, this method does not scale as the number of participants increases,
+for the following reasons -
+
+1. The number of messages sent over the network increases with the number of participants.
+2. Handling the X3DH key exchange for each participant is computationally expensive.
+
+The above issues are addressed in [56/STATUS-COMMUNITIES](../56/communities.md),
+with other trade-offs.
+
+### Flow
+
+The following flow describes how a group chat is created and maintained.
+
+#### Membership Update Flow
+
+Membership updates have the following wire format:
+
+```protobuf
+message MembershipUpdateMessage {
+  // The chat id of the private group chat
+  // derived in the following way:
+  // chat_id = hex(chat_creator_public_key) + "-" + random_uuid
+  // This chat_id MUST be validated by all participants
+  string chat_id = 1;
+  // A list of events for this group chat, first 65 bytes are the signature, 
+  then is a 
+  // protobuf encoded MembershipUpdateEvent
+  repeated bytes events = 2;
+  oneof chat_entity {
+    // An optional chat message
+    ChatMessage message = 3;
+    // An optional reaction to a message
+    EmojiReaction emoji_reaction = 4; 
+  }
+}
+```
+
+Note that in `events`, the first element is the signature, and
+all other elements after are encoded `MembershipUpdateEvent`'s.
+
+where `MembershipUpdateEvent` is defined as follows:
+
+```protobuf
+message MembershipUpdateEvent {
+  // Lamport timestamp of the event
+  uint64 clock = 1;
+  // Optional list of public keys of the targets of the action
+  repeated string members = 2;
+  // Name of the chat for the CHAT_CREATED/NAME_CHANGED event types
+  string name = 3;
+  // The type of the event
+  EventType type = 4;
+  // Color of the chat for the CHAT_CREATED/COLOR_CHANGED event types
+  string color = 5;
+  // Chat image
+  bytes image = 6;
+
+  enum EventType {
+    UNKNOWN = 0;
+    CHAT_CREATED = 1; // See [CHAT_CREATED](#chat-created)
+    NAME_CHANGED = 2; // See [NAME_CHANGED](#name-changed)
+    MEMBERS_ADDED = 3; // See [MEMBERS_ADDED](#members-added)
+    MEMBER_JOINED = 4; // See [MEMBER_JOINED](#member-joined)
+    MEMBER_REMOVED = 5; // See [MEMBER_REMOVED](#member-removed)
+    ADMINS_ADDED = 6; // See [ADMINS_ADDED](#admins-added)
+    ADMIN_REMOVED = 7; // See [ADMIN_REMOVED](#admin-removed)
+    COLOR_CHANGED = 8; // See [COLOR_CHANGED](#color-changed)
+    IMAGE_CHANGED = 9; // See [IMAGE_CHANGED](#image-changed)
+  }
+}
+```
+<!-- Note: 
+I don't like defining wire formats which are out of the scope of the rfc this way. 
+Should explore alternatives -->
+Note that the definitions for `ChatMessage` and
+`EmojiReaction` can be found in
+[chat_message.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/chat_message.proto#L1)
+and [emoji_reaction.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/emoji_reaction.proto).
+
+##### Chat Created
+
+When creating a group chat, this is the first event that MUST be sent.
+Any event with a clock value lower than this MUST be discarded.
+Upon receiving this event a client MUST validate the `chat_id`
+provided with the update and
+create a chat with identified by `chat_id`.
+
+By default, the creator of the group chat is the only group admin.
+
+##### Name Changed
+
+To change the name of the group chat, group admins MUST use a `NAME_CHANGED` event.
+Upon receiving this event,
+a client MUST validate the `chat_id` provided with the updates and
+MUST ensure the author of the event is an admin of the chat,
+otherwise the event MUST be ignored.
+If the event is valid the chat name SHOULD be changed according to the provided message.
+
+##### Members Added
+
+To add members to the chat, group admins MUST use a `MEMBERS_ADDED` event.
+Upon receiving this event,
+a participant MUST validate the `chat_id` provided with the updates and
+MUST ensure the author of the event is an admin of the chat,
+otherwise the event MUST be ignored.
+If the event is valid,
+a participant MUST update the list of members of the chat who have not joined,
+adding the members received.
+
+##### Member Joined
+
+To signal the intent to start receiving messages from a given chat,
+new participants MUST use a `MEMBER_JOINED` event.
+Upon receiving this event,
+a participant MUST validate the `chat_id` provided with the updates.
+If the event is valid a participant,
+a participant MUST add the new participant to the list of participants stored locally.
+Any message sent to the group chat MUST now include the new participant.
+
+##### Member Removed
+
+There are two ways in which a member MAY be removed from a group chat:
+
+- A member MAY leave the chat by sending a `MEMBER_REMOVED` event,
+with the `members` field containing their own public key.
+- An admin MAY remove a member by sending a `MEMBER_REMOVED` event,
+with the `members` field containing the public key of the member to be removed.
+
+Each participant MUST validate the `chat_id` provided with the updates and
+MUST ensure the author of the event is an admin of the chat,
+otherwise the event MUST be ignored.
+If the event is valid, a participant MUST update the local list of members accordingly.
+
+##### Admins Added
+
+To promote participants to group admin, group admins MUST use an `ADMINS_ADDED` event.
+Upon receiving this event,
+a participant MUST validate the `chat_id` provided with the updates,
+MUST ensure the author of the event is an admin of the chat,
+otherwise the event MUST be ignored.
+If the event is valid,
+a participant MUST update the list of admins of the chat accordingly.
+
+##### Admin Removed
+
+Group admins MUST NOT be able to remove other group admins.
+An admin MAY remove themselves by sending an `ADMIN_REMOVED` event,
+with the `members` field containing their own public key.
+Each participant MUST validate the `chat_id` provided with the updates and
+MUST ensure the author of the event is an admin of the chat,
+otherwise the event MUST be ignored.
+If the event is valid, a participant MUST update the list of admins of the chat accordingly.
+
+##### Color Changed
+
+To change the text color of the group chat name,
+group admins MUST use a `COLOR_CHANGED` event.
+
+##### Image Changed
+
+To change the display image of the group chat,
+group admins MUST use an `IMAGE_CHANGED` event.
+
+## Security Considerations
+
+1. Inherits the security considerations of the key-exchange mechanism used,
+e.g., [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) or [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md)
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
+2. [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md)
+3. [65/STATUS-ACCOUNT](../65/account-address.md)
+4. [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
+5. [WAKU2-NOISE-SESSIONS](https://github.com/waku-org/specs/blob/master/standards/application/noise-sessions.md)
+6. [56/STATUS-COMMUNITIES](../56/communities.md)
+7. [chat_message.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/chat_message.proto#L1)
+8. [emoji_reaction.proto](https://github.com/status-im/status-go/blob/5fd9e93e9c298ed087e6716d857a3951dbfb3c1e/protocol/protobuf/emoji_reaction.proto)

status/56/communities.md

@@ -1,459 +1,526 @@
----
-slug: 56
-title: 56/STATUS-COMMUNITIES
-name: Status Communities that run over Waku v2
-status: draft
-category: Standards Track
-tags: waku-application
-description: Status Communities allow multiple users to communicate in a discussion space. This is a key feature of the Status application.
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-- Andrea Piana <andreap@status.im>
----
-
-## Abstract
-
-This document describes the design of Status Communities over Waku v2, allowing for multiple users to communicate in a discussion space. 
-This is a key feature for the Status messaging app. 
-
-## Background and Motivation
-
-The purpose of Status communities, as specified in this document, is allowing for large group chats.
-Communities can have further substructure, e.g. specific channels.
-
-Smaller group chats, on the other hand, are out of scope for this document and can be built over [55/STATUS-1TO1-CHAT](../55/1to1-chat.md). 
-We refer to these smaller group chats simply as "group chats", to differentiate them from Communities.
-
-For group chats based on [55/STATUS-1TO1-CHAT](../55/1to1-chat.md), the key exchange mechanism MUST be X3DH, as described in [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md).
-
-However, this method does not scale as the number of participants increases, for the following reasons -
-1. The number of messages sent over the network increases with the number of participants.
-2. Handling the X3DH key exchange for each participant is computationally expensive.
-
-Having multicast channels reduces the overhead of a group chat based on 1:1 chat.
-Additionally, if all the participants of the group chat have a shared key, then the number of messages sent over the network is reduced to one per message.
-
-## Terminology
-
-- **Community**: A group of peers that can communicate with each other.
-- **Member**: A peer that is part of a community.
-- **Admin**: A member that has administrative privileges. Used interchangeably with "owner".
-- **Channel**: A designated subtopic for a community. Used interchangeably with "chat".
-
-## Design Requirements
-
-Due to the nature of communities, the following requirements are necessary for the design of communities  -
-
-1. The creator of the Community is the owner of the Community.
-2. The Community owner is trusted.
-3. The Community owner can add or remove members from the Community.
-This extends to banning and kicking members.
-4. The Community owner can add, edit and remove channels.
-5. Community members can send/receive messages to the channels which they have access to.
-6. Communities may be encrypted (private) or unencrypted (public).
-7. A Community is uniquely identified by a public key.
-8. The public key of the Community is shared out of band.
-9. The metadata of the Community can be found by listening on a content topic derived from the public key of the Community.
-10. Community members run their own Waku nodes, with the configuration described in [Waku-Protocols](#waku-protocols).
-Light nodes solely implementing [19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md) may not be able to run their own Waku node with the configuration described.
-
-## Design
-
-### Cryptographic Primitives
-
-The following cryptographic primitives are used in the design -
-
-- X3DH
-- Single Ratchet 
-    - The single ratchet is used to encrypt the messages sent to the Community.
-    - The single ratchet is re-keyed when a member is added/removed from the Community.
-
-## Wire format
-
-<!--   
-The wire format is described first to give an overview of the protocol.
-It is referenced in the flow of community creation and community management.
-More or less an intersection of https://github.com/status-im/specs/blob/403b5ce316a270565023fc6a1f8dec138819f4b0/docs/raw/organisation-channels.md and https://github.com/status-im/status-go/blob/6072bd17ab1e5d9fc42cf844fcb8ad18aa07760c/protocol/protobuf/communities.proto,
-
--->
-
-```protobuf
-syntax = "proto3";
-
-message IdentityImage {
-  // payload is a context based payload for the profile image data,
-  // context is determined by the `source_type`
-  bytes payload = 1;
-  // source_type signals the image payload source
-  SourceType source_type = 2;
-  // image_type signals the image type and method of parsing the payload
-  ImageType image_type = 3;
-  // encryption_keys is a list of encrypted keys that can be used to decrypt an encrypted payload
-  repeated bytes encryption_keys = 4;
-  // encrypted signals the encryption state of the payload, default is false.
-  bool encrypted = 5;
-  // SourceType are the predefined types of image source allowed
-  enum SourceType {
-    UNKNOWN_SOURCE_TYPE = 0;
-
-    // RAW_PAYLOAD image byte data
-    RAW_PAYLOAD = 1;
-
-    // ENS_AVATAR uses the ENS record's resolver get-text-data.avatar data
-    // The `payload` field will be ignored if ENS_AVATAR is selected
-    // The application will read and parse the ENS avatar data as image payload data, URLs will be ignored
-    // The parent `ChatMessageIdentity` must have a valid `ens_name` set
-    ENS_AVATAR = 2;
-  }
-}
-
-// SocialLinks represents social link associated with given chat identity (personal/community)
-message SocialLink {
-  // Type of the social link
-  string text = 1;
-  // URL of the social link
-  string url = 2;
-}
-// ChatIdentity represents identity of a community/chat
-message ChatIdentity {
-  // Lamport timestamp of the message
-  uint64 clock = 1;
-  // ens_name is the valid ENS name associated with the chat key
-  string ens_name = 2;
-  // images is a string indexed mapping of images associated with an identity
-  map<string, IdentityImage> images = 3;
-  // display name is the user set identity
-  string display_name = 4;
-  // description is the user set description
-  string description = 5;
-  string color = 6;
-  string emoji = 7;
-  repeated SocialLink social_links = 8;
-  // first known message timestamp in seconds (valid only for community chats for now)
-  // 0 - unknown
-  // 1 - no messages
-  uint32 first_message_timestamp = 9;
-}
-
-message Grant {
-  // Community ID (The public key of the community)
-  bytes community_id = 1;
-  // The member ID (The public key of the member)
-  bytes member_id = 2;
-  // The chat for which the grant is given
-  string chat_id = 3;
-  // The Lamport timestamp of the grant
-  uint64 clock = 4;
-}
-
-message CommunityMember {
-  // The roles a community member MAY have
-  enum Roles {
-    UNKNOWN_ROLE = 0;
-    ROLE_ALL = 1;
-    ROLE_MANAGE_USERS = 2;
-    ROLE_MODERATE_CONTENT = 3;
-  }
-  repeated Roles roles = 1;
-}
-
-message CommunityPermissions {
-  // The type of access a community MAY have
-  enum Access {
-    UNKNOWN_ACCESS = 0;
-    NO_MEMBERSHIP = 1;
-    INVITATION_ONLY = 2;
-    ON_REQUEST = 3;
-  }
-
-  // If the community should be available only to ens users
-  bool ens_only = 1;
-  // If the community is private
-  bool private = 2;
-  Access access = 3;
-}
-
-message CommunityAdminSettings {
-  // If the Community admin may pin messages
-  bool pin_message_all_members_enabled = 1;
-}
-
-message CommunityChat {
-  // A map of members in the community to their roles in a chat
-  map<string,CommunityMember> members = 1;
-  // The permissions of the chat
-  CommunityPermissions permissions = 2;
-  // The metadata of the chat
-  ChatIdentity identity = 3;
-  // The category of the chat
-  string category_id = 4;
-  // The position of chat in the display
-  int32 position = 5;
-}
-
-message CommunityCategory {
-  // The category id 
-  string category_id = 1;
-  // The name of the category
-  string name = 2;
-  // The position of the category in the display
-  int32 position = 3;
-}
-
-message CommunityInvitation {
-  // Encrypted/unencrypted community description
-  bytes community_description = 1;
-  // The grant offered by the community
-  bytes grant = 2;
-  // The chat id requested to join
-  string chat_id = 3;
-  // The public key of the community
-  bytes public_key = 4;
-}
-
-message CommunityRequestToJoin {
-  // The Lamport timestamp of the request  
-  uint64 clock = 1;
-  // The ENS name of the requester
-  string ens_name = 2;
-  // The chat id requested to join
-  string chat_id = 3;
-  // The public key of the community
-  bytes community_id = 4;
-  // The display name of the requester
-  string display_name = 5;
-}
-
-message CommunityCancelRequestToJoin {
-  // The Lamport timestamp of the request
-  uint64 clock = 1;
-  // The ENS name of the requester
-  string ens_name = 2;
-  // The chat id requested to join
-  string chat_id = 3;
-  // The public key of the community
-  bytes community_id = 4;
-  // The display name of the requester
-  string display_name = 5;
-  // Magnet uri for community history protocol
-  string magnet_uri = 6;
-}
-
-message CommunityRequestToJoinResponse {
-  // The Lamport timestamp of the request
-  uint64 clock = 1;
-  // The community description
-  CommunityDescription community = 2;
-  // If the request was accepted
-  bool accepted = 3;
-  // The grant offered by the community
-  bytes grant = 4;
-  // The community public key
-  bytes community_id = 5;
-}
-
-message CommunityRequestToLeave {
-  // The Lamport timestamp of the request
-  uint64 clock = 1;
-  // The community public key
-  bytes community_id = 2;
-}
-
-message CommunityDescription {
-  // The Lamport timestamp of the message
-  uint64 clock = 1;
-  // A mapping of members in the community to their roles
-  map<string,CommunityMember> members = 2;
-  // The permissions of the Community
-  CommunityPermissions permissions = 3;
-  // The metadata of the Community
-  ChatIdentity identity = 5;
-  // A mapping of chats to their details
-  map<string,CommunityChat> chats = 6;
-  // A list of banned members
-  repeated string ban_list = 7;
-  // A mapping of categories to their details
-  map<string,CommunityCategory> categories = 8;
-  // The admin settings of the Community
-  CommunityAdminSettings admin_settings = 10;
-  // If the community is encrypted
-  bool encrypted = 13;
-  // The list of tags
-  repeated string tags = 14;
-}
-```
-
-Note: The usage of the clock is described in the [Clock](#clock) section.
-
-### Content topic usage
-
-"Content topic" refers to the field in [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md/#message-attributes), further elaborated in [10/WAKU2](../../waku/standards/core/10/waku2.md/#overview-of-protocol-interaction).
-
-#### Advertising a Community
-
-The content topic that the community is advertised on MUST be derived from the public key of the community.
-The content topic MUST be the first four bytes of the keccak-256 hash of the compressed (33 bytes) public key of the community encoded into a hex string.
-
-```
-hash = hex(keccak256(encodeToHex(compressedPublicKey)))
-
-topicLen = 4
-if len(hash) < topicLen {
-    topicLen = len(hash)
-}
-
-var topic [4]byte
-for i = 0; i < topicLen; i++ {
-    topic[i] = hash[i]
-}
-
-contentTopic = "/waku/1/0x" + topic + "/rfc26"
-```
-
-#### Community channels/chats
-
-The unique identifier for a community channel/chat is the chat id.
-<!-- Don't enforce any constraints on the unique id generation -->
-The content topic that Community channels/chats use MUST be the hex-encoded keccak-256 hash of the public key of the community concatenated with the chat id.
-
-```
-hash = hex(keccak256(encodeToHex(compressedPublicKey + chatId)))
-
-topicLen = 4
-if len(hash) < topicLen {
-    topicLen = len(hash)
-}
-var topic [4]byte
-for i = 0; i < topicLen; i++ {
-    topic[i] = hash[i]
-}
-
-contentTopic = "/waku/1/0x" + topic + "/rfc26"
-```
-
-
-#### Community event messages
-
-Requests to leave, join, kick and ban, as well as key exchange messages, MUST be sent to the content topic derived from the public key of the community.
-The content topic MUST be the hex-encoded keccak-256 hash of the public key of the community.
-
-```
-hash = hex(keccak256(encodeToHex(publicKey)))
-
-topicLen = 4
-if len(hash) < topicLen {
-    topicLen = len(hash)
-}
-var topic [4]byte
-for i = 0; i < topicLen; i++ {
-    topic[i] = hash[i]
-}
-
-contentTopic = "/waku/1/0x" + topic + "/rfc26"
-``` 
-
-### Community Management
-
-The flows for Community management are as described below.
-
-#### Community Creation Flow
-
-1. The Community owner generates a public/private key pair.
-2. The Community owner configures the Community metadata, according to the wire format "CommunityDescription".
-3. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. 
-the Community metadata SHOULD be encrypted with the public key of the Community. <!-- TODO: Verify this-->
-The Community metadata MAY be sent during fixed intervals, to ensure that the Community metadata is available to members.
-The Community metadata SHOULD be sent every time the Community metadata is updated.
-4. The Community owner MAY advertise the Community out of band, by sharing the public key of the Community on other mediums of communication.
-
-#### Community Join Flow (peer requests to join a Community)
-
-1. A peer and the Community owner establish a 1:1 chat as described in [55/STATUS-1TO1-CHAT](../55/1to1-chat.md).
-2. The peer requests to join a Community by sending a "CommunityRequestToJoin" message to the Community.
-At this point, the peer MAY send a "CommunityCancelRequestToJoin" message to cancel the request.
-3. The Community owner MAY accept or reject the request.
-4. If the request is accepted, the Community owner sends a "CommunityRequestToJoinResponse" message to the peer.
-5. The Community owner then adds the member to the Community metadata, and publishes the updated Community metadata.
-
-#### Community Join Flow (peer is invited to join a Community)
-
-1. The Community owner and peer establish a 1:1 chat as described in [55/STATUS-1TO1-CHAT](../55/1to1-chat.md).
-2. The peer is invited to join a Community by the Community owner, by sending a "CommunityInvitation" message.
-3. The peer decrypts the "CommunityInvitation" message, and verifies the signature.
-4. The peer requests to join a Community by sending a "CommunityRequestToJoin" message to the Community.
-5. The Community owner MAY accept or reject the request.
-6. If the request is accepted, the Community owner sends a "CommunityRequestToJoinResponse" message to the peer.
-7. The Community owner then adds the member to the Community metadata, and publishes the updated Community metadata.
-
-#### Community Leave Flow
-
-1. A member requests to leave a Community by sending a "CommunityRequestToLeave" message to the Community.
-2. The Community owner MAY accept or reject the request.
-3. If the request is accepted, the Community owner removes the member from the Community metadata, and publishes the updated Community metadata.
-
-#### Community Ban Flow
-
-1. The Community owner adds a member to the ban list, revokes their grants, and publishes the updated Community metadata.
-2. If the Community is Private, Re-keying is performed between the members of the Community, to ensure that the banned member is unable to decrypt any messages.
-
-### Waku Protocols 
-
-The following Waku protocols SHOULD be used to implement Status Communities -
-
-1. [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md) - To send and receive messages
-2. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) - To encrypt and decrypt messages
-3. [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md) - To handle session keys
-4. [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md) - To wrap community messages in a Waku message
-5. [13/WAKU2-STORE](../../waku/standards/core/13/store.md) - To store and retrieve messages for offline devices
-
-
-The following Waku protocols MAY be used to implement Status Communities -
-
-1. [12/WAKU2-FILTER](../../waku/standards/core/12/filter.md) - Content filtering for resource restricted devices
-2. [19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md) - Allows Light clients to participate in the network
-
-### Backups
-
-The member MAY back up their local settings, by encrypting it with their public key, and sending it to a given content topic.
-The member MAY then rely on this backup to restore their local settings, in case of a data loss.
-This feature relies on [13/WAKU2-STORE](../../waku/standards/core/13/store.md) for storing and retrieving messages.
-
-### Clock
-
-The clock used in the wire format refers to the Lamport timestamp of the message.
-The Lamport timestamp is a logical clock that is used to determine the order of events in a distributed system.
-This allows ordering of messages in an asynchronous network where messages may be received out of order.
-
-## Security Considerations
-
-1. The Community owner is a single point of failure. If the Community owner is compromised, the Community is compromised.
-
-2. Follows the same security considerations as the [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) protocol.
-
-## Future work
-
-1. To scale and optimize the Community management, the Community metadata should be stored on a decentralized storage system, and only the references to the Community metadata should be broadcasted. The following document describes this method in more detail - [Optimizing the `CommunityDescription` dissemination](https://hackmd.io/rD1OfIbJQieDe3GQdyCRTw)
-
-2. Token gating for communities
-
-3. Sharding the content topic used for [#Community Event Messages](#community-event-messages), since members of the community don't need to receive all the control messages.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [55/STATUS-1TO1-CHAT](../55/1to1-chat.md)
-- [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
-- [19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md)
-- [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md)
-- [10/WAKU2](../../waku/standards/core/10/waku2.md)
-- [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)
-- [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
-- [13/WAKU2-STORE](../../waku/standards/core/13/store.md)
-- [12/WAKU2-FILTER](../../waku/standards/core/12/filter.md)
-
-### informative
-- [community.go](https://github.com/status-im/status-go/blob/6072bd17ab1e5d9fc42cf844fcb8ad18aa07760c/protocol/communities/community.go)
-- [organisation-channels.md](https://github.com/status-im/specs/blob/403b5ce316a270565023fc6a1f8dec138819f4b0/docs/raw/organisation-channels.md)
+---
+slug: 56
+title: 56/STATUS-COMMUNITIES
+name: Status Communities that run over Waku v2
+status: draft
+category: Standards Track
+tags: waku-application
+description: Status Communities allow multiple users to communicate in a discussion space. This is a key feature of the Status application.
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Andrea Piana <andreap@status.im>
+---
+
+## Abstract
+
+This document describes the design of Status Communities over Waku v2,
+allowing for multiple users to communicate in a discussion space.
+This is a key feature for the Status messaging app.
+
+## Background and Motivation
+
+The purpose of Status communities, as specified in this document,
+is allowing for large group chats.
+Communities can have further substructure, e.g. specific channels.
+
+Smaller group chats, on the other hand,
+are out of scope for this document and
+can be built over [55/STATUS-1TO1-CHAT](../55/1to1-chat.md).
+We refer to these smaller group chats simply as "group chats",
+to differentiate them from Communities.
+
+For group chats based on [55/STATUS-1TO1-CHAT](../55/1to1-chat.md),
+the key exchange mechanism MUST be X3DH,
+as described in [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md).
+
+However, this method does not scale as the number of participants increases,
+for the following reasons -
+
+1. The number of messages sent over the network increases with the number of participants.
+2. Handling the X3DH key exchange for each participant is computationally expensive.
+
+Having multicast channels reduces the overhead of a group chat based on 1:1 chat.
+Additionally, if all the participants of the group chat have a shared key,
+then the number of messages sent over the network is reduced to one per message.
+
+## Terminology
+
+- **Community**: A group of peers that can communicate with each other.
+- **Member**: A peer that is part of a community.
+- **Admin**: A member that has administrative privileges.
+Used interchangeably with "owner".
+- **Channel**: A designated subtopic for a community. Used interchangeably with "chat".
+
+## Design Requirements
+
+Due to the nature of communities,
+the following requirements are necessary for the design of communities  -
+
+1. The creator of the Community is the owner of the Community.
+2. The Community owner is trusted.
+3. The Community owner can add or remove members from the Community.
+This extends to banning and kicking members.
+4. The Community owner can add, edit and remove channels.
+5. Community members can send/receive messages
+to the channels which they have access to.
+6. Communities may be encrypted (private) or unencrypted (public).
+7. A Community is uniquely identified by a public key.
+8. The public key of the Community is shared out of band.
+9. The metadata of the Community can be found by listening on a content topic
+derived from the public key of the Community.
+10. Community members run their own Waku nodes,
+with the configuration described in [Waku-Protocols](#waku-protocols).
+Light nodes solely implementing
+[19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md)
+may not be able to run their own Waku node with the configuration described.
+
+## Design
+
+### Cryptographic Primitives
+
+The following cryptographic primitives are used in the design -
+
+- X3DH
+- Single Ratchet
+  - The single ratchet is used to encrypt the messages sent to the Community.
+  - The single ratchet is re-keyed when a member is added/removed from the Community.
+
+## Wire format
+
+<!--   
+The wire format is described first to give an overview of the protocol.
+It is referenced in the flow of community creation and community management.
+More or less an intersection of https://github.com/status-im/specs/blob/403b5ce316a270565023fc6a1f8dec138819f4b0/docs/raw/organisation-channels.md 
+and https://github.com/status-im/status-go/blob/6072bd17ab1e5d9fc42cf844fcb8ad18aa07760c/protocol/protobuf/communities.proto,
+
+-->
+
+```protobuf
+syntax = "proto3";
+
+message IdentityImage {
+  // payload is a context based payload for the profile image data,
+  // context is determined by the `source_type`
+  bytes payload = 1;
+  // source_type signals the image payload source
+  SourceType source_type = 2;
+  // image_type signals the image type and method of parsing the payload
+  ImageType image_type = 3;
+  // encryption_keys is a list of encrypted keys that can be used to decrypt an 
+  // encrypted payload
+  repeated bytes encryption_keys = 4;
+  // encrypted signals the encryption state of the payload, default is false.
+  bool encrypted = 5;
+  // SourceType are the predefined types of image source allowed
+  enum SourceType {
+    UNKNOWN_SOURCE_TYPE = 0;
+
+    // RAW_PAYLOAD image byte data
+    RAW_PAYLOAD = 1;
+
+    // ENS_AVATAR uses the ENS record's resolver get-text-data.avatar data
+    // The `payload` field will be ignored if ENS_AVATAR is selected
+    // The application will read and 
+    // parse the ENS avatar data as image payload data, URLs will be ignored
+    // The parent `ChatMessageIdentity` must have a valid `ens_name` set
+    ENS_AVATAR = 2;
+  }
+}
+
+// SocialLinks represents social link associated with given chat identity (personal/community)
+message SocialLink {
+  // Type of the social link
+  string text = 1;
+  // URL of the social link
+  string url = 2;
+}
+// ChatIdentity represents identity of a community/chat
+message ChatIdentity {
+  // Lamport timestamp of the message
+  uint64 clock = 1;
+  // ens_name is the valid ENS name associated with the chat key
+  string ens_name = 2;
+  // images is a string indexed mapping of images associated with an identity
+  map<string, IdentityImage> images = 3;
+  // display name is the user set identity
+  string display_name = 4;
+  // description is the user set description
+  string description = 5;
+  string color = 6;
+  string emoji = 7;
+  repeated SocialLink social_links = 8;
+  // first known message timestamp in seconds 
+  // (valid only for community chats for now)
+  // 0 - unknown
+  // 1 - no messages
+  uint32 first_message_timestamp = 9;
+}
+
+message Grant {
+  // Community ID (The public key of the community)
+  bytes community_id = 1;
+  // The member ID (The public key of the member)
+  bytes member_id = 2;
+  // The chat for which the grant is given
+  string chat_id = 3;
+  // The Lamport timestamp of the grant
+  uint64 clock = 4;
+}
+
+message CommunityMember {
+  // The roles a community member MAY have
+  enum Roles {
+    UNKNOWN_ROLE = 0;
+    ROLE_ALL = 1;
+    ROLE_MANAGE_USERS = 2;
+    ROLE_MODERATE_CONTENT = 3;
+  }
+  repeated Roles roles = 1;
+}
+
+message CommunityPermissions {
+  // The type of access a community MAY have
+  enum Access {
+    UNKNOWN_ACCESS = 0;
+    NO_MEMBERSHIP = 1;
+    INVITATION_ONLY = 2;
+    ON_REQUEST = 3;
+  }
+
+  // If the community should be available only to ens users
+  bool ens_only = 1;
+  // If the community is private
+  bool private = 2;
+  Access access = 3;
+}
+
+message CommunityAdminSettings {
+  // If the Community admin may pin messages
+  bool pin_message_all_members_enabled = 1;
+}
+
+message CommunityChat {
+  // A map of members in the community to their roles in a chat
+  map<string,CommunityMember> members = 1;
+  // The permissions of the chat
+  CommunityPermissions permissions = 2;
+  // The metadata of the chat
+  ChatIdentity identity = 3;
+  // The category of the chat
+  string category_id = 4;
+  // The position of chat in the display
+  int32 position = 5;
+}
+
+message CommunityCategory {
+  // The category id 
+  string category_id = 1;
+  // The name of the category
+  string name = 2;
+  // The position of the category in the display
+  int32 position = 3;
+}
+
+message CommunityInvitation {
+  // Encrypted/unencrypted community description
+  bytes community_description = 1;
+  // The grant offered by the community
+  bytes grant = 2;
+  // The chat id requested to join
+  string chat_id = 3;
+  // The public key of the community
+  bytes public_key = 4;
+}
+
+message CommunityRequestToJoin {
+  // The Lamport timestamp of the request  
+  uint64 clock = 1;
+  // The ENS name of the requester
+  string ens_name = 2;
+  // The chat id requested to join
+  string chat_id = 3;
+  // The public key of the community
+  bytes community_id = 4;
+  // The display name of the requester
+  string display_name = 5;
+}
+
+message CommunityCancelRequestToJoin {
+  // The Lamport timestamp of the request
+  uint64 clock = 1;
+  // The ENS name of the requester
+  string ens_name = 2;
+  // The chat id requested to join
+  string chat_id = 3;
+  // The public key of the community
+  bytes community_id = 4;
+  // The display name of the requester
+  string display_name = 5;
+  // Magnet uri for community history protocol
+  string magnet_uri = 6;
+}
+
+message CommunityRequestToJoinResponse {
+  // The Lamport timestamp of the request
+  uint64 clock = 1;
+  // The community description
+  CommunityDescription community = 2;
+  // If the request was accepted
+  bool accepted = 3;
+  // The grant offered by the community
+  bytes grant = 4;
+  // The community public key
+  bytes community_id = 5;
+}
+
+message CommunityRequestToLeave {
+  // The Lamport timestamp of the request
+  uint64 clock = 1;
+  // The community public key
+  bytes community_id = 2;
+}
+
+message CommunityDescription {
+  // The Lamport timestamp of the message
+  uint64 clock = 1;
+  // A mapping of members in the community to their roles
+  map<string,CommunityMember> members = 2;
+  // The permissions of the Community
+  CommunityPermissions permissions = 3;
+  // The metadata of the Community
+  ChatIdentity identity = 5;
+  // A mapping of chats to their details
+  map<string,CommunityChat> chats = 6;
+  // A list of banned members
+  repeated string ban_list = 7;
+  // A mapping of categories to their details
+  map<string,CommunityCategory> categories = 8;
+  // The admin settings of the Community
+  CommunityAdminSettings admin_settings = 10;
+  // If the community is encrypted
+  bool encrypted = 13;
+  // The list of tags
+  repeated string tags = 14;
+}
+```
+
+Note: The usage of the clock is described in the [Clock](#clock) section.
+
+### Content topic usage
+
+"Content topic" refers to the field in [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md/#message-attributes),
+further elaborated in [10/WAKU2](../../waku/standards/core/10/waku2.md/#overview-of-protocol-interaction).
+
+#### Advertising a Community
+
+The content topic that the community is advertised on
+MUST be derived from the public key of the community.
+The content topic MUST be the first four bytes of the keccak-256 hash
+of the compressed (33 bytes) public key of the community encoded into a hex string.
+
+```js
+hash = hex(keccak256(encodeToHex(compressedPublicKey)))
+
+topicLen = 4
+if len(hash) < topicLen {
+    topicLen = len(hash)
+}
+
+var topic [4]byte
+for i = 0; i < topicLen; i++ {
+    topic[i] = hash[i]
+}
+
+contentTopic = "/waku/1/0x" + topic + "/rfc26"
+```
+
+#### Community channels/chats
+
+The unique identifier for a community channel/chat is the chat id.
+<!-- Don't enforce any constraints on the unique id generation -->
+The content topic, that Community channels/chats uses,
+MUST be the hex-encoded keccak-256 hash of the public key of the community
+concatenated with the chat id.
+
+```js
+hash = hex(keccak256(encodeToHex(compressedPublicKey + chatId)))
+
+topicLen = 4
+if len(hash) < topicLen {
+    topicLen = len(hash)
+}
+var topic [4]byte
+for i = 0; i < topicLen; i++ {
+    topic[i] = hash[i]
+}
+
+contentTopic = "/waku/1/0x" + topic + "/rfc26"
+```
+
+#### Community event messages
+
+Requests to leave, join, kick and ban, as well as key exchange messages,
+MUST be sent to the content topic derived from the public key of the community.
+The content topic
+MUST be the hex-encoded keccak-256 hash of the public key of the community.
+
+```js
+hash = hex(keccak256(encodeToHex(publicKey)))
+
+topicLen = 4
+if len(hash) < topicLen {
+    topicLen = len(hash)
+}
+var topic [4]byte
+for i = 0; i < topicLen; i++ {
+    topic[i] = hash[i]
+}
+
+contentTopic = "/waku/1/0x" + topic + "/rfc26"
+```
+
+### Community Management
+
+The flows for Community management are as described below.
+
+#### Community Creation Flow
+
+1. The Community owner generates a public/private key pair.
+2. The Community owner configures the Community metadata,
+according to the wire format "CommunityDescription".
+3. The Community owner publishes the Community metadata on a content topic
+derived from the public key of the Community.
+the Community metadata SHOULD be encrypted with the public key of the Community.
+<!-- TODO: Verify this-->
+The Community metadata MAY be sent during fixed intervals,
+to ensure that the Community metadata is available to members.
+The Community metadata SHOULD be sent every time the Community metadata is updated.
+4. The Community owner MAY advertise the Community out of band,
+by sharing the public key of the Community on other mediums of communication.
+
+#### Community Join Flow (peer requests to join a Community)
+
+1. A peer and the Community owner establish a 1:1 chat as described in [55/STATUS-1TO1-CHAT](../55/1to1-chat.md).
+2. The peer requests to join a Community by sending a
+"CommunityRequestToJoin" message to the Community.
+At this point, the peer MAY send a
+"CommunityCancelRequestToJoin" message to cancel the request.
+3. The Community owner MAY accept or reject the request.
+4. If the request is accepted,
+the Community owner sends a "CommunityRequestToJoinResponse" message to the peer.
+5. The Community owner then adds the member to the Community metadata, and
+publishes the updated Community metadata.
+
+#### Community Join Flow (peer is invited to join a Community)
+
+1. The Community owner and peer establish a 1:1 chat as described in [55/STATUS-1TO1-CHAT](../55/1to1-chat.md).
+2. The peer is invited to join a Community by the Community owner,
+by sending a "CommunityInvitation" message.
+3. The peer decrypts the "CommunityInvitation" message, and verifies the signature.
+4. The peer requests to join a Community by sending a
+"CommunityRequestToJoin" message to the Community.
+5. The Community owner MAY accept or reject the request.
+6. If the request is accepted,
+the Community owner sends a "CommunityRequestToJoinResponse" message to the peer.
+7. The Community owner then adds the member to the Community metadata, and
+publishes the updated Community metadata.
+
+#### Community Leave Flow
+
+1. A member requests to leave a Community by sending a
+"CommunityRequestToLeave" message to the Community.
+2. The Community owner MAY accept or reject the request.
+3. If the request is accepted,
+the Community owner removes the member from the Community metadata,
+and publishes the updated Community metadata.
+
+#### Community Ban Flow
+
+1. The Community owner adds a member to the ban list, revokes their grants,
+and publishes the updated Community metadata.
+2. If the Community is Private,
+Re-keying is performed between the members of the Community,
+to ensure that the banned member is unable to decrypt any messages.
+
+### Waku Protocols
+
+The following Waku protocols SHOULD be used to implement Status Communities -
+
+1. [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md) -
+To send and receive messages
+2. [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) -
+To encrypt and decrypt messages
+3. [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)-
+To handle session keys
+4. [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md) -
+To wrap community messages in a Waku message
+5. [13/WAKU2-STORE](../../waku/standards/core/13/store.md) -
+To store and retrieve messages for offline devices
+
+The following Waku protocols MAY be used to implement Status Communities -
+
+1. [12/WAKU2-FILTER](../../waku/standards/core/12/filter.md) -
+Content filtering for resource restricted devices
+2. [19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md) -
+Allows Light clients to participate in the network
+
+### Backups
+
+The member MAY back up their local settings,
+by encrypting it with their public key, and
+sending it to a given content topic.
+The member MAY then rely on this backup to restore their local settings,
+in case of a data loss.
+This feature relies on
+[13/WAKU2-STORE](../../waku/standards/core/13/store.md)
+for storing and retrieving messages.
+
+### Clock
+
+The clock used in the wire format refers to the Lamport timestamp of the message.
+The Lamport timestamp is a logical clock that is used to determine the order of events
+in a distributed system.
+This allows ordering of messages in an asynchronous network
+where messages may be received out of order.
+
+## Security Considerations
+
+1. The Community owner is a single point of failure.
+If the Community owner is compromised, the Community is compromised.
+
+2. Follows the same security considerations as the
+[53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) protocol.
+
+## Future work
+
+1. To scale and optimize the Community management,
+the Community metadata should be stored on a decentralized storage system, and
+only the references to the Community metadata should be broadcasted.
+The following document describes this method in more detail -
+[Optimizing the `CommunityDescription` dissemination](https://hackmd.io/rD1OfIbJQieDe3GQdyCRTw)
+
+2. Token gating for communities
+
+3. Sharding the content topic used for [#Community Event Messages](#community-event-messages),
+since members of the community don't need to receive all the control messages.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [55/STATUS-1TO1-CHAT](../55/1to1-chat.md)
+- [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
+- [19/WAKU2-LIGHTPUSH](../../waku/standards/core/19/lightpush.md)
+- [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md)
+- [10/WAKU2](../../waku/standards/core/10/waku2.md)
+- [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)
+- [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
+- [13/WAKU2-STORE](../../waku/standards/core/13/store.md)
+- [12/WAKU2-FILTER](../../waku/standards/core/12/filter.md)
+
+### informative
+
+- [community.go](https://github.com/status-im/status-go/blob/6072bd17ab1e5d9fc42cf844fcb8ad18aa07760c/protocol/communities/community.go)
+- [organisation-channels.md](https://github.com/status-im/specs/blob/403b5ce316a270565023fc6a1f8dec138819f4b0/docs/raw/organisation-channels.md)

status/61/community-history-service.md

@@ -1,392 +1,596 @@
----
-slug: 61
-title: 61/STATUS-Community-History-Service
-name: Status Community History Service
-status: draft
-category: Standards Track
-description: Explains how new members of a Status community can request historical messages from archive nodes.
-editor: r4bbit <r4bbit@status.im>
-contributors:
-  - Sanaz Taheri <sanaz@status.im>
-  - John Lea <john@status.im>
----
-
-## Abstract
-
-Messages are stored permanently by store nodes ([13/WAKU2-STORE](../../waku/standards/core/13/store.md)) for up to a certain configurable period of time, limited by the overall storage provided by a store node.
-Messages older than that period are no longer provided by store nodes, making it impossible for other nodes to request historical messages that go beyond that time range. 
-This raises issues in the case of Status communities, where recently joined members of a community are not able to request complete message histories of the community channels.
-
-This specification describes how **Control Nodes** (which are specific nodes in Status communities) archive historical message data of their communities, beyond the time range limit provided by Store Nodes using the [BitTorrent](https://bittorrent.org) protocol.
-It also describes how the archives are distributed to community members via the Status network, so they can fetch them and gain access to a complete message history.
-
-## Terminology
-
-The following terminology is used throughout this specification. Notice that some actors listed here are nodes that operate in Waku networks only, while others operate in the Status communities layer):
-
-| Name                 | References |
-| -------------------- | --- |
-| Waku node            | An Waku node ([10/WAKU2](../../waku/standards/core/10/waku2.md)) that implements [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)|
-| Store node           | A Waku node that implements [13/WAKU2-STORE](../../waku/standards/core/13/store.md) |
-| Waku network         | A group of Waku nodes forming a graph, connected via [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md) |
-| Status user          | An Status account that is used in a Status consumer product, such as Status Mobile or Status Desktop |
-| Status node          | A Status client run by a Status application |
-| Control node      | A Status node that owns the private key for a Status community |
-| Community member     | A Status user that is part of a Status community, not owning the private key of the community |
-| Community member node| A Status node with message archive capabilities enabled, run by a community member |
-| Live messages        | Waku messages received through the Waku network |
-| BitTorrent client    | A program implementing the [BitTorrent](https://bittorrent.org) protocol |
-| Torrent/Torrent file | A file containing metadata about data to be downloaded by BitTorrent clients |
-| Magnet link          | A link encoding the metadata provided by a torrent file ([Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme)) |
-
-## Requirements / Assumptions
-
-This specification has the following assumptions:
-
-- Store nodes([13/WAKU2-STORE](../../waku/standards/core/13/store.md)) are available 24/7, ensuring constant live message availability.
-- The storage time range limit is 30 days.
-- Store nodes have enough storage to persist historical messages for up to 30 days.
-- No store nodes have storage to persist historical messages older than 30 days.
-- All nodes are honest.
-- The network is reliable.
-
-Furthermore, it assumes that:
-
-- Control nodes have enough storage to persist historical messages older than 30 days.
-- Control nodes provide archives with historical messages **at least** every 30 days.
-- Control nodes receive all community messages.
-- Control nodes are honest.
-- Control nodes know at least one store node from which it can query historical messages.
-
-These assumptions are less than ideal and will be enhanced in future work. This [forum discussion](https://forum.vac.dev/t/status-communities-protocol-and-product-point-of-view/114) provides more details.
-
-## Overview
-
-The following is a high-level overview of the user flow and features this specification describes. For more detailed descriptions, read the dedicated sections in this specification.
-
-### Serving community history archives
-
-Control nodes go through the following (high level) process to provide community members with message histories:
-
-1. Community owner creates a Status community (previously known as [org channels](https://github.com/status-im/specs/pull/151)) which makes its node a Control node.
-2. Community owner enables message archive capabilities (on by default but can be turned off as well - see [UI feature spec](https://github.com/status-im/feature-specs/pull/36)).
-3. A special type of channel to exchange metadata about the archival data is created, this channel is not visible in the user interface.
-4. Community owner invites community members.
-5. Control node receives messages published in channels and stores them into a local database.
-6. After 7 days, the control node exports and compresses last 7 days worth of messages from database and bundles it together with a [message archive index](#waku-message-archive-index) into a torrent, from which it then creates a magnet link ([Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme), [Extensions for Peers to Send Metadata Files](https://www.bittorrent.org/beps/bep_0009.html)). 
-7. Control node sends the magnet link created in step 6 to community members via special channel created in step 3 through the Waku network.
-8. Every subsequent 7 days, steps 6 and 7 are repeated and the new message archive data is appended to the previously created message archive data.
-
-### Serving archives for missed messages
-
-If the control node goes offline (where "offline" means, the control node's main process is no longer running), it MUST go through the following process:
-
-1. Control node restarts
-2. Control node requests messages from store nodes for the missed time range for all channels in their community
-3. All missed messages are stored into control node's local message database
-4. If 7 or more days have elapsed since the last message history torrent was created, the control node will perform step 6 and 7 of [Serving community history archives](#serving-community-history-archives) for every 7 days worth of messages in the missed time range (e.g. if the node was offline for 30 days, it will create 4 message history archives)
-
-### Receiving community history archives
-
-Community member nodes go through the following (high level) process to fetch and restore community message histories:
-
-1. User joins community and becomes community member (see [org channels spec](../56/communities.md))
-2. By joining a community, member nodes automatically subscribe to special channel for message archive metadata exchange provided by the community
-3. Member node requests live message history (last 30 days) of all the community channels including the special channel from store nodes
-4. Member node receives Waku message ([14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md)) that contains the metadata magnet link from the special channel
-5. Member node extracts the magnet link from the Waku message and passes it to torrent client
-6. Member node downloads [message archive index](#message-history-archive-index) file and determines which message archives are not downloaded yet (all or some)
-7. Member node fetches missing message archive data via torrent
-8. Member node unpacks and decompresses message archive data to then hydrate its local database, deleting any messages for that community that the database previously stored in the same time range as covered by the message history archive
-
-## Storing live messages
-
-For archival data serving, the control node MUST store live messages as [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md).
-This is in addition to their database of application messages. 
-This is required to provide confidentiality, authenticity, and integrity of message data distributed via the BitTorrent layer, and later validated by community members when they unpack message history archives.
-
-Control nodes SHOULD remove those messages from their local databases once they are older than 30 days and after they have been turned into message archives and distributed to the BitTorrent network.
-
-### Exporting messages for bundling
-
-Control nodes export Waku messages from their local database for creating and bundling history archives using the following criteria:
-
-- Waku messages to be exported MUST have a `contentTopic` that match any of the topics of the community channels
-- Waku messages to be exported MUST have a `timestamp` that lies within a time range of 7 days
-
-The `timestamp` is determined by the context in which the control node attempts to create a message history archives as described below:
-
-1. The control node attempts to create an archive periodically for the past seven days (including the current day). In this case, the `timestamp` has to lie within those 7 days.
-2. The control node has been offline (control node's main process has stopped and needs restart) and attempts to create archives for all the live messages it has missed since it went offline. In this case, the `timestamp` has to lie within the day the latest message was received and the current day.
-
-Exported messages MUST be restored as [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md) for bundling. Waku messages that are older than 30 days and have been exported for bundling can be removed from the control node's database (control nodes still maintain a database of application messages).
-
-## Message history archives
-
-Message history archives are represented as `WakuMessageArchive` and created from Waku messages exported from the local database. 
-Message history archives are implemented using the following protocol buffer.
-
-### WakuMessageHistoryArchive
-
-The `from` field SHOULD contain a timestamp of the time range's lower bound.
-The type parallels the `timestamp` of [WakuMessage](../../waku/standards/core/14/message.md/#payloads).
-
-The `to` field SHOULD contain a timestamp of the time range's the higher bound.
-
-The `contentTopic` field MUST contain a list of all communiity channel topics.
-
-The `messages` field MUST contain all messages that belong into the archive given its `from`, `to` and `contentTopic` fields.
-
-The `padding` field MUST contain the amount of zero bytes needed so that the overall byte size of the protobuf encoded `WakuMessageArchive` is a multiple of the `pieceLength` used to divide the message archive data into pieces.
-This is needed for seamless encoding and decoding of archival data in interation with BitTorrent as explained in [creating message archive torrents](#creating-message-archive-torrents).
-
-```
-syntax = "proto3"
-
-message WakuMessageArchiveMetadata {
-  uint8 version = 1
-  uint64 from = 2
-  uint64 to = 3
-  repeated string contentTopic = 4
-}
-
-message WakuMessageArchive {
-  uint8 version = 1
-  WakuMessageArchiveMetadata metadata = 2
-  repeated WakuMessage messages = 3 // `WakuMessage` is provided by 14/WAKU2-MESSAGE
-  bytes padding = 4
-}
-```
-
-## Message history archive index
-
-Control nodes MUST provide message archives for the entire community history.
-The entirey history consists of a set of `WakuMessageArchive`'s where each archive contains a subset of historical `WakuMessage`s for a time range of seven days. 
-All the `WakuMessageArchive`s are concatenated into a single file as a byte string (see [Ensuring reproducible data pieces](#ensuring-reproducible-data-pieces)).
-
-Control nodes MUST create a message history archive index (`WakuMessageArchiveIndex`) with metadata that allows receiving nodes to only fetch the message history archives they are interested in.
-
-### WakuMessageArchiveIndex
-
-A `WakuMessageArchiveIndex` is a map where the key is the KECCAK-256 hash of the `WakuMessageArchiveIndexMetadata` derived from a 7-day archive and the value is an instance of that `WakuMessageArchiveIndexMetadata` corresponding to that archive.
-
-The `offset` field MUST contain the position at which the message history archive starts in the byte string of the total message archive data. This MUST be the sum of the length of all previously created message archives in bytes (see [Creating message archive torrents](#creating-message-archive-torrents)).
-
-```
-syntax = "proto3"
-
-message WakuMessageArchiveIndexMetadata {
-  uint8 version = 1
-  WakuMessageArchiveMetadata metadata = 2
-  uint64 offset = 3
-  uint64 num_pieces = 4
-}
-
-message WakuMessageArchiveIndex {
-  map<string, WakuMessageArchiveIndexMetadata> archives = 1
-}
-```
-
-The control node MUST update the `WakuMessageArchiveIndex` every time it creates one or more `WakuMessageArchive`s and bundle it into a new torrent.
-For every created `WakuMessageArchive`, there MUST be a `WakuMessageArchiveIndexMetadata` entry in the `archives` field `WakuMessageArchiveIndex`.
-
-# Creating message archive torrents
-
-Control nodes MUST create a torrent file ("torrent") containing metadata to all message history archives.
-To create a torrent file, and later serve the message archive data in the BitTorrent network, control nodes MUST store the necessary data in dedicated files on the file system.
-
-A torrent's source folder MUST contain the following two files:
-
-- `data` - Contains all protobuf encoded `WakuMessageArchive`'s (as bit strings) concatenated in ascending order based on their time
-- `index` - Contains the protobuf encoded `WakuMessageArchiveIndex`
-
-Control nodes SHOULD store these files in a dedicated folder that is identifiable via the community id.
-
-### Ensuring reproducible data pieces
-
-The control node MUST ensure that the byte string resulting from the protobuf encoded `data` is equal to the byte string `data` from the previously generated message archive torrent, plus the data of the latest 7 days worth of messages encoded as `WakuMessageArchive`.
-Therefore, the size of `data` grows every seven days as it's append only.
-
-The control nodes also MUST ensure that the byte size of every individual `WakuMessageArchive` encoded protobuf is a multiple of `pieceLength: ???` (**TODO**) using the `padding` field.
-If the protobuf encoded 'WakuMessageArchive` is not a multiple of `pieceLength`, its `padding` field MUST be filled with zero bytes and the `WakuMessageArchive` MUST be re-encoded until its size becomes multiple of `pieceLength`.
-
-This is necessary because the content of the `data` file will be split into pieces of `pieceLength` when the torrent file is created, and the SHA1 hash of every piece is then stored in the torrent file and later used by other nodes to request the data for each individual data piece.
-
-By fitting message archives into a multiple of `pieceLength` and ensuring they fill possible remaining space with zero bytes, control nodes prevent the **next** message archive to occupy that remaining space of the last piece, which will result in a different SHA1 hash for that piece.
-
-#### **Example: Without padding**
-
-Let `WakuMessageArchive` "A1" be of size 20 bytes:
-
-```
- 0 11 22 33 44 55 66 77 88 99
-10 11 12 13 14 15 16 17 18 19 
-```
-
-With a `pieceLength` of 10 bytes, A1 will fit into `20 / 10 = 2` pieces:
-
-```
- 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
-10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
-```
-
-#### **Example: With padding**
-
-Let `WakuMessageArchive` "A2" be of size 21 bytes:
-
-```
- 0 11 22 33 44 55 66 77 88 99
-10 11 12 13 14 15 16 17 18 19
-20
-```
-
-With a `pieceLength` of 10 bytes, A2 will fit into `21 / 10 = 2` pieces. The remainder will introduce a third piece:
-
-```
- 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
-10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
-20                            // piece[2] SHA1: 0x789
-```
-
-The next `WakuMessageArchive` "A3" will be appended ("#3") to the existing data and occupy the remaining space of the third data piece. The piece at index 2 will now produce a different SHA1 hash:
-
-```
- 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
-10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
-20 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[2] SHA1: 0xeef
-#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3]
-```
-
-By filling up the remaining space of the third piece with A2 using its `padding` field, it is guaranteed that its SHA1 will stay the same:
-
-```
- 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
-10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
-20  0  0  0  0  0  0  0  0  0 // piece[2] SHA1: 0x999
-#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3]
-#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[4]
-```
-
-### Seeding message history archives
-
-The control node MUST seed the [generated torrent](#creating-message-archive-torrents) until a new `WakuMessageArchive` is created.
-
-The control node SHOULD NOT seed torrents for older message history archives. Only one torrent at a time should be seeded.
-
-### Creating magnet links
-
-Once a torrent file for all message archives is created, the control node MUST derive a magnet link following the [Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme) using the underlying BitTorrent protocol client.
-
-### Message archive distribution
-
-Message archives are available via the BitTorrent network as they are being [seeded by the control node](#seeding-message-history-archives).
-Other community member nodes will download the message archives from the BitTorrent network once they receive a magnet link that contains a message archive index.
-
-The control node MUST send magnet links containing message archives and the message archive index to a special community channel. 
-The topic of that special channel follows the following format:
-
-```
-/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}
-```
-
-All messages sent with this topic MUST be instances of `ApplicationMetadataMessage` ([62/STATUS-PAYLOAD](../62/payload.md)) with a `payload` of `CommunityMessageArchiveIndex`.
-
-Only the control node MAY post to the special channel. Other messages on this specified channel MUST be ignored by clients.
-Community members MUST NOT have permission to send messages to the special channel.
-However, community member nodes MUST subscribe to special channel to receive Waku messages containing magnet links for message archives.
-
-### Canonical message histories
-
-Only control nodes are allowed to distribute messages with magnet links via the special channel for magnet link exchange.
-Community members MUST NOT be allowed to post any messages to the special channel.
-
-Status nodes MUST ensure that any message that isn't signed by the control node in the special channel is ignored.
-
-Since the magnet links are created from the control node's database (and previously distributed archives), the message history provided by the control node becomes the canonical message history and single source of truth for the community.
-
-Community member nodes MUST replace messages in their local databases with the messages extracted from archives within the same time range.
-Messages that the control node didn't receive  MUST be removed and are no longer part of the message history of interest, even if it already existed in a community member node's database.
-
-## Fetching message history archives
-
-Generally, fetching message history archives is a three step process:
-
-1. Receive [message archive index](#message-history-archive-index) magnet link as described in [Message archive distribution], download `index` file from torrent, then determine which message archives to download
-2. Download individual archives
-
-Community member nodes subscribe to the special channel that control nodes publish magnet links for message history archives to. 
-There are two scenarios in which member nodes can receive such a magnet link message from the special channel:
-
-1. The member node receives it via live messages, by listening to the special channel 
-2. The member node requests messages for a time range of up to 30 days from store nodes (this is the case when a new community member joins a community)
-
-### Downloading message archives
-When member nodes receive a message with a `CommunityMessageHistoryArchive` ([62/STATUS-PAYLOAD](../62/payload.md)) from the aforementioned channnel, they MUST extract the `magnet_uri` and pass it to their underlying BitTorrent client so they can fetch the latest message history archive index, which is the `index` file of the torrent (see [Creating message archive torrents](#creating-message-archive-torrents)).
-
-Due to the nature of distributed systems, there's no guarantee that a received message is the "last" message. This is especially true when member nodes request historical messages from store nodes. 
-
-Therefore, member nodes MUST wait for 20 seconds after receiving the last `CommunityMessageArchive` before they start extracting the magnet link to fetch the latest archive index.
-
-Once a message history archive index is downloaded and parsed back into `WakuMessageArchiveIndex`, community member nodes use a local lookup table to determine which of the listed archives are missing using the KECCAK-256 hashes stored in the index.
-
-For this lookup to work, member nodes MUST store the KECCAK-256 hashes of the `WakuMessageArchiveIndexMetadata` provided by the `index` file for all of the message history archives that have been downlaoded in their local database.
-
-Given a `WakuMessageArchiveIndex`, member nodes can access individual `WakuMessageArchiveIndexMetadata` to download individual archives.
-
-Community member nodes MUST choose one of the following options:
-
-1. **Download all archives** - Request and download all data pieces for `data` provided by the torrent (this is the case for new community member nodes that haven't downloaded any archives yet)
-2. **Download only the latest archive** - Request and download all pieces starting at the `offset` of the latest `WakuMessageArchiveIndexMetadata` (this the case for any member node that already has downloaded all previous history and is now interested in only the latst archive)
-3. **Download specific archives** - Look into `from` and `to` fields of every `WakuMessageArchiveIndexMetadata` and determine the pieces for archives of a specific time range (can be the case for member nodes that have recently joined the network and are only interested in a subset of the complete history)
-
-### Storing historical messages
-
-When message archives are fetched, community member nodes MUST unwrap the resulting `WakuMessage` instances into `ApplicationMetadataMessage` instances and store them in their local database.
-Community member nodes SHOULD NOT store the wrapped `WakuMessage` messages.
-
-All message within the same time range MUST be replaced with the messages provided by the message history archive.
-
-Community members nodes MUST ignore the expiration state of each archive message.
-
-## Considerations
-
-The following are things to cosider when implementing this specification.
-
-## Control node honesty
-
-This spec assumes that all control nodes are honest and behave according to the spec. Meaning they don't inject their own messages into, or remove any messages from historic archives.
-
-## Bandwidth consumption
-
-Community member nodes will download the latest archive they've received from the archive index, which includes messages from the last seven days. Assuming that community members nodes were online for that time range, they have already downloaded that message data and will now download an archive that contains the same.
-
-This means there's a possibility member nodes will download the same data at least twice.
-
-## Multiple community owners
-
-It is possible for control nodes to export the private key of their owned community and pass it to other users so they become control nodes as well.
-This means, it's possible for multiple control nodes to exist.
-
-This might conflict with the assumption that the control node serves as a single source of thruth. Multiple control nodes can have different message histories.
-
-Not only will multiple control nodes multiply the amount of archive index messages being distributed to the network, they might also contain different sets of magnet links and their corresponding hashes.
-
-Even if just a single message is missing in one of the histories, the hashes presented in archive indices will look completely different, resulting in the community member node to download the corresponding archive (which might be identical to an archive that was already downloaded, except for that one message).
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-* [13/WAKU2-STORE](../../waku/standards/core/13/store.md)
-* [BitTorrent](https://bittorrent.org)
-* [10/WAKU2](../../waku/standards/core/10/waku2.md)
-* [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)
-* [Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme)
-* [forum discussion](https://forum.vac.dev/t/status-communities-protocol-and-product-point-of-view/114)
-* [org channels](https://github.com/status-im/specs/pull/151)
-* [UI feature spec](https://github.com/status-im/feature-specs/pull/36)
-* [Extensions for Peers to Send Metadata Files](https://www.bittorrent.org/beps/bep_0009.html)
-* [org channels spec](../56/communities.md)
-* [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md)
-* [62/STATUS-PAYLOAD](../62/payload.md)
+---
+slug: 61
+title: 61/STATUS-Community-History-Service
+name: Status Community History Service
+status: draft
+category: Standards Track
+description: Explains how new members of a Status community can request historical messages from archive nodes.
+editor: r4bbit <r4bbit@status.im>
+contributors:
+  - Sanaz Taheri <sanaz@status.im>
+  - John Lea <john@status.im>
+---
+
+## Abstract
+
+Messages are stored permanently by store nodes
+([13/WAKU2-STORE](../../waku/standards/core/13/store.md))
+for up to a certain configurable period of time,
+limited by the overall storage provided by a store node.
+Messages older than that period are no longer provided by store nodes,
+making it impossible for other nodes to request historical messages
+that go beyond that time range.
+This raises issues in the case of Status communities,
+where recently joined members of a community
+ are not able to request complete message histories of the community channels.
+
+This specification describes how **Control Nodes**
+(which are specific nodes in Status communities)
+archive historical message data of their communities,
+beyond the time range limit provided by Store Nodes using
+the [BitTorrent](https://bittorrent.org) protocol.
+It also describes how the archives are distributed to community members via
+the Status network,
+so they can fetch them and gain access to a complete message history.
+
+## Terminology
+
+The following terminology is used throughout this specification.
+Notice that some actors listed here are nodes that operate in Waku networks only,
+while others operate in the Status communities layer):
+
+| Name                 | References |
+| -------------------- | --- |
+| Waku node            | An Waku node ([10/WAKU2](../../waku/standards/core/10/waku2.md)) that implements [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)|
+| Store node           | A Waku node that implements [13/WAKU2-STORE](../../waku/standards/core/13/store.md) |
+| Waku network         | A group of Waku nodes forming a graph, connected via [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md) |
+| Status user          | An Status account that is used in a Status consumer product, such as Status Mobile or Status Desktop |
+| Status node          | A Status client run by a Status application |
+| Control node      | A Status node that owns the private key for a Status community |
+| Community member     | A Status user that is part of a Status community, not owning the private key of the community |
+| Community member node| A Status node with message archive capabilities enabled, run by a community member |
+| Live messages        | Waku messages received through the Waku network |
+| BitTorrent client    | A program implementing the [BitTorrent](https://bittorrent.org) protocol |
+| Torrent/Torrent file | A file containing metadata about data to be downloaded by BitTorrent clients |
+| Magnet link          | A link encoding the metadata provided by a torrent file ([Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme)) |
+
+## Requirements / Assumptions
+
+This specification has the following assumptions:
+
+- Store nodes,
+([13/WAKU2-STORE](../../waku/standards/core/13/store.md)),
+ are available 24/7 ensuring constant live message availability.
+- The storage time range limit is 30 days.
+- Store nodes have enough storage to persist historical messages for up to 30 days.
+- No store nodes have storage to persist historical messages older than 30 days.
+- All nodes are honest.
+- The network is reliable.
+
+Furthermore, it assumes that:
+
+- Control nodes have enough storage to persist historical messages
+older than 30 days.
+- Control nodes provide archives with historical messages **at least** every 30 days.
+- Control nodes receive all community messages.
+- Control nodes are honest.
+- Control nodes know at least one store node from which it can query historical messages.
+
+These assumptions are less than ideal and will be enhanced in future work.
+This [forum discussion](https://forum.vac.dev/t/status-communities-protocol-and-product-point-of-view/114)
+provides more details.
+
+## Overview
+
+The following is a high-level overview of the user flow and
+features this specification describes.
+ For more detailed descriptions, read the dedicated sections in this specification.
+
+### Serving community history archives
+
+Control nodes go through the following
+(high level) process to provide community members with message histories:
+
+1. Community owner creates a Status community
+(previously known as [org channels](https://github.com/status-im/specs/pull/151))
+which makes its node a Control node.
+2. Community owner enables message archive capabilities
+(on by default but can be turned off as well - see [UI feature spec](https://github.com/status-im/feature-specs/pull/36)).
+3. A special type of channel to exchange metadata about the archival data is created,
+this channel is not visible in the user interface.
+4. Community owner invites community members.
+5. Control node receives messages published in channels and
+stores them into a local database.
+6. After 7 days, the control node exports and
+compresses last 7 days worth of messages from database and
+bundles it together with a
+[message archive index](#wakumessagearchiveindex) into a torrent,
+from which it then creates a magnet link ([Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme),
+[Extensions for Peers to Send Metadata Files](https://www.bittorrent.org/beps/bep_0009.html)).
+7. Control node sends the magnet link created in step 6 to community members via
+special channel created in step 3 through the Waku network.
+8. Every subsequent 7 days,
+steps 6 and 7 are repeated and
+the new message archive data
+is appended to the previously created message archive data.
+
+### Serving archives for missed messages
+
+If the control node goes offline
+(where "offline" means, the control node's main process is no longer running),
+it MUST go through the following process:
+
+1. Control node restarts
+2. Control node requests messages from store nodes
+for the missed time range for all channels in their community
+3. All missed messages are stored into control node's local message database
+4. If 7 or more days have elapsed since the last message history torrent was created,
+the control node will perform step 6 and
+7 of [Serving community history archives](#serving-community-history-archives)
+for every 7 days worth of messages in the missed time range
+(e.g. if the node was offline for 30 days, it will create 4 message history archives)
+
+### Receiving community history archives
+
+Community member nodes go through the following (high level) process to fetch and
+restore community message histories:
+
+1. User joins community and becomes community member (see [org channels spec](../56/communities.md))
+2. By joining a community,
+member nodes automatically subscribe to special channel for
+message archive metadata exchange provided by the community
+3. Member node requests live message history
+(last 30 days) of all the community channels,
+including the special channel from store nodes
+4. Member node receives Waku message
+([14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md))
+that contains the metadata magnet link from the special channel
+5. Member node extracts the magnet link from the Waku message and
+passes it to torrent client
+6. Member node downloads
+[message archive index](#message-history-archive-index) file and
+determines which message archives are not downloaded yet (all or some)
+7. Member node fetches missing message archive data via torrent
+8. Member node unpacks and
+decompresses message archive data to then hydrate its local database,
+deleting any messages,
+for that community that the database previously stored in the same time range,
+as covered by the message history archive
+
+## Storing live messages
+
+For archival data serving, the control node MUST store live messages as [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md).
+This is in addition to their database of application messages.
+This is required to provide confidentiality, authenticity,
+and integrity of message data distributed via the BitTorrent layer, and
+later validated by community members when they unpack message history archives.
+
+Control nodes SHOULD remove those messages from their local databases
+once they are older than 30 days and
+after they have been turned into message archives and
+distributed to the BitTorrent network.
+
+### Exporting messages for bundling
+
+Control nodes export Waku messages from their local database for creating and
+bundling history archives using the following criteria:
+
+- Waku messages to be exported MUST have a `contentTopic`
+that match any of the topics of the community channels
+- Waku messages to be exported MUST have a `timestamp`
+that lies within a time range of 7 days
+
+The `timestamp` is determined by the context in which the control node attempts
+to create a message history archives as described below:
+
+1. The control node attempts to create an archive periodically
+for the past seven days (including the current day).
+In this case, the `timestamp` has to lie within those 7 days.
+2. The control node has been offline
+(control node's main process has stopped and needs restart) and
+attempts to create archives for all the live messages it has missed
+since it went offline.
+In this case,
+the `timestamp` has to lie within the day the latest message was received and
+the current day.
+
+Exported messages MUST be restored as
+[14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md) for bundling.
+Waku messages that are older than 30 days and
+have been exported for bundling can be removed from the control node's database
+(control nodes still maintain a database of application messages).
+
+## Message history archives
+
+Message history archives are represented as `WakuMessageArchive` and
+created from Waku messages exported from the local database.
+Message history archives are implemented using the following protocol buffer.
+
+### WakuMessageHistoryArchive
+
+The `from` field SHOULD contain a timestamp of the time range's lower bound.
+The type parallels the `timestamp` of [WakuMessage](../../waku/standards/core/14/message.md/#payloads).
+
+The `to` field SHOULD contain a timestamp of the time range's the higher bound.
+
+The `contentTopic` field MUST contain a list of all communiity channel topics.
+
+The `messages` field MUST contain all messages that belong into the archive
+given its `from`, `to` and `contentTopic` fields.
+
+The `padding` field MUST contain the amount of zero bytes needed so
+that the overall byte size of the protobuf encoded `WakuMessageArchive`
+is a multiple of the `pieceLength` used to divide the message archive data into pieces.
+This is needed for seamless encoding and
+decoding of archival data in interation with BitTorrent,
+as explained in [creating message archive torrents](#creating-message-archive-torrents).
+
+```protobuf
+syntax = "proto3"
+
+message WakuMessageArchiveMetadata {
+  uint8 version = 1
+  uint64 from = 2
+  uint64 to = 3
+  repeated string contentTopic = 4
+}
+
+message WakuMessageArchive {
+  uint8 version = 1
+  WakuMessageArchiveMetadata metadata = 2
+  repeated WakuMessage messages = 3 // `WakuMessage` is provided by 14/WAKU2-MESSAGE
+  bytes padding = 4
+}
+```
+
+## Message History Archive Index
+
+Control nodes MUST provide message archives for the entire community history.
+The entirey history consists of a set of `WakuMessageArchive`'s
+where each archive contains a subset of historical `WakuMessage`s
+for a time range of seven days.
+All the `WakuMessageArchive`s are concatenated into a single file as a byte string
+(see [Ensuring reproducible data pieces](#ensuring-reproducible-data-pieces)).
+
+Control nodes MUST create a message history archive index
+(`WakuMessageArchiveIndex`) with metadata that allows receiving nodes
+to only fetch the message history archives they are interested in.
+
+### WakuMessageArchiveIndex
+
+A `WakuMessageArchiveIndex` is a map where the key is the KECCAK-256 hash of
+the `WakuMessageArchiveIndexMetadata` derived from a 7-day archive and
+the value is an instance of that `WakuMessageArchiveIndexMetadata`
+corresponding to that archive.
+
+The `offset` field MUST contain the position at which the message history archive
+starts in the byte string of the total message archive data.
+This MUST be the sum of the length of all previously created message archives
+in bytes (see [Creating message archive torrents](#creating-message-archive-torrents)).
+
+```protobuf
+syntax = "proto3"
+
+message WakuMessageArchiveIndexMetadata {
+  uint8 version = 1
+  WakuMessageArchiveMetadata metadata = 2
+  uint64 offset = 3
+  uint64 num_pieces = 4
+}
+
+message WakuMessageArchiveIndex {
+  map<string, WakuMessageArchiveIndexMetadata> archives = 1
+}
+```
+
+The control node MUST update the `WakuMessageArchiveIndex`
+every time it creates one or
+more `WakuMessageArchive`s and bundle it into a new torrent.
+For every created `WakuMessageArchive`,
+there MUST be a `WakuMessageArchiveIndexMetadata` entry in the `archives` field `WakuMessageArchiveIndex`.
+
+## Creating message archive torrents
+
+Control nodes MUST create a torrent file ("torrent")
+containing metadata to all message history archives.
+To create a torrent file, and
+later serve the message archive data in the BitTorrent network,
+control nodes MUST store the necessary data in dedicated files on the file system.
+
+A torrent's source folder MUST contain the following two files:
+
+- `data` - Contains all protobuf encoded `WakuMessageArchive`'s (as bit strings)
+concatenated in ascending order based on their time
+- `index` - Contains the protobuf encoded `WakuMessageArchiveIndex`
+
+Control nodes SHOULD store these files in a dedicated folder that is identifiable,
+via the community id.
+
+### Ensuring reproducible data pieces
+
+The control node MUST ensure that the byte string resulting from
+the protobuf encoded `data` is equal to the byte string `data`
+from the previously generated message archive torrent,
+plus the data of the latest 7 days worth of messages encoded as `WakuMessageArchive`.
+Therefore, the size of `data` grows every seven days as it's append only.
+
+The control nodes also MUST ensure that the byte size of every individual `WakuMessageArchive`
+encoded protobuf is a multiple of `pieceLength: ???` (**TODO**)
+using the `padding` field.
+If the protobuf encoded `WakuMessageArchive` is not a multiple of `pieceLength`,
+its `padding` field MUST be filled with zero bytes and
+the `WakuMessageArchive` MUST be re-encoded until its size becomes multiple of `pieceLength`.
+
+This is necessary because the content of the `data` file
+will be split into pieces of `pieceLength` when the torrent file is created,
+and the SHA1 hash of every piece is then stored in the torrent file and
+later used by other nodes to request the data for each individual data piece.
+
+By fitting message archives into a multiple of `pieceLength` and
+ensuring they fill possible remaining space with zero bytes,
+control nodes prevent the **next** message archive to
+occupy that remaining space of the last piece,
+ which will result in a different SHA1 hash for that piece.
+
+#### **Example: Without padding**
+
+Let `WakuMessageArchive` "A1" be of size 20 bytes:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99
+10 11 12 13 14 15 16 17 18 19 
+```
+
+With a `pieceLength` of 10 bytes, A1 will fit into `20 / 10 = 2` pieces:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
+10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
+```
+
+#### **Example: With padding**
+
+Let `WakuMessageArchive` "A2" be of size 21 bytes:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99
+10 11 12 13 14 15 16 17 18 19
+20
+```
+
+With a `pieceLength` of 10 bytes, A2 will fit into `21 / 10 = 2` pieces.
+The remainder will introduce a third piece:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
+10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
+20                            // piece[2] SHA1: 0x789
+```
+
+The next `WakuMessageArchive` "A3" will be appended ("#3") to the existing data
+and occupy the remaining space of the third data piece.
+The piece at index 2 will now produce a different SHA1 hash:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
+10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
+20 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[2] SHA1: 0xeef
+#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3]
+```
+
+By filling up the remaining space of the third piece
+with A2 using its `padding` field,
+ it is guaranteed that its SHA1 will stay the same:
+
+```json
+ 0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123
+10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456
+20  0  0  0  0  0  0  0  0  0 // piece[2] SHA1: 0x999
+#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3]
+#3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[4]
+```
+
+### Seeding message history archives
+
+The control node MUST seed the
+[generated torrent](#creating-message-archive-torrents)
+until a new `WakuMessageArchive` is created.
+
+The control node SHOULD NOT seed torrents for older message history archives.
+Only one torrent at a time should be seeded.
+
+### Creating magnet links
+
+Once a torrent file for all message archives is created,
+the control node MUST derive a magnet link following the
+[Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme)
+using the underlying BitTorrent protocol client.
+
+### Message archive distribution
+
+Message archives are available via the BitTorrent network as they are being
+[seeded by the control node](#seeding-message-history-archives).
+Other community member nodes will download the message archives
+from the BitTorrent network once they receive a magnet link
+that contains a message archive index.
+
+The control node MUST send magnet links containing message archives and
+the message archive index to a special community channel.
+The topic of that special channel follows the following format:
+
+```text
+/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}
+```
+
+All messages sent with this topic MUST be instances of `ApplicationMetadataMessage`
+([62/STATUS-PAYLOAD](../62/payload.md)) with a `payload` of `CommunityMessageArchiveIndex`.
+
+Only the control node MAY post to the special channel.
+Other messages on this specified channel MUST be ignored by clients.
+Community members MUST NOT have permission to send messages to the special channel.
+However, community member nodes MUST subscribe to special channel
+to receive Waku messages containing magnet links for message archives.
+
+### Canonical message histories
+
+Only control nodes are allowed to distribute messages with magnet links via
+the special channel for magnet link exchange.
+Community members MUST NOT be allowed to post any messages to the special channel.
+
+Status nodes MUST ensure that any message
+that isn't signed by the control node in the special channel is ignored.
+
+Since the magnet links are created from the control node's database
+(and previously distributed archives),
+the message history provided by the control node becomes the canonical message history
+and single source of truth for the community.
+
+Community member nodes MUST replace messages in their local databases
+with the messages extracted from archives within the same time range.
+Messages that the control node didn't receive MUST be removed and
+are no longer part of the message history of interest,
+even if it already existed in a community member node's database.
+
+## Fetching message history archives
+
+Generally, fetching message history archives is a three step process:
+
+1. Receive [message archive index](#message-history-archive-index)
+magnet link as described in [Message archive distribution],
+download `index` file from torrent, then determine which message archives to download
+2. Download individual archives
+
+Community member nodes subscribe to the special channel
+that control nodes publish magnet links for message history archives to.
+There are two scenarios in which member nodes can receive such a magnet link message
+from the special channel:
+
+1. The member node receives it via live messages, by listening to the special channel
+2. The member node requests messages for a time range of up to 30 days
+from store nodes (this is the case when a new community member joins a community)
+
+### Downloading message archives
+
+When member nodes receive a message with a `CommunityMessageHistoryArchive`
+([62/STATUS-PAYLOADS](../62/payloads.md)) from the aforementioned channnel,
+they MUST extract the `magnet_uri` and
+pass it to their underlying BitTorrent client
+so they can fetch the latest message history archive index,
+which is the `index` file of the torrent (see [Creating message archive torrents](#creating-message-archive-torrents)).
+
+Due to the nature of distributed systems,
+there's no guarantee that a received message is the "last" message.
+This is especially true
+when member nodes request historical messages from store nodes.
+
+Therefore, member nodes MUST wait for 20 seconds
+after receiving the last `CommunityMessageArchive`
+before they start extracting the magnet link to fetch the latest archive index.
+
+Once a message history archive index is downloaded and
+parsed back into `WakuMessageArchiveIndex`,
+community member nodes use a local lookup table
+to determine which of the listed archives are missing
+using the KECCAK-256 hashes stored in the index.
+
+For this lookup to work,
+member nodes MUST store the KECCAK-256 hashes
+of the `WakuMessageArchiveIndexMetadata` provided by the `index` file
+for all of the message history archives that have been downlaoded
+in their local database.
+
+Given a `WakuMessageArchiveIndex`,
+member nodes can access individual `WakuMessageArchiveIndexMetadata`
+to download individual archives.
+
+Community member nodes MUST choose one of the following options:
+
+1. **Download all archives** - Request and
+download all data pieces for `data` provided by the torrent
+(this is the case for new community member nodes
+that haven't downloaded any archives yet)
+2. **Download only the latest archive** -
+Request and download all pieces starting at the `offset` of the latest `WakuMessageArchiveIndexMetadata`
+(this the case for any member node
+that already has downloaded all previous history and
+is now interested in only the latst archive)
+3. **Download specific archives** -
+Look into `from` and
+`to` fields of every `WakuMessageArchiveIndexMetadata` and
+determine the pieces for archives of a specific time range
+(can be the case for member nodes that have recently joined the network and
+are only interested in a subset of the complete history)
+
+### Storing historical messages
+
+When message archives are fetched,
+community member nodes MUST unwrap the resulting `WakuMessage` instances
+into `ApplicationMetadataMessage` instances and store them in their local database.
+Community member nodes SHOULD NOT store the wrapped `WakuMessage` messages.
+
+All message within the same time range
+MUST be replaced with the messages provided by the message history archive.
+
+Community members nodes MUST ignore the expiration state of each archive message.
+
+## Considerations
+
+The following are things to cosider when implementing this specification.
+
+## Control node honesty
+
+This spec assumes that all control nodes are honest and behave according to the spec.
+Meaning they don't inject their own messages into, or
+remove any messages from historic archives.
+
+## Bandwidth consumption
+
+Community member nodes will download the latest archive
+they've received from the archive index,
+which includes messages from the last seven days.
+Assuming that community members nodes were online for that time range,
+they have already downloaded that message data and
+will now download an archive that contains the same.
+
+This means there's a possibility member nodes
+will download the same data at least twice.
+
+## Multiple community owners
+
+It is possible for control nodes
+to export the private key of their owned community and
+pass it to other users so they become control nodes as well.
+This means, it's possible for multiple control nodes to exist.
+
+This might conflict with the assumption that the control node
+serves as a single source of thruth.
+Multiple control nodes can have different message histories.
+
+Not only will multiple control nodes
+multiply the amount of archive index messages being distributed to the network,
+they might also contain different sets of magnet links and their corresponding hashes.
+
+Even if just a single message is missing in one of the histories,
+the hashes presented in archive indices will look completely different,
+resulting in the community member node to download the corresponding archive
+(which might be identical to an archive that was already downloaded,
+except for that one message).
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [13/WAKU2-STORE](../../waku/standards/core/13/store.md)
+- [BitTorrent](https://bittorrent.org)
+- [10/WAKU2](../../waku/standards/core/10/waku2.md)
+- [11/WAKU2-RELAY](../../waku/standards/core/11/relay.md)
+- [Magnet URI scheme](https://en.wikipedia.org/wiki/Magnet_URI_scheme)
+- [forum discussion](https://forum.vac.dev/t/status-communities-protocol-and-product-point-of-view/114)
+- [org channels](https://github.com/status-im/specs/pull/151)
+- [UI feature spec](https://github.com/status-im/feature-specs/pull/36)
+- [Extensions for Peers to Send Metadata Files](https://www.bittorrent.org/beps/bep_0009.html)
+- [org channels spec](../56/communities.md)
+- [14/WAKU2-MESSAGE](../../waku/standards/core/14/message.md)
+- [62/STATUS-PAYLOADS](../62/payloads.md)

status/63/keycard-usage.md

@@ -1,356 +1,365 @@
----
-slug: 63
-title: 63/STATUS-Keycard-Usage
-name: Status Keycard Usage
-status: draft
-category: Standards Track
-description: Describes how an application can use the Status Keycard to create, store and transact with different account addresses.
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-  - Jimmy Debe <jimmy@status.im>
----
-
-## Terminology
-
-- **Account**: A valid [BIP-32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) compliant key.
-- **Multiaccount**: An account from which multiple Accounts can be derived.
-
-## Abstract
-
-This specification describes how an application can use the Status Keycard to -
-
-1. Create Multiaccounts
-2. Store Multiaccounts
-3. Use Multiaccounts for transaction or message signing
-4. Derive Accounts from Multiaccounts
-
-More documentation on the Status Keycard can be found [here](https://keycard.tech/docs/)
-
-## Motivation
-
-The Status Keycard is a hardware wallet that can be used to store and sign transactions.
-For the purpose of the Status App, this specification describes how the Keycard SHOULD be used to store and sign transactions.
-
-## Usage
-
-### Endpoints
-
-#### 1. Initialize Keycard (`/init-keycard`)
-
-To initialize the keycard for use with the application.
-The keycard is locked with a 6 digit pin.
-
-#### Request wire format
-
-```json
-{
-  "pin": 6_digit_pin
-}
-```
-
-#### Response wire format
-
-```json
-{
-  "password": password_to_unlock_keycard,
-  "puk": 12_digit_recovery_code,
-  "pin": provided_pin,
-}
-```
-
-The keycard MUST be initialized before it can be used with the application.
-The application SHOULD provide a way to recover the keycard in case the pin is forgotten.
-
-### 2. Get Application Info (`/get-application-info`)
-
-To fetch if the keycard is ready to be used by the application.
-
-#### Request wire format
-
-The requester MAY add a `pairing` field to filter through the generated keys
-```json
-{
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt> OR null
-}
-```
-
-#### Response wire format
-
-##### If the keycard is not initialized yet
-
-```json
-{
-  "initialized?": false
-}
-```
-
-##### If the keycard is initialized
-
-```json
-{
-  "free-pairing-slots": number, 
-  "app-version": major_version.minor_version, 
-  "secure-channel-pub-key": valid_bip32_key,, 
-  "key-uid": unique_id_of_the_default_key,
-  "instance-uid": unique_instance_id, 
-  "paired?": bool,
-  "has-master-key?": bool, 
-  "initialized?" true
-}
-```
-
-### 3. Pairing the Keycard to the Client device (`/pair`)
-
-To establish a secure communication channel described [here](https://keycard.tech/docs/apdu/opensecurechannel.html), the keycard and the client device need to be paired.
-
-#### Request wire format
-
-```json
-{
-  "password": password_to_unlock_keycard
-}
-```
-
-#### Response wire format
-
-```json
-"<shared_secret>/<pairing_index>/<256_bit_salt>"
-```
-
-### 4. Generate a new set of keys (`/generate-and-load-keys`)
-
-To generate a new set of keys and load them onto the keycard.
-
-#### Request wire format
-
-```json
-{
-  "mnemonic": 12_word_mnemonic,
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
-  "pin": 6_digit_pin
-}
-```
-
-#### Response wire format
-
-```json
-{
-  "whisper-address": 20_byte_whisper_compatible_address, 
-  "whisper-private-key": whisper_private_key, 
-  "wallet-root-public-key": 256_bit_wallet_root_public_key, 
-  "encryption-public-key": 256_bit_encryption_public_key,, 
-  "wallet-root-address": 20_byte_wallet_root_address, 
-  "whisper-public-key": 256_bit_whisper_public_key,
-  "address": 20_byte_address, 
-  "wallet-address": 20_byte_wallet_address,, 
-  "key-uid": 64_byte_unique_key_id, 
-  "wallet-public-key": 256_bit_wallet_public_key,
-  "public-key": 256_bit_public_key,
-  "instance-uid": 32_byte_unique_instance_id,
-}
-```
-
-### 5. Get a set of generated keys (`/get-keys`)
-
-To fetch the keys that are currently loaded on the keycard.
-
-#### Request wire format
-
-```json
-{
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
-  "pin": 6_digit_pin
-}
-```
-
-#### Response wire format
-
-```json
-{
-  "whisper-address": 20_byte_whisper_compatible_address, 
-  "whisper-private-key": whisper_private_key, 
-  "wallet-root-public-key": 256_bit_wallet_root_public_key, 
-  "encryption-public-key": 256_bit_encryption_public_key,
-  "wallet-root-address": 20_byte_wallet_root_address, 
-  "whisper-public-key": 256_bit_whisper_public_key,
-  "address": 20_byte_address, 
-  "wallet-address": 20_byte_wallet_address, 
-  "key-uid": 64_byte_unique_key_id, 
-  "wallet-public-key": 256_bit_wallet_public_key,
-  "public-key": 256_bit_public_key,
-  "instance-uid": 32_byte_unique_instance_id,
-}
-```
-
-### 6. Sign a transaction (`/sign`)
-
-To sign a transaction using the keycard, passing in the pairing information and the transaction to be signed.
-
-#### Request wire format
-
-```json
-{
-  "hash": 64_byte_hash_of_the_transaction,
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
-  "pin": 6_digit_pin,
-  "path": bip32_path_to_the_key
-}
-```
-
-#### Response wire format
-
-```json
-<256_bit_signature>
-```
-
-### 7. Export a key (`/export-key`)
-
-To export a key from the keycard, passing in the pairing information and the path to the key to be exported.
-
-#### Request wire format
-
-```json
-{
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
-  "pin": 6_digit_pin,
-  "path": bip32_path_to_the_key
-}
-```
-
-#### Response wire format
-
-```json
-<256_bit_public_key>
-``` 
-
-### 8. Verify a pin (`/verify-pin`)
-
-To verify the pin of the keycard.
-
-#### Request wire format
-
-```json
-{
-  "pin": 6_digit_pin
-}
-```
-
-#### Response wire format
-
-```json
-1_digit_status_code
-```
-
-Status code reference:
-- 3: PIN is valid
-<!--TODO: what are the other status codes?-->
-
-
-### 9. Change the pin (`/change-pin`)
-
-To change the pin of the keycard.
-
-#### Request wire format
-
-```json
-{
-  "new-pin": 6_digit_new_pin,
-  "current-pin": 6_digit_new_pin,
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>
-}
-```
-
-#### Response wire format
-
-##### If the operation was successful
-
-```json
-true
-```
-
-##### If the operation was unsuccessful
-
-```json
-false
-```
-
-### 10. Unblock the keycard (`/unblock-pin`)
-
-If the Keycard is blocked due to too many incorrect pin attempts, it can be unblocked using the PUK.
-
-#### Request wire format
-
-```json
-{
-  "puk": 12_digit_recovery_code,
-  "new-pin": 6_digit_new_pin,
-  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>
-}
-```
-
-#### Response wire format
-
-##### If the operation was successful
-
-```json
-true
-```
-
-##### If the operation was unsuccessful
-
-```json
-false
-```
-
-## Flows
-
-Any application that uses the Status Keycard MAY implement the following flows according to the actions listed above.
-
-### 1. A new user wants to use the Keycard with the application
-
-1. The user initializes the Keycard using the `/init-keycard` endpoint.
-2. The user pairs the Keycard with the client device using the `/pair` endpoint.
-3. The user generates a new set of keys using the `/generate-and-load-keys` endpoint.
-4. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
-
-### 2. An existing user wants to use the Keycard with the application
-
-1. The user pairs the Keycard with the client device using the `/pair` endpoint.
-2. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
-
-### 3. An existing user wants to use the Keycard with a new client device
-
-1. The user pairs the Keycard with the new client device using the `/pair` endpoint.
-2. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
-
-### 4. An existing user wishes to verify the pin of the Keycard
-
-1. The user verifies the pin of the Keycard using the `/verify-pin` endpoint.
-
-### 5. An existing user wishes to change the pin of the Keycard
-
-1. The user changes the pin of the Keycard using the `/change-pin` endpoint.
-
-### 6. An existing user wishes to unblock the Keycard
-
-1. The user unblocks the Keycard using the `/unblock-pin` endpoint.
-
-
-## Security Considerations
-
-Inherits the security considerations of [Status Keycard](https://keycard.tech/docs/)
-
-## Privacy Considerations
-
-Inherits the privacy considerations of [Status Keycard](https://keycard.tech/docs/)
-
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [BIP-32 specification](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)
-2. [Keycard documentation](https://keycard.tech/docs/)
-3. [16/Keycard-Usage](https://specs.status.im/draft/16)
+---
+slug: 63
+title: 63/STATUS-Keycard-Usage
+name: Status Keycard Usage
+status: draft
+category: Standards Track
+description: Describes how an application can use the Status Keycard to create, store and transact with different account addresses.
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+  - Jimmy Debe <jimmy@status.im>
+---
+
+## Terminology
+
+- **Account**: A valid
+[BIP-32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)
+compliant key.
+- **Multiaccount**: An account from which multiple Accounts can be derived.
+
+## Abstract
+
+This specification describes how an application can use the Status Keycard to -
+
+1. Create Multiaccounts
+2. Store Multiaccounts
+3. Use Multiaccounts for transaction or message signing
+4. Derive Accounts from Multiaccounts
+
+More documentation on the Status Keycard can be found [here](https://keycard.tech/docs/)
+
+## Motivation
+
+The Status Keycard is a hardware wallet that can be used to store and
+sign transactions.
+For the purpose of the Status App,
+this specification describes how the Keycard SHOULD be used to store and
+sign transactions.
+
+## Usage
+
+### Endpoints
+
+#### 1. Initialize Keycard (`/init-keycard`)
+
+To initialize the keycard for use with the application.
+The keycard is locked with a 6 digit pin.
+
+#### Request wire format
+
+```json
+{
+  "pin": 6_digit_pin
+}
+```
+
+#### Response wire format
+
+```json
+{
+  "password": password_to_unlock_keycard,
+  "puk": 12_digit_recovery_code,
+  "pin": provided_pin,
+}
+```
+
+The keycard MUST be initialized before it can be used with the application.
+The application SHOULD provide a way to recover the keycard in case the pin is forgotten.
+
+### 2. Get Application Info (`/get-application-info`)
+
+To fetch if the keycard is ready to be used by the application.
+
+#### Request wire format
+
+The requester MAY add a `pairing` field to filter through the generated keys
+
+```json
+{
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt> OR null
+}
+```
+
+#### Response wire format
+
+##### If the keycard is not initialized yet
+
+```json
+{
+  "initialized?": false
+}
+```
+
+##### If the keycard is initialized
+
+```json
+{
+  "free-pairing-slots": number, 
+  "app-version": major_version.minor_version, 
+  "secure-channel-pub-key": valid_bip32_key,, 
+  "key-uid": unique_id_of_the_default_key,
+  "instance-uid": unique_instance_id, 
+  "paired?": bool,
+  "has-master-key?": bool, 
+  "initialized?" true
+}
+```
+
+### 3. Pairing the Keycard to the Client device (`/pair`)
+
+To establish a secure communication channel described [here](https://keycard.tech/docs/apdu/opensecurechannel.html),
+the keycard and the client device need to be paired.
+
+#### Request wire format
+
+```json
+{
+  "password": password_to_unlock_keycard
+}
+```
+
+#### Response wire format
+
+```json
+"<shared_secret>/<pairing_index>/<256_bit_salt>"
+```
+
+### 4. Generate a new set of keys (`/generate-and-load-keys`)
+
+To generate a new set of keys and load them onto the keycard.
+
+#### Request wire format
+
+```json
+{
+  "mnemonic": 12_word_mnemonic,
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
+  "pin": 6_digit_pin
+}
+```
+
+#### Response wire format
+
+```json
+{
+  "whisper-address": 20_byte_whisper_compatible_address, 
+  "whisper-private-key": whisper_private_key, 
+  "wallet-root-public-key": 256_bit_wallet_root_public_key, 
+  "encryption-public-key": 256_bit_encryption_public_key,, 
+  "wallet-root-address": 20_byte_wallet_root_address, 
+  "whisper-public-key": 256_bit_whisper_public_key,
+  "address": 20_byte_address, 
+  "wallet-address": 20_byte_wallet_address,, 
+  "key-uid": 64_byte_unique_key_id, 
+  "wallet-public-key": 256_bit_wallet_public_key,
+  "public-key": 256_bit_public_key,
+  "instance-uid": 32_byte_unique_instance_id,
+}
+```
+
+### 5. Get a set of generated keys (`/get-keys`)
+
+To fetch the keys that are currently loaded on the keycard.
+
+#### Request wire format
+
+```json
+{
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
+  "pin": 6_digit_pin
+}
+```
+
+#### Response wire format
+
+```json
+{
+  "whisper-address": 20_byte_whisper_compatible_address, 
+  "whisper-private-key": whisper_private_key, 
+  "wallet-root-public-key": 256_bit_wallet_root_public_key, 
+  "encryption-public-key": 256_bit_encryption_public_key,
+  "wallet-root-address": 20_byte_wallet_root_address, 
+  "whisper-public-key": 256_bit_whisper_public_key,
+  "address": 20_byte_address, 
+  "wallet-address": 20_byte_wallet_address, 
+  "key-uid": 64_byte_unique_key_id, 
+  "wallet-public-key": 256_bit_wallet_public_key,
+  "public-key": 256_bit_public_key,
+  "instance-uid": 32_byte_unique_instance_id,
+}
+```
+
+### 6. Sign a transaction (`/sign`)
+
+To sign a transaction using the keycard, passing in the pairing information and
+the transaction to be signed.
+
+#### Request wire format
+
+```json
+{
+  "hash": 64_byte_hash_of_the_transaction,
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
+  "pin": 6_digit_pin,
+  "path": bip32_path_to_the_key
+}
+```
+
+#### Response wire format
+
+```json
+<256_bit_signature>
+```
+
+### 7. Export a key (`/export-key`)
+
+To export a key from the keycard, passing in the pairing information and
+the path to the key to be exported.
+
+#### Request wire format
+
+```json
+{
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>,
+  "pin": 6_digit_pin,
+  "path": bip32_path_to_the_key
+}
+```
+
+#### Response wire format
+
+```json
+<256_bit_public_key>
+```
+
+### 8. Verify a pin (`/verify-pin`)
+
+To verify the pin of the keycard.
+
+#### Request wire format
+
+```json
+{
+  "pin": 6_digit_pin
+}
+```
+
+#### Response wire format
+
+```json
+1_digit_status_code
+```
+
+Status code reference:
+
+- 3: PIN is valid
+<!--TODO: what are the other status codes?-->
+
+### 9. Change the pin (`/change-pin`)
+
+To change the pin of the keycard.
+
+#### Request wire format
+
+```json
+{
+  "new-pin": 6_digit_new_pin,
+  "current-pin": 6_digit_new_pin,
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>
+}
+```
+
+#### Response wire format
+
+##### If the operation was successful
+
+```json
+true
+```
+
+##### If the operation was unsuccessful
+
+```json
+false
+```
+
+### 10. Unblock the keycard (`/unblock-pin`)
+
+If the Keycard is blocked due to too many incorrect pin attempts,
+it can be unblocked using the PUK.
+
+#### Request wire format
+
+```json
+{
+  "puk": 12_digit_recovery_code,
+  "new-pin": 6_digit_new_pin,
+  "pairing": <shared_secret>/<pairing_index>/<256_bit_salt>
+}
+```
+
+#### Response wire format
+
+##### If the operation was successful
+
+```json
+true
+```
+
+##### If the operation was unsuccessful
+
+```json
+false
+```
+
+## Flows
+
+Any application that uses the Status Keycard
+MAY implement the following flows according to the actions listed above.
+
+### 1. A new user wants to use the Keycard with the application
+
+1. The user initializes the Keycard using the `/init-keycard` endpoint.
+2. The user pairs the Keycard with the client device using the `/pair` endpoint.
+3. The user generates a new set of keys using the `/generate-and-load-keys` endpoint.
+4. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
+
+### 2. An existing user wants to use the Keycard with the application
+
+1. The user pairs the Keycard with the client device using the `/pair` endpoint.
+2. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
+
+### 3. An existing user wants to use the Keycard with a new client device
+
+1. The user pairs the Keycard with the new client device using the `/pair` endpoint.
+2. The user can now use the Keycard to sign transactions using the `/sign` endpoint.
+
+### 4. An existing user wishes to verify the pin of the Keycard
+
+1. The user verifies the pin of the Keycard using the `/verify-pin` endpoint.
+
+### 5. An existing user wishes to change the pin of the Keycard
+
+1. The user changes the pin of the Keycard using the `/change-pin` endpoint.
+
+### 6. An existing user wishes to unblock the Keycard
+
+1. The user unblocks the Keycard using the `/unblock-pin` endpoint.
+
+## Security Considerations
+
+Inherits the security considerations of [Status Keycard](https://keycard.tech/docs/)
+
+## Privacy Considerations
+
+Inherits the privacy considerations of [Status Keycard](https://keycard.tech/docs/)
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [BIP-32 specification](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)
+2. [Keycard documentation](https://keycard.tech/docs/)
+3. [16/Keycard-Usage](https://specs.status.im/draft/16)

status/65/account-address.md

@@ -1,116 +1,141 @@
----
-slug: 65
-title: 65/STATUS-ACCOUNT-ADDRESS
-name: Status Account Address
-status: draft
-category: Standards Track
-description: Details of what a Status account address is and how account addresses are created and used.
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-- Corey Petty <corey@status.im>
-- Oskar Thorén <oskarth@titanproxy.com>
-- Samuel Hawksby-Robinson <samuel@status.im>
----
-
-## Abstract
-
-This specification details what a Status account address is and how account addresses are created and used.
-
-## Background
-
-The core concept of an account in Status is a set of cryptographic keypairs. Namely, the combination of the following:
-1. a Waku chat identity keypair
-1. a set of cryptocurrency wallet keypairs
-
-The Status node verifies or derives everything else associated with the contact from the above items, including:
-- Ethereum address (future verification, currently the same base keypair)
-- identicon
-- message signatures
-
-## Initial Key Generation
-### Public/Private Keypairs 
-- An ECDSA (secp256k1 curve) public/private keypair MUST be generated via a [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki) derived path from a [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) mnemonic seed phrase.
-- The default paths are defined as such:
-    - Waku Chat Key (`IK`): `m/43'/60'/1581'/0'/0`  (post Multiaccount integration)
-        - following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
-    - Status Wallet paths: `m/44'/60'/0'/0/i` starting at `i=0`
-        - following [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
-        - NOTE: this (`i=0`) is also the current (and only) path for Waku key before Multiaccount integration
-
-## Account Broadcasting
-- A user is responsible for broadcasting certain information publicly so that others may contact them.
-
-### X3DH Prekey bundles
-- Refer to [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) for details on the X3DH prekey bundle broadcasting, as well as regeneration.
-
-## Optional Account additions
-
-### ENS Username
-- A user MAY register a public username on the Ethereum Name System (ENS).  This username is a user-chosen subdomain of the `stateofus.eth` ENS registration that maps to their Waku identity key (`IK`). 
-
-### User Profile Picture
-- An account MAY edit the `IK` generated identicon with a chosen picture.  This picture will become part of the publicly broadcasted profile of the account.
-
-<!-- TODO: Elaborate on wallet account and multiaccount -->
-
-## Wire Format
-
-Below is the wire format for the account information that is broadcasted publicly.
-An Account is referred to as a Multiaccount in the wire format.
-
-```proto
-message MultiAccount {
-  string name = 1; // name of the account
-  int64 timestamp = 2; // timestamp of the message
-  string identicon = 3; // base64 encoded identicon
-  repeated ColorHash color_hash = 4; // color hash of the identicon
-  int64 color_id = 5; // color id of the identicon
-  string keycard_pairing = 6; // keycard pairing code
-  string key_uid = 7; // unique identifier of the account
-  repeated IdentityImage images = 8; // images associated with the account
-  string customization_color = 9; // color of the identicon
-  uint64 customization_color_clock = 10; // clock of the identicon color, to track updates
-
-  message ColorHash {
-    repeated int64 index = 1;
-  }
-
-  message IdentityImage {
-    string key_uid = 1; // unique identifier of the image
-    string name = 2; // name of the image
-    bytes payload = 3; // payload of the image
-    int64 width = 4; // width of the image
-    int64 height = 5; // height of the image
-    int64 filesize = 6; // filesize of the image
-    int64 resize_target = 7; // resize target of the image
-    uint64 clock = 8; // clock of the image, to track updates
-  }
-}
-```
-
-The above payload is broadcasted when 2 devices that belong to a user need to be paired.
-
-## Security Considerations
-
-- This specification inherits security considerations of [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) and [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md).
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-### normative
-
-- [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
-- [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
-- [55/STATUS-1TO1-CHAT](../55/1to1-chat.md)
-
-## informative
-
-- [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
-- [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
-- [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
-- [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
-- [Ethereum Name System](https://ens.domains/)
-- [Status Multiaccount](../63/account-address.md)
+---
+slug: 65
+title: 65/STATUS-ACCOUNT-ADDRESS
+name: Status Account Address
+status: draft
+category: Standards Track
+description: Details of what a Status account address is and how account addresses are created and used.
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Corey Petty <corey@status.im>
+- Oskar Thorén <oskarth@titanproxy.com>
+- Samuel Hawksby-Robinson <samuel@status.im>
+---
+
+## Abstract
+
+This specification details what a Status account address is and
+how account addresses are created and used.
+
+## Background
+
+The core concept of an account in Status is a set of cryptographic keypairs.
+Namely, the combination of the following:
+
+1. a Waku chat identity keypair
+1. a set of cryptocurrency wallet keypairs
+
+The Status node verifies or
+derives everything else associated with the contact from the above items, including:
+
+- Ethereum address (future verification, currently the same base keypair)
+- identicon
+- message signatures
+
+## Initial Key Generation
+
+### Public/Private Keypairs
+
+- An ECDSA (secp256k1 curve) public/private keypair MUST be generated via a
+[BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
+derived path from a
+[BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
+mnemonic seed phrase.
+
+- The default paths are defined as such:
+  - Waku Chat Key (`IK`): `m/43'/60'/1581'/0'/0`  (post Multiaccount integration)
+    - following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
+  - Status Wallet paths: `m/44'/60'/0'/0/i` starting at `i=0`
+    - following [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
+    - NOTE: this (`i=0`) is also the current (and only)
+    path for Waku key before Multiaccount integration
+
+## Account Broadcasting
+
+- A user is responsible for broadcasting certain information publicly so
+that others may contact them.
+
+### X3DH Prekey bundles
+
+- Refer to [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
+for details on the X3DH prekey bundle broadcasting, as well as regeneration.
+
+## Optional Account additions
+
+### ENS Username
+
+- A user MAY register a public username on the Ethereum Name System (ENS).
+This username is a user-chosen subdomain of the `stateofus.eth`
+ENS registration that maps to their Waku identity key (`IK`).
+
+### User Profile Picture
+
+- An account MAY edit the `IK` generated identicon with a chosen picture.
+This picture will become part of the publicly broadcasted profile of the account.
+
+<!-- TODO: Elaborate on wallet account and multiaccount -->
+
+## Wire Format
+
+Below is the wire format for the account information that is broadcasted publicly.
+An Account is referred to as a Multiaccount in the wire format.
+
+```proto
+message MultiAccount {
+  string name = 1; // name of the account
+  int64 timestamp = 2; // timestamp of the message
+  string identicon = 3; // base64 encoded identicon
+  repeated ColorHash color_hash = 4; // color hash of the identicon
+  int64 color_id = 5; // color id of the identicon
+  string keycard_pairing = 6; // keycard pairing code
+  string key_uid = 7; // unique identifier of the account
+  repeated IdentityImage images = 8; // images associated with the account
+  string customization_color = 9; // color of the identicon
+  uint64 customization_color_clock = 10; // clock of the identicon color, to track updates
+
+  message ColorHash {
+    repeated int64 index = 1;
+  }
+
+  message IdentityImage {
+    string key_uid = 1; // unique identifier of the image
+    string name = 2; // name of the image
+    bytes payload = 3; // payload of the image
+    int64 width = 4; // width of the image
+    int64 height = 5; // height of the image
+    int64 filesize = 6; // filesize of the image
+    int64 resize_target = 7; // resize target of the image
+    uint64 clock = 8; // clock of the image, to track updates
+  }
+}
+```
+
+The above payload is broadcasted when 2 devices
+that belong to a user need to be paired.
+
+## Security Considerations
+
+- This specification inherits security considerations of
+[53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md) and
+[54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md).
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+### normative
+
+- [53/WAKU2-X3DH](../../waku/standards/application/53/x3dh.md)
+- [54/WAKU2-X3DH-SESSIONS](../../waku/standards/application/54/x3dh-sessions.md)
+- [55/STATUS-1TO1-CHAT](../55/1to1-chat.md)
+
+## informative
+
+- [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
+- [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
+- [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
+- [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
+- [Ethereum Name System](https://ens.domains/)
+- [Status Multiaccount](../63/account-address.md)

status/README.md

@@ -1,4 +1,4 @@
-# Status RFCs
-
-Status is a communitication tool providing privacy features for the user.
-Specifcations can also be viewd at [Status](https://status.app/specs).
+# Status RFCs
+
+Status is a communitication tool providing privacy features for the user.
+Specifcations can also be viewd at [Status](https://status.app/specs).

status/raw/status-mvds.md

@@ -0,0 +1,129 @@
+---
+title: STATUS-MVDS-USAGE
+name: MVDS Usage in Status
+status: raw
+category: Best Current Practice
+description: Defines how MVDS protocol used by different message types in Status.
+editor: Kaichao Sun <kaichao@status.im>
+contributors:
+---
+
+## Abstract
+
+This document lists the types of messages that are using [MVDS](/vac/2/mvds.md)
+in the Status application.
+
+## Background
+
+Status app uses MVDS to ensure messages going through Waku
+are acknolwedged by the recipient.
+This is to ensure that the messages are not missed by any interested parties.
+
+## Message types
+
+Various Message Types contain distinct information defined by the app
+to facilitate convenient serialization and deserialization.
+
+E2E reliability is a feature that ensures messages are delivered to the recipient.
+This is initially achieved by using MVDS in Status.
+
+Chat Type specifies the category of chat that a message belongs to.
+It can be OneToOne (aka Direct Message), GroupChat, or CommunityChat.
+These are the three main types of chats in Status.
+
+| Message Type                                                               | Use MVDS                            | Need e2e reliability | Chat Type               |
+|----------------------------------------------------------------------------|-------------------------------------|----------------------|-------------------------|
+| ApplicationMetadataMessage_UNKNOWN                                         | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_CHAT_MESSAGE                                    | Yes for OneToOne & PrivateGroupChat | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_CONTACT_UPDATE                                  | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_MEMBERSHIP_UPDATE_MESSAGE                       | No                                  | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_PAIR_INSTALLATION                          | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_DEPRECATED_SYNC_INSTALLATION                    | No                                  | No                   | Pair                    |
+| ApplicationMetadataMessage_REQUEST_ADDRESS_FOR_TRANSACTION                 | Yes for OneToOne                    | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_ACCEPT_REQUEST_ADDRESS_FOR_TRANSACTION          | Yes for OneToOne                    | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_DECLINE_REQUEST_ADDRESS_FOR_TRANSACTION         | Yes for OneToOne                    | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_REQUEST_TRANSACTION                             | Yes for OneToOne                    | Yes                  | OneToOne & GroupChat    |
+| ApplicationMetadataMessage_SEND_TRANSACTION                                | Yes for OneToOne                    | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_DECLINE_REQUEST_TRANSACTION                     | Yes for OneToOne                    | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_SYNC_INSTALLATION_CONTACT_V2                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_INSTALLATION_ACCOUNT                       | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_CONTACT_CODE_ADVERTISEMENT                      | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_REGISTRATION                  | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_REGISTRATION_RESPONSE         | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_QUERY                         | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_QUERY_RESPONSE                | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_REQUEST                       | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_PUSH_NOTIFICATION_RESPONSE                      | No                                  | No                   | One & Group & Community |
+| ApplicationMetadataMessage_EMOJI_REACTION                                  | No                                  | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_GROUP_CHAT_INVITATION                           | Yes                                 | Yes                  | GroupChat               |
+| ApplicationMetadataMessage_CHAT_IDENTITY                                   | No                                  | No                   | OneToOne                |
+| ApplicationMetadataMessage_COMMUNITY_DESCRIPTION                           | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_INVITATION                            | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_REQUEST_TO_JOIN                       | No                                  | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_PIN_MESSAGE                                     | Yes for OneToOne & PrivateGroupChat | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_EDIT_MESSAGE                                    | Yes for OneToOne & PrivateGroupChat | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_STATUS_UPDATE                                   | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_DELETE_MESSAGE                                  | Yes for OneToOne & PrivateGroupChat | Yes                  | One & Group & Community |
+| ApplicationMetadataMessage_SYNC_INSTALLATION_COMMUNITY                     | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_ANONYMOUS_METRIC_BATCH                          | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_SYNC_CHAT_REMOVED                               | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_CHAT_MESSAGES_READ                         | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_BACKUP                                          | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_READ                       | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_ACCEPTED                   | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_DISMISSED                  | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_BOOKMARK                                   | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_CLEAR_HISTORY                              | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_SETTING                                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK            | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_PROFILE_PICTURES                           | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACCOUNT                                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_ACCEPT_CONTACT_REQUEST                          | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_RETRACT_CONTACT_REQUEST                         | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_COMMUNITY_REQUEST_TO_JOIN_RESPONSE              | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_COMMUNITY_SETTINGS                         | Yes                                 | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_REQUEST_CONTACT_VERIFICATION                    | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_ACCEPT_CONTACT_VERIFICATION                     | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_DECLINE_CONTACT_VERIFICATION                    | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_SYNC_TRUSTED_USER                               | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_VERIFICATION_REQUEST                       | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_CONTACT_REQUEST_DECISION                   | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_REQUEST_TO_LEAVE                      | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_DELETE_FOR_ME_MESSAGE                      | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_SAVED_ADDRESS                              | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_CANCEL_REQUEST_TO_JOIN                | No                                  | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_CANCEL_CONTACT_VERIFICATION                     | Yes                                 | Yes                  | OneToOne                |
+| ApplicationMetadataMessage_SYNC_KEYPAIR                                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_SOCIAL_LINKS                               | No                                  | No                   | Not Applied             |
+| ApplicationMetadataMessage_SYNC_ENS_USERNAME_DETAIL                        | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_EVENTS_MESSAGE                        | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_EDIT_SHARED_ADDRESSES                 | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_ACCOUNT_CUSTOMIZATION_COLOR                | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACCOUNTS_POSITIONS                         | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_PRIVILEGED_USER_SYNC_MESSAGE          | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_SHARD_KEY                             | Yes                                 | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_CHAT                                       | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_DELETED                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_UNREAD                     | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_ACTIVITY_CENTER_COMMUNITY_REQUEST_DECISION | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_SYNC_TOKEN_PREFERENCES                          | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_PUBLIC_SHARD_INFO                     | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_COLLECTIBLE_PREFERENCES                    | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_USER_KICKED                           | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_SYNC_PROFILE_SHOWCASE_PREFERENCES               | Yes                                 | Yes                  | Pair                    |
+| ApplicationMetadataMessage_COMMUNITY_PUBLIC_STORENODES_INFO                | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_REEVALUATE_PERMISSIONS_REQUEST        | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_DELETE_COMMUNITY_MEMBER_MESSAGES                | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_UPDATE_GRANT                          | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_ENCRYPTION_KEYS_REQUEST               | No                                  | Yes                  | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_TOKEN_ACTION                          | No                                  | Weak Yes             | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_SHARED_ADDRESSES_REQUEST              | No                                  | No                   | CommunityChat           |
+| ApplicationMetadataMessage_COMMUNITY_SHARED_ADDRESSES_RESPONSE             | No                                  | No                   | CommunityChat           |
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [MVDS](/vac/2/mvds.md)

status/raw/status-waku-usage.md

@@ -1,204 +1,218 @@
----
-title: STATUS-WAKU2-USAGE
-name: Status Waku2 Usage
-status: raw
-category: Best Current Practice
-description: Defines how the Status application uses the Waku protocols.
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors: 
-- Jimmy Debe <jimmy@status.im>
-
----
-
-## Abstract
-
-Status is a chat application which has several features, including, but not limited to -
-- Private 1:1 chats, described by [55/STATUS-1TO1-CHAT](/spec/55)
-- Large scale group chats, described by [56/STATUS-COMMUNITIES](/spec/56)
-
-This specification describes how a Status implementation will make use of the underlying infrastructure, 
-Waku, which is described in [10/WAKU2](/spec/10).
-
-## Background 
-
-The Status application aspires to achieve censorship resistance and incorporates specific privacy features, 
-leveraging the comprehensive set of protocols offered by Waku to enhance these attributes. 
-Waku protocols provide secure communication capabilities over decentralized networks. 
-Once integrated, an application will benefit from privacy-preserving, 
-censorship resistance and spam protected communcation. 
-
-Since Status uses a large set of Waku protocols, 
-it is imperative to describe how each are used. 
-
-## Terminology
-
-| Name  | Description |
-| --------------- | --------- |
-| `RELAY`| This refers to the Waku Relay protocol, described in [11/WAKU2-RELAY](/spec/11) |
-| `FILTER` | This refers to the Waku Filter protocol, described in [12/WAKU2-FILTER](/spec/12) |
-| `STORE` | This refers to the Waku Store protocol, described in [13/WAKU2-STORE](/spec/13) |
-| `MESSAGE` | This refers to the Waku Message format, described in [14/WAKU2-MESSAGE](/spec/14) |
-| `LIGHTPUSH` | This refers to the Waku Lightpush protocol, described in [19/WAKU2-LIGHTPUSH](/spec/19) |
-| Discovery | This refers to a peer discovery method used by a Waku node. |
-| `Pubsub Topic` / `Content Topic` | This refers to the routing of messages within the Waku network, described in [23/WAKU2-TOPICS](/spec/23/) |
-
-### Waku Node:
-
-Software that is configured with a set of Waku protocols.
-A Status client comprises of a Waku node that is a `RELAY` node or a non-relay node.
-
-
-### Light Client:
-
-A Status client that operates within resource constrained environments is a node configured as light client.
-Light clients do not run a `RELAY`.
-Instead, Status light clients,
-can request services from other `RELAY` node that provide `LIGHTPUSH` service.
-
-## Protocol Usage
-
-The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, 
-“NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
-
-The following is a list of Waku Protocols used by a Status application.
-
-### 1. `RELAY`
-
-The `RELAY` MUST NOT be used by Status light clients.
-The `RELAY` is used to broadcast messages between Status clients.
-All Status messages are transformed into [14/WAKU2-MESSAGE](/spec/14), which are sent over the wire.
-
-All Status message types are described in [62/STATUS-PAYLOAD](/spec/62).
-Status Clients MUST transform the following object into a `MESSAGE` as described below -
-
-```go
-
-type StatusMessage struct {
-    SymKey[]     []byte // [optional] The symmetric key used to encrypt the message
-    PublicKey    []byte // [optional] The public key to use for asymmetric encryption
-    Sig          string // [optional] The private key used to sign the message
-    PubsubTopic  string // The Pubsub topic to publish the message to     
-    ContentTopic string // The Content topic to publish the message to
-    Payload      []byte // A serialized representation of a Status message to be sent
-    Padding      []byte // Padding that must be applied to the Payload
-    TargetPeer   string // [optional] The target recipient of the message
-    Ephemeral    bool   // If the message is not to be stored, this is set to `true` 
-}
-
-```
-
-1. A user MUST only provide either a Symmetric key OR an Asymmetric keypair to encrypt the message.
-If both are received, the implementation MUST throw an error.
-2. `WakuMessage.Payload` MUST be set to `StatusMessage.Payload` 
-3. `WakuMessage.Key` MUST be set to `StatusMessage.SymKey`
-4. `WakuMessage.Version` MUST be set to `1`
-5. `WakuMessage.Ephemeral` MUST be set to `StatusMessage.Ephemeral`
-6. `WakuMessage.ContentTopic` MUST be set to `StatusMessage.ContentTopic`
-7. `WakuMessage.Timestamp` MUST be set to the current Unix epoch timestamp (in nanosecond precision)
-
-### 2. `STORE`
-
-This protocol MUST remain optional according to the user's preferences,
-it MAY be enabled on Light clients as well.
-
-Messages received via [11/WAKU2-RELAY](/spec/11), are stored in a database.
-When Waku node running this protocol is service node, 
-it MUST provide the complete list of network messages.
-Status clients SHOULD request historical messages from this service node.
-
-The messages that have the `WakuMessage.Ephemeral` flag set to true will not be stored.
-
-The Status client MAY provide a method to prune the database of older records to save storage.
-
-### 3. `FILTER`
-
-This protocol SHOULD be enabled on Light clients.
-
-This protocol SHOULD be used to filter messages based on a given criteria, such as the `Content Topic` of a `MESSAGE`.
-This allows a reduction in bandwidth consumption by the Status client.
-
-#### Content filtering protocol identifers:
-The `filter-subcribe` SHOULD be implemented on `RELAY` nodes to provide `FILTER` services.
-
-`filter-subscribe`:
-
-    /vac/waku/filter-subscribe/2.0.0-beta1
-
-The `filter-push` SHOULD be implemented on light clients to receive messages.
-
-`filter-push`:
-
-    /vac/waku/filter-push/2.0.0-beta1
-
-Status clients SHOULD apply a filter for all the `Content Topic` they are interested in, 
-such as `Content Topic` derived from -
-1. 1:1 chats with other users, described in [55/STATUS-1TO1-CHAT](/spec/55)
-2. Group chats
-3. Community Channels, described in [56/STATUS-COMMUNITIES](/spec/56)
-
-### 4. `LIGHTPUSH`
-
-The `LIGHTPUSH` protocol MUST be enabled on Status light clients.
-A Status `RELAY` node MAY implement `LIGHTPUSH` to support light clients. 
-Peers will be able to publish messages,
-without running a full-fledged [11/WAKU2-RELAY](/spec/11) protocol.
-
-When a Status client is publishing a message, 
-it MUST check if Light mode is enabled,
-and if so, it MUST publish the message via this protocol.
-
-### 5. Discovery
-
-A discovery method MUST be supported by Light clients and Full clients
-
-Status clients SHOULD make use of the following peer discovery methods that are provided by Waku,
-such as -
-
-1. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
-2. [33/WAKU2-DISCV5](/spec/33): 
-A node discovery protocol to create decentralized network of interconnected Waku nodes. 
-3. [34/WAKU2-PEER-EXCHANGE](/spec/34):
-A peer discovery protocol for resource restricted devices.
-
-Status clients MAY use any combination of the above peer discovery methods, 
-which is suited best for their implementation.
-
-## Security/Privacy Considerations
-
-This specification inherits the security and privacy considerations from the following specifications -
-
-1. [10/WAKU2](/spec/10)
-2. [11/WAKU2-RELAY](/spec/11)
-3. [12/WAKU2-FILTER](/spec/12)
-4. [13/WAKU2-STORE](/spec/13)
-5. [14/WAKU2-MESSAGE](/spec/14)
-6. [23/WAKU2-TOPICS](/spec/23)
-7. [19/WAKU2-LIGHTPUSH](/spec/19)
-8. [55/STATUS-1TO1-CHAT](/spec/55)
-9. [56/STATUS-COMMUNITIES](/spec/56)
-10. [62/STATUS-PAYLOAD](/spec/62)
-11. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
-12. [33/WAKU2-DISCV5](/spec/33)
-13. [34/WAKU2-PEER-EXCHANGE](/spec/34)
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-
-1. [55/STATUS-1TO1-CHAT](/spec/55)
-2. [56/STATUS-COMMUNITIES](/spec/56)
-3. [10/WAKU2](/spec/10)
-4. [11/WAKU2-RELAY](/spec/11)
-5. [12/WAKU2-FILTER](/spec/12)
-6. [13/WAKU2-STORE](/spec/13)
-7. [14/WAKU2-MESSAGE](/spec/14)
-8. [23/WAKU2-TOPICS](/spec/23)
-9. [19/WAKU2-LIGHTPUSH](/spec/19)
-10. [64/WAKU2-NETWORK](/spec/64)
-11. [62/STATUS-PAYLOAD](/spec/62)
-12. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
-13. [33/WAKU2-DISCV5](/spec/33)
-14. [34/WAKU2-PEER-EXCHANGE](/spec/34)
+---
+title: STATUS-WAKU2-USAGE
+name: Status Waku2 Usage
+status: raw
+category: Best Current Practice
+description: Defines how the Status application uses the Waku protocols.
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors: 
+- Jimmy Debe <jimmy@status.im>
+
+---
+
+## Abstract
+
+Status is a chat application which has several features,
+including, but not limited to -
+
+- Private 1:1 chats, described by [55/STATUS-1TO1-CHAT](/spec/55)
+- Large scale group chats, described by [56/STATUS-COMMUNITIES](/spec/56)
+
+This specification describes how a Status implementation will make use of
+the underlying infrastructure, Waku,
+which is described in [10/WAKU2](/spec/10).
+
+## Background
+
+The Status application aspires to achieve censorship resistance and
+incorporates specific privacy features,
+leveraging the comprehensive set of protocols offered by Waku to enhance these attributes.
+Waku protocols provide secure communication capabilities over decentralized networks.
+Once integrated, an application will benefit from privacy-preserving,
+censorship resistance and spam protected communcation.
+
+Since Status uses a large set of Waku protocols,
+it is imperative to describe how each are used.
+
+## Terminology
+
+| Name  | Description |
+| --------------- | --------- |
+| `RELAY`| This refers to the Waku Relay protocol, described in [11/WAKU2-RELAY](/spec/11) |
+| `FILTER` | This refers to the Waku Filter protocol, described in [12/WAKU2-FILTER](/spec/12) |
+| `STORE` | This refers to the Waku Store protocol, described in [13/WAKU2-STORE](/spec/13) |
+| `MESSAGE` | This refers to the Waku Message format, described in [14/WAKU2-MESSAGE](/spec/14) |
+| `LIGHTPUSH` | This refers to the Waku Lightpush protocol, described in [19/WAKU2-LIGHTPUSH](/spec/19) |
+| Discovery | This refers to a peer discovery method used by a Waku node. |
+| `Pubsub Topic` / `Content Topic` | This refers to the routing of messages within the Waku network, described in [23/WAKU2-TOPICS](/spec/23/) |
+
+### Waku Node
+
+Software that is configured with a set of Waku protocols.
+A Status client comprises of a Waku node that is a `RELAY` node or a non-relay node.
+
+### Light Client
+
+A Status client that operates within resource constrained environments
+is a node configured as light client.
+Light clients do not run a `RELAY`.
+Instead, Status light clients,
+can request services from other `RELAY` node that provide `LIGHTPUSH` service.
+
+## Protocol Usage
+
+The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”,
+“SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and
+“OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
+
+The following is a list of Waku Protocols used by a Status application.
+
+### 1. `RELAY`
+
+The `RELAY` MUST NOT be used by Status light clients.
+The `RELAY` is used to broadcast messages between Status clients.
+All Status messages are transformed into [14/WAKU2-MESSAGE](/spec/14),
+which are sent over the wire.
+
+All Status message types are described in [62/STATUS-PAYLOAD](/spec/62).
+Status Clients MUST transform the following object into a `MESSAGE`
+as described below -
+
+```go
+
+type StatusMessage struct {
+    SymKey[]     []byte // [optional] The symmetric key used to encrypt the message
+    PublicKey    []byte // [optional] The public key to use for asymmetric encryption
+    Sig          string // [optional] The private key used to sign the message
+    PubsubTopic  string // The Pubsub topic to publish the message to     
+    ContentTopic string // The Content topic to publish the message to
+    Payload      []byte // A serialized representation of a Status message to be sent
+    Padding      []byte // Padding that must be applied to the Payload
+    TargetPeer   string // [optional] The target recipient of the message
+    Ephemeral    bool   // If the message is not to be stored, this is set to `true` 
+}
+
+```
+
+1. A user MUST only provide either a Symmetric key OR
+an Asymmetric keypair to encrypt the message.
+If both are received, the implementation MUST throw an error.
+2. `WakuMessage.Payload` MUST be set to `StatusMessage.Payload`
+3. `WakuMessage.Key` MUST be set to `StatusMessage.SymKey`
+4. `WakuMessage.Version` MUST be set to `1`
+5. `WakuMessage.Ephemeral` MUST be set to `StatusMessage.Ephemeral`
+6. `WakuMessage.ContentTopic` MUST be set to `StatusMessage.ContentTopic`
+7. `WakuMessage.Timestamp` MUST be set to the current Unix epoch timestamp
+(in nanosecond precision)
+
+### 2. `STORE`
+
+This protocol MUST remain optional according to the user's preferences,
+it MAY be enabled on Light clients as well.
+
+Messages received via [11/WAKU2-RELAY](/spec/11), are stored in a database.
+When Waku node running this protocol is service node,
+it MUST provide the complete list of network messages.
+Status clients SHOULD request historical messages from this service node.
+
+The messages that have the `WakuMessage.Ephemeral` flag set to true will not be stored.
+
+The Status client MAY provide a method to prune the database of
+older records to save storage.
+
+### 3. `FILTER`
+
+This protocol SHOULD be enabled on Light clients.
+
+This protocol SHOULD be used to filter messages based on a given criteria,
+such as the `Content Topic` of a `MESSAGE`.
+This allows a reduction in bandwidth consumption by the Status client.
+
+#### Content filtering protocol identifers
+
+The `filter-subcribe` SHOULD be implemented on `RELAY` nodes
+to provide `FILTER` services.
+
+`filter-subscribe`:
+
+> /vac/waku/filter-subscribe/2.0.0-beta1
+
+The `filter-push` SHOULD be implemented on light clients to receive messages.
+
+`filter-push`:
+
+> /vac/waku/filter-push/2.0.0-beta1
+
+Status clients SHOULD apply a filter for all the `Content Topic`
+they are interested in, such as `Content Topic` derived from -
+
+1. 1:1 chats with other users, described in [55/STATUS-1TO1-CHAT](/spec/55)
+2. Group chats
+3. Community Channels, described in [56/STATUS-COMMUNITIES](/spec/56)
+
+### 4. `LIGHTPUSH`
+
+The `LIGHTPUSH` protocol MUST be enabled on Status light clients.
+A Status `RELAY` node MAY implement `LIGHTPUSH` to support light clients.
+Peers will be able to publish messages,
+without running a full-fledged [11/WAKU2-RELAY](/spec/11) protocol.
+
+When a Status client is publishing a message,
+it MUST check if Light mode is enabled,
+and if so, it MUST publish the message via this protocol.
+
+### 5. Discovery
+
+A discovery method MUST be supported by Light clients and Full clients
+
+Status clients SHOULD make use of the following peer discovery methods
+that are provided by Waku, such as -
+
+1. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
+2. [33/WAKU2-DISCV5](/spec/33): A node discovery protocol to
+create decentralized network of interconnected Waku nodes.
+3. [34/WAKU2-PEER-EXCHANGE](/spec/34):
+A peer discovery protocol for resource restricted devices.
+
+Status clients MAY use any combination of the above peer discovery methods,
+which is suited best for their implementation.
+
+## Security/Privacy Considerations
+
+This specification inherits the security and
+privacy considerations from the following specifications -
+
+1. [10/WAKU2](/spec/10)
+2. [11/WAKU2-RELAY](/spec/11)
+3. [12/WAKU2-FILTER](/spec/12)
+4. [13/WAKU2-STORE](/spec/13)
+5. [14/WAKU2-MESSAGE](/spec/14)
+6. [23/WAKU2-TOPICS](/spec/23)
+7. [19/WAKU2-LIGHTPUSH](/spec/19)
+8. [55/STATUS-1TO1-CHAT](/spec/55)
+9. [56/STATUS-COMMUNITIES](/spec/56)
+10. [62/STATUS-PAYLOAD](/spec/62)
+11. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
+12. [33/WAKU2-DISCV5](/spec/33)
+13. [34/WAKU2-PEER-EXCHANGE](/spec/34)
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [55/STATUS-1TO1-CHAT](/spec/55)
+2. [56/STATUS-COMMUNITIES](/spec/56)
+3. [10/WAKU2](/spec/10)
+4. [11/WAKU2-RELAY](/spec/11)
+5. [12/WAKU2-FILTER](/spec/12)
+6. [13/WAKU2-STORE](/spec/13)
+7. [14/WAKU2-MESSAGE](/spec/14)
+8. [23/WAKU2-TOPICS](/spec/23)
+9. [19/WAKU2-LIGHTPUSH](/spec/19)
+10. [64/WAKU2-NETWORK](/spec/64)
+11. [62/STATUS-PAYLOAD](/spec/62)
+12. [EIP-1459: DNS-Based Discovery](https://eips.ethereum.org/EIPS/eip-1459)
+13. [33/WAKU2-DISCV5](/spec/33)
+14. [34/WAKU2-PEER-EXCHANGE](/spec/34)

vac/1/coss.md

@@ -1,228 +1,328 @@
----
-slug: 1
-title: 1/COSS
-name: Consensus-Oriented Specification System
-status: draft
-category: Best Current Practice
-editor: Oskar Thoren <oskarth@titanproxy.com>
-contributors:
-  - Pieter Hintjens <ph@imatix.com>
-  - André Rebentisch <andre@openstandards.de>
-  - Alberto Barrionuevo <abarrio@opentia.es>
-  - Chris Puttick <chris.puttick@thehumanjourney.net>
-  - Yurii Rashkovskii <yrashk@gmail.com>
-  - Daniel Kaiser <danielkaiser@status.im>
----
-
-This document describes a consensus-oriented specification system (COSS) for building interoperable technical specifications.
-COSS is based on a lightweight editorial process that seeks to engage the widest possible range of interested parties and move rapidly to consensus through working code.
-
-This specification is based on [Unprotocols 2/COSS](https://github.com/unprotocols/rfc/blob/master/2/README.md), used by the [ZeromMQ](https://rfc.zeromq.org/) project.
-It is equivalent except for some areas:
-
-- recommending the use of a permissive licenses, such as CC0 (with the exception of this document);
-- miscellaneous metadata, editor, and format/link updates;
-- more inheritance from the [IETF Standards Process][https://www.rfc-editor.org/rfc/rfc2026.txt],
-  e.g. using RFC categories: Standards Track, Informational, and Best Common Practice;
-- standards track specifications SHOULD follow a specific structure that both streamlines editing,
-  and helps implementers to quickly comprehend the specification
-- specifications MUST feature a header providing specific meta information
-
-## License
-
-Copyright (c) 2008-22 the Editor and Contributors.
-
-This Specification is free software;
-you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation;
-either version 3 of the License, or (at your option) any later version.
-
-This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
-without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-See the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License along with this program;
-if not, see http://www.gnu.org/licenses.
-
-## Change Process
-
-This document is governed by the [1/COSS](./coss.md) (COSS).
-
-## Language
-
-The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this document are to be interpreted as described in
-[RFC 2119](http://tools.ietf.org/html/rfc2119).
-
-## Goals
-
-The primary goal of COSS is to facilitate the process of writing, proving, and improving new technical specifications.
-A "technical specification" defines a protocol, a process, an API, a use of language, a methodology,
-or any other aspect of a technical environment that can usefully be documented for the purposes of technical or social interoperability.
-
-COSS is intended to above all be economical and rapid, so that it is useful to small teams with little time to spend on more formal processes.
-
-Principles:
-
-* We aim for rough consensus and running code; [inspired by the IETF Tao](https://www.ietf.org/about/participate/tao/).
-* Specifications are small pieces, made by small teams.
-* Specifications should have a clearly responsible editor.
-* The process should be visible, objective, and accessible to anyone.
-* The process should clearly separate experiments from solutions.
-* The process should allow deprecation of old specifications.
-
-Specifications should take minutes to explain, hours to design, days to write, weeks to prove, months to become mature, and years to replace.
-
-Specifications have no special status except that accorded by the community.
-
-## Architecture
-
-COSS is designed around fast, easy to use communications tools.
-Primarily, COSS uses a wiki model for editing and publishing specifications texts.
-
-* The *domain* is the conservancy for a set of specifications in a certain area.
-* Each domain is implemented as an Internet domain, hosting a wiki and optionally other communications tools.
-* Each specification is a set of wiki pages, together with comments, attached files, and other resources.
-* Important specifications may also exist as subdomains, i.e. child wikis.
-
-Individuals can become members of the domain by completing the necessary legal clearance.
-The copyright, patent, and trademark policies of the domain must be clarified in an Intellectual Property policy that applies to the domain.
-
-Specifications exist as multiple pages, one page per version of the specification (see "Branching and Merging", below), which may be assigned URIs that include an incremental number.
-Thus, we refer to a specification by specifying its domain, number, and short name.
-New versions of the same specification will have new numbers.
-The syntax for a specification reference is:
-
-    <domain>/spec/<number>/<shortname>
-
-For example, this specification is **rfc.vac.dev/spec/1/COSS**.
-The short form **1/COSS** may be used when referring to the specification from other specifications in the same domain.
-
-Every specification (including branches) carries a different number.
-
-## COSS Lifecycle
-
-Every specification has an independent lifecycle that documents clearly its current status.
-
-A specification has six possible states that reflect its maturity and contractual weight:
-
-![Lifecycle diagram](./images/lifecycle.png)
-
-### Raw Specifications
-
-All new specifications are **raw** specifications.
-Changes to raw specifications can be unilateral and arbitrary.
-Those seeking to implement a raw specification should ask for it to be made a draft specification.
-Raw specifications have no contractual weight.
-
-### Draft Specifications
-
-When raw specifications can be demonstrated, they become **draft** specifications.
-Changes to draft specifications should be done in consultation with users.
-Draft specifications are contracts between the editors and implementers.
-
-### Stable Specifications
-
-When draft specifications are used by third parties, they become **stable** specifications.
-Changes to stable specifications should be restricted to cosmetic ones, errata and clarifications.
-Stable specifications are contracts between editors, implementers, and end-users.
-
-### Deprecated Specifications
-
-When stable specifications are replaced by newer draft specifications, they become **deprecated** specifications.
-Deprecated specifications should not be changed except to indicate their replacements, if any.
-Deprecated specifications are contracts between editors, implementers and end-users.
-
-### Retired Specifications
-
-When deprecated specifications are no longer used in products, they become **retired** specifications.
-Retired specifications are part of the historical record.
-They should not be changed except to indicate their replacements, if any.
-Retired specifications have no contractual weight.
-
-### Deleted Specifications
-
-Deleted specifications are those that have not reached maturity (stable) and were discarded.
-They should not be used and are only kept for their historical value.
-Only Raw and Draft specifications can be deleted.
-
-## Editorial control
-
-A specification MUST have a single responsible editor,
-the only person who SHALL change the status of the specification through the lifecycle stages.
-
-A specification MAY also have additional contributors who contribute changes to it.
-It is RECOMMENDED to use a process similar to [C4 process](https://github.com/unprotocols/rfc/blob/master/1/README.md)
-to maximize the scale and diversity of contributions.
-
-Unlike the original C4 process however, it is RECOMMENDED to use CC0 as a more permissive license alternative.
-We SHOULD NOT use GPL or GPL-like license.
-One exception is this specification, as this was the original license for this specification.
-
-The editor is responsible for accurately maintaining the state of specifications and for handling all comments on the specification.
-
-## Branching and Merging
-
-Any member of the domain MAY branch a specification at any point.
-This is done by copying the existing text, and creating a new specification with the same name and content, but a new number.
-The ability to branch a specification is necessary in these circumstances:
-
-* To change the responsible editor for a specification, with or without the cooperation of the current responsible editor.
-* To rejuvenate a specification that is stable but needs functional changes.
-  This is the proper way to make a new version of a specification that is in stable or deprecated status.
-* To resolve disputes between different technical opinions.
-
-The responsible editor of a branched specification is the person who makes the branch.
-
-Branches, including added contributions, are derived works and thus licensed under the same terms as the original specification.
-This means that contributors are guaranteed the right to merge changes made in branches back into their original specifications.
-
-Technically speaking, a branch is a *different* specification, even if it carries the same name.
-Branches have no special status except that accorded by the community.
-
-## Conflict resolution
-
-COSS resolves natural conflicts between teams and vendors by allowing anyone to define a new specification.
-There is no editorial control process except that practised by the editor of a new specification.
-The administrators of a domain (moderators) may choose to interfere in editorial conflicts,
-and may suspend or ban individuals for behaviour they consider inappropriate.
-
-## Specification Structure
-
-### Meta Information
-
-Specifications MUST contain the following metadata.
-It is RECOMMENDED that specification metadata is specified as a YAML header (where possible).
-This will enable programmatic access to specification metadata.
-
-| Key              | Value                | Type   | Example                                                                                                                                                                                                                             |
-|------------------|----------------------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **shortname**    | short name           | string | 1/COSS                                                                                                                                                                                                                              |
-| **title**        | full name            | string | Consensus-Oriented Specification System                                                                                                                                                                                             |
-| **status**       | status               | string | draft                                                                                                                                                                                                                               |
-| **category**     | category             | string | Best Current Practice                                                                                                                                                                                                                            |
-| **tags**         | 0 or several tags    | list   | waku-application, waku-core-protocol                                                                                                                                                                                                |
-| **editor**       | editor name/email    | string | Oskar Thoren <oskarth@titanproxy.com>                                                                                                                                                                                                      |
-| **contributors** | contributors         | list   | - Pieter Hintjens <ph@imatix.com><br> - André Rebentisch <andre@openstandards.de><br> - Alberto Barrionuevo <abarrio@opentia.es><br> - Chris Puttick <chris.puttick@thehumanjourney.net><br> - Yurii Rashkovskii <yrashk@gmail.com> |
-
-### Specification Template
-
-Standards Track specifications SHOULD be based on the [Vac RFC template](./images/template.md).
-
-## Conventions
-
-Where possible editors and contributors are encouraged to:
-
-* Refer to and build on existing work when possible, especially IETF specifications.
-* Contribute to existing specifications rather than reinvent their own.
-* Use collaborative branching and merging as a tool for experimentation.
-* Use Semantic Line Breaks: https://sembr.org/.
-
-## Appendix A. Color Coding
-
-It is RECOMMENDED to use color coding to indicate specification's status. Color coded specifications SHOULD use the following color scheme:
-
-* ![raw](https://raw.githubusercontent.com/unprotocols/rfc/master/2/raw.svg)
-* ![draft](https://raw.githubusercontent.com/unprotocols/rfc/master/2/draft.svg)
-* ![stable](https://raw.githubusercontent.com/unprotocols/rfc/master/2/stable.svg)
-* ![deprecated](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deprecated.svg)
-* ![retired](https://raw.githubusercontent.com/unprotocols/rfc/master/2/retired.svg)
-* ![deleted](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deleted.svg)
+---
+slug: 1
+title: 1/COSS
+name: Consensus-Oriented Specification System
+status: draft
+category: Best Current Practice
+editor: Daniel Kaiser <danielkaiser@status.im>
+contributors:
+  - Oskar Thoren <oskarth@titanproxy.com>
+  - Pieter Hintjens <ph@imatix.com>
+  - André Rebentisch <andre@openstandards.de>
+  - Alberto Barrionuevo <abarrio@opentia.es>
+  - Chris Puttick <chris.puttick@thehumanjourney.net>
+  - Yurii Rashkovskii <yrashk@gmail.com>
+  - Jimmy Debe <jimmy@status.im>
+---
+
+This document describes a consensus-oriented specification system (COSS)
+for building interoperable technical specifications.
+COSS is based on a lightweight editorial process that
+seeks to engage the widest possible range of interested parties and
+move rapidly to consensus through working code.
+
+This specification is based on [Unprotocols 2/COSS](https://github.com/unprotocols/rfc/blob/master/2/README.md),
+used by the [ZeromMQ](https://rfc.zeromq.org/) project.
+It is equivalent except for some areas:
+
+- recommending the use of a permissive licenses,
+such as CC0 (with the exception of this document);
+- miscellaneous metadata, editor, and format/link updates;
+- more inheritance from the [IETF Standards Process](https://www.rfc-editor.org/rfc/rfc2026.txt),
+  e.g. using RFC categories: Standards Track, Informational, and Best Common Practice;
+- standards track specifications SHOULD
+follow a specific structure that both streamlines editing,
+and helps implementers to quickly comprehend the specification
+- specifications MUST feature a header providing specific meta information
+- raw specifications will not be assigned numbers
+- section explaining the [IFT](https://free.technology/)
+Request For Comments specification process managed by the Vac service department
+
+## License
+
+Copyright (c) 2008-24 the Editor and Contributors.
+
+This Specification is free software;
+you can redistribute it and/or
+modify it under the terms of the GNU General Public License
+as published by the Free Software Foundation;
+either version 3 of the License, or (at your option) any later version.
+
+This specification is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY;
+without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.
+See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program;
+if not, see [gnu.org](http://www.gnu.org/licenses).
+
+## Change Process
+
+This document is governed by the [1/COSS](./coss.md) (COSS).
+
+## Language
+
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
+"SHOULD NOT", "RECOMMENDED",  "MAY", and
+"OPTIONAL" in this document are to be interpreted as described in
+[RFC 2119](http://tools.ietf.org/html/rfc2119).
+
+## Goals
+
+The primary goal of COSS is to facilitate the process of writing, proving, and
+improving new technical specifications.
+A "technical specification" defines a protocol, a process, an API, a use of language,
+a methodology, or any other aspect of a technical environment that
+can usefully be documented for the purposes of technical or social interoperability.
+
+COSS is intended to above all be economical and rapid,
+so that it is useful to small teams with little time to spend on more formal processes.
+
+Principles:
+
+- We aim for rough consensus and running code; [inspired by the IETF Tao](https://www.ietf.org/about/participate/tao/).
+- Specifications are small pieces, made by small teams.
+- Specifications should have a clearly responsible editor.
+- The process should be visible, objective, and accessible to anyone.
+- The process should clearly separate experiments from solutions.
+- The process should allow deprecation of old specifications.
+
+Specifications should take minutes to explain, hours to design, days to write,
+weeks to prove, months to become mature, and years to replace.
+Specifications have no special status except that accorded by the community.
+
+## Architecture
+
+COSS is designed around fast, easy to use communications tools.
+Primarily, COSS uses a wiki model for editing and publishing specifications texts.
+
+- The *domain* is the conservancy for a set of specifications.
+- The *domain* is implemented as an Internet domain.
+- Each specification is a document together with references and attached resources.
+- A *sub-domain* is a initiative under a specific domain.
+
+Individuals can become members of the *domain*
+by completing the necessary legal clearance.
+The copyright, patent, and trademark policies of the domain must be clarified
+in an Intellectual Property policy that applies to the domain.
+
+Specifications exist as multiple pages, one page per version,
+(discussed below in "Branching and Merging"),
+which should be assigned URIs that MAY include an number identifier.
+
+Thus, we refer to new specifications by specifying its domain,
+its sub-domain and short name.
+The syntax for a new specification reference is:
+
+    <domain>/<sub-domain>/<shortname>
+
+For example, this specification should be **rfc.vac.dev/vac/COSS**,
+if the status were **raw**.
+
+A number will be assigned to the specification when obtaining **draft** status.
+New versions of the same specification will be assigned a new number.
+The syntax for a specification reference is:
+
+    <domain>/<sub-domain>/<number>/<shortname>
+
+For example, this specification is **rfc.vac.dev/vac/1/COSS**.
+The short form **1/COSS** may be used when referring to the specification
+from other specifications in the same domain.
+
+Specifications (excluding raw specifications)
+carries a different number including branches.
+
+## COSS Lifecycle
+
+Every specification has an independent lifecycle that
+documents clearly its current status.
+For a specification to receive a lifecycle status,
+a new specification SHOULD be presented by the team of the sub-domain.
+After discussion amongst the contributors has reached a rough consensus,
+as described in [RFC7282](https://www.rfc-editor.org/rfc/rfc7282.html),
+the specification MAY begin the process to upgrade it's status.
+
+A specification has five possible states that reflect its maturity and
+contractual weight:
+
+![Lifecycle diagram](./images/lifecycle.png)
+
+### Raw Specifications
+
+All new specifications are **raw** specifications.
+Changes to raw specifications can be unilateral and arbitrary.
+A sub-domain MAY use the **raw** status for new specifications
+that live under their domain.
+Raw specifications have no contractual weight.
+
+### Draft Specifications
+
+When raw specifications can be demonstrated,
+they become **draft** specifications and are assigned numbers.
+Changes to draft specifications should be done in consultation with users.
+Draft specifications are contracts between the editors and implementers.
+
+### Stable Specifications
+
+When draft specifications are used by third parties, they become **stable** specifications.
+Changes to stable specifications should be restricted to cosmetic ones,
+errata and clarifications.
+Stable specifications are contracts between editors, implementers, and end-users.
+
+### Deprecated Specifications
+
+When stable specifications are replaced by newer draft specifications,
+they become **deprecated** specifications.
+Deprecated specifications should not be changed except
+to indicate their replacements, if any.
+Deprecated specifications are contracts between editors, implementers and end-users.
+
+### Retired Specifications
+
+When deprecated specifications are no longer used in products,
+they become **retired** specifications.
+Retired specifications are part of the historical record.
+They should not be changed except to indicate their replacements, if any.
+Retired specifications have no contractual weight.
+
+### Deleted Specifications
+
+Deleted specifications are those that have not reached maturity (stable) and
+were discarded.
+They should not be used and are only kept for their historical value.
+Only Raw and Draft specifications can be deleted.
+
+## Editorial control
+
+A specification MUST have a single responsible editor,
+the only person who SHALL change the status of the specification
+through the lifecycle stages.
+
+A specification MAY also have additional contributors who contribute changes to it.
+It is RECOMMENDED to use a process similar to [C4 process](https://github.com/unprotocols/rfc/blob/master/1/README.md)
+to maximize the scale and diversity of contributions.
+
+Unlike the original C4 process however,
+it is RECOMMENDED to use CC0 as a more permissive license alternative.
+We SHOULD NOT use GPL or GPL-like license.
+One exception is this specification, as this was the original license for this specification.
+
+The editor is responsible for accurately maintaining the state of specifications,
+for retiring different versions that may live in other places and
+for handling all comments on the specification.
+
+## Branching and Merging
+
+Any member of the domain MAY branch a specification at any point.
+This is done by copying the existing text, and
+creating a new specification with the same name and content, but a new number.
+Since **raw** specifications are not assigned a number,
+branching by any member of a sub-domain MAY differentiate specifications
+based on date, contributors, or
+version number within the document.
+The ability to branch a specification is necessary in these circumstances:
+
+- To change the responsible editor for a specification,
+with or without the cooperation of the current responsible editor.
+- To rejuvenate a specification that is stable but needs functional changes.
+This is the proper way to make a new version of a specification
+that is in stable or deprecated status.
+- To resolve disputes between different technical opinions.
+
+The responsible editor of a branched specification is the person who makes the branch.
+
+Branches, including added contributions, are derived works and
+thus licensed under the same terms as the original specification.
+This means that contributors are guaranteed the right to merge changes made in branches
+back into their original specifications.
+
+Technically speaking, a branch is a *different* specification,
+even if it carries the same name.
+Branches have no special status except that accorded by the community.
+
+## Conflict resolution
+
+COSS resolves natural conflicts between teams and
+vendors by allowing anyone to define a new specification.
+There is no editorial control process except
+that practised by the editor of a new specification.
+The administrators of a domain (moderators)
+may choose to interfere in editorial conflicts,
+and may suspend or ban individuals for behaviour they consider inappropriate.
+
+## Specification Structure
+
+### Meta Information
+
+Specifications MUST contain the following metadata.
+It is RECOMMENDED that specification metadata is specified as a YAML header
+(where possible).
+This will enable programmatic access to specification metadata.
+
+| Key              | Value                | Type   | Example                                                                                                                                                                                                                             |
+|------------------|----------------------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **shortname**    | short name           | string | 1/COSS                                                                                                                                                                                                                              |
+| **title**        | full name            | string | Consensus-Oriented Specification System                                                                                                                                                                                             |
+| **status**       | status               | string | draft                                                                                                                                                                                                                               |
+| **category**     | category             | string | Best Current Practice                                                                                                                                                                                                                            |
+| **tags**         | 0 or several tags    | list   | waku-application, waku-core-protocol                                                                                                                                                                                                |
+| **editor**       | editor name/email    | string | Oskar Thoren <oskarth@titanproxy.com>                                                                                                                                                                                                      |
+| **contributors** | contributors         | list   | - Pieter Hintjens <ph@imatix.com> - André Rebentisch <andre@openstandards.de> - Alberto Barrionuevo <abarrio@opentia.es> - Chris Puttick <chris.puttick@thehumanjourney.net> - Yurii Rashkovskii <yrashk@gmail.com> |
+
+### IFT/Vac RFC Process
+
+> [!Note]
+This section is introduced to allow contributors to understand the IFT
+(Institute of Free Technology) Vac RFC specification process.
+Other organizations may make changes to this section according to their needs.
+
+Vac is a department under the IFT organization that provides RFC (Request For Comments)
+specification services.
+This service works to help facilitate the RFC process, assuring standards are followed.
+Contributors within the service SHOULD assist a *sub-domain* in creating a new specification,
+editing a specification, and
+promoting the status of a specification along with other tasks.
+Once a specification reaches some level of maturity by rough consensus,
+the specification SHOULD enter the [Vac RFC](rfc.vac.dev) process.
+Similar to the IETF working group adoption described in [RFC6174](https://www.rfc-editor.org/rfc/rfc6174.html),
+the Vac RFC process SHOULD facilitate all updates to the specification.
+
+Specifications are introduced by projects,
+under a specific *domain*, with the intention of becoming technically mature documents.
+The IFT domain currently houses the following projects:
+
+- [Status](status.app)
+- [Waku](https://waku.org/)
+- [Codex](https://codex.storage/)
+- [Nimbus](https://nimbus.team/)
+- [Nomos](https://nomos.tech/)
+
+When a specification is promoted to *draft* status,
+the number that is assigned MAY be incremental
+or by the *sub-domain* and the Vac RFC process.
+Standards track specifications MUST be based on the
+[Vac RFC template](../template.md) before obtaining a new status.
+All changes, comments, and contributions SHOULD be documented.
+
+## Conventions
+
+Where possible editors and contributors are encouraged to:
+
+- Refer to and build on existing work when possible, especially IETF specifications.
+- Contribute to existing specifications rather than reinvent their own.
+- Use collaborative branching and merging as a tool for experimentation.
+- Use Semantic Line Breaks: [sembr](https://sembr.org/).
+
+## Appendix A. Color Coding
+
+It is RECOMMENDED to use color coding to indicate specification's status.
+Color coded specifications SHOULD use the following color scheme:
+
+- ![raw](https://raw.githubusercontent.com/unprotocols/rfc/master/2/raw.svg)
+- ![draft](https://raw.githubusercontent.com/unprotocols/rfc/master/2/draft.svg)
+- ![stable](https://raw.githubusercontent.com/unprotocols/rfc/master/2/stable.svg)
+- ![deprecated](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deprecated.svg)
+- ![retired](https://raw.githubusercontent.com/unprotocols/rfc/master/2/retired.svg)
+- ![deleted](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deleted.svg)

vac/2/images/batch.msc

@@ -0,0 +1,14 @@
+# Alice and Bob: batch data sync
+msc {
+  hscale="2", wordwraparcs=on;
+
+  alice [label="Alice"],
+  bob [label="Bob"];
+
+  --- [label="batch data sync"];
+  alice => alice [label="add messages to payload state"];
+  alice >> bob [label="send payload with messages"];
+
+  bob => bob [label="add acks to payload state"];
+  bob >> alice [label="send payload with acks"];
+}

vac/2/images/interactive.msc

@@ -0,0 +1,20 @@
+# Alice and Bob: interactive data sync
+msc {
+  hscale="2", wordwraparcs=on;
+
+  alice [label="Alice"],
+  bob [label="Bob"];
+
+  --- [label="interactive data sync"];
+  alice => alice [label="add offers to payload state"];
+  alice >> bob [label="send payload with offers"];
+    
+  bob => bob [label="add requests to payload state"];
+  bob >> alice [label="send payload with requests"];
+  
+  alice => alice [label="add requested messages to state"];
+  alice >> bob [label="send payload with messages"];
+  
+  bob => bob [label="add acks to payload state"];
+  bob >> alice [label="send payload with acks"];
+}

vac/2/mvds.md

@@ -1,150 +1,194 @@
----
-slug: 2
-title: 2/MVDS
-name: Minimum Viable Data Synchronization
-status: stable
-editor: Sanaz Taheri <sanaz@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
----
-
-In this specification, we describe a minimum viable protocol for data synchronization inspired by the Bramble Synchronization Protocol[^1]. This protocol is designed to ensure reliable messaging between peers across an unreliable peer-to-peer (P2P) network where they may be unreachable or unresponsive.
-
-We present a reference implementation[^2] including a simulation to demonstrate its performance.
-
-## Definitions
-
-| Term       | Description                                                                         |
-|------------|-------------------------------------------------------------------------------------|
-| **Peer**   | The other nodes that a node is connected to.                                        |
-| **Record** | Defines a payload element of either the type `OFFER`, `REQUEST`, `MESSAGE` or `ACK` |
-| **Node**   | Some process that is able to store data, do processing and communicate for MVDS.    |
-
-## Wire Protocol
-
-### Secure Transport
-
-This specification does not define anything related to the transport of packets. It is assumed that this is abstracted in such a way that any secure transport protocol could be easily implemented. Likewise, properties such as confidentiality, integrity, authenticity and forward secrecy are assumed to be provided by a layer below.
-
-### Payloads
-
-Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
-
-```protobuf
-syntax = "proto3";
-
-package vac.mvds;
-
-message Payload {
-  repeated bytes acks = 5001;
-  repeated bytes offers = 5002;
-  repeated bytes requests = 5003;
-  repeated Message messages = 5004;
-}
-
-message Message {
-  bytes group_id = 6001;
-  int64 timestamp = 6002;
-  bytes body = 6003;
-}
-```
-
-*The payload field numbers are kept more "unique" to ensure no overlap with other protocol buffers.*
-
-Each payload contains the following fields:
-
-- **Acks:** This field contains a list (can be empty) of `message identifiers` informing the recipient that sender holds a specific message.
-- **Offers:** This field contains a list (can be empty) of `message identifiers` that the sender would like to give to the recipient.
-- **Requests:** This field contains a list (can be empty) of `message identifiers` that the sender would like to receive from the recipient.
-- **Messages:** This field contains a list of messages (can be empty).
-
-**Message Identifiers:** Each `message` has a message identifier calculated by hashing the `group_id`, `timestamp` and `body` fields as follows:
-
-```
-HASH("MESSAGE_ID", group_id, timestamp, body);
-```
-
-**Group Identifiers:** Each `message` is assigned into a **group** using the `group_id` field, groups are independent synchronization contexts between peers.
-
-The current `HASH` function used is `sha256`.
-
-## Synchronization
-
-### State
-
-We refer to `state` as set of records for the types `OFFER`, `REQUEST` and `MESSAGE` that every node SHOULD store per peer. `state` MUST NOT contain `ACK` records as we do not retransmit those periodically. The following information is stored for records:
-
- - **Type** - Either `OFFER`, `REQUEST` or `MESSAGE`
- - **Send Count** - The amount of times a record has been sent to a peer.
- - **Send Epoch** - The next epoch at which a record can be sent to a peer.
-
-### Flow
-
-A maximum of one payload SHOULD be sent to peers per epoch, this payload contains all `ACK`, `OFFER`, `REQUEST` and `MESSAGE` records for the specific peer. Payloads are created every epoch, containing reactions to previously received records by peers or new records being sent out by nodes. 
-
-Nodes MAY have two modes with which they can send records: `BATCH` and `INTERACTIVE` mode. The following rules dictate how nodes construct payloads every epoch for any given peer for both modes.
-
-> ***NOTE:** A node may send messages both in interactive and in batch mode.*
-
-#### Interactive Mode
-
- - A node initially offers a `MESSAGE` when attempting to send it to a peer. This means an `OFFER` is added to the next payload and state for the given peer.
- - When a node receives an `OFFER`, a `REQUEST` is added to the next payload and state for the given peer. 
- - When a node receives a `REQUEST` for a previously sent `OFFER`, the `OFFER` is removed from the state and the corresponding `MESSAGE` is added to the next payload and state for the given peer.
- - When a node receives a `MESSAGE`, the `REQUEST` is removed from the state and an `ACK` is added to the next payload for the given peer.
- - When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
- - All records that require retransmission are added to the payload, given `Send Epoch` has been reached.
-
-<p align="center">
-    <img src="../assets/mvds/interactive.png" />
-    <br />
-    Figure 1: Delivery without retransmissions in interactive mode.
-</p>
-
-#### Batch Mode
-
- 1. When a node sends a `MESSAGE`, it is added to the next payload and the state for the given peer.
- 2. When a node receives a `MESSAGE`, an `ACK` is added to the next payload for the corresponding peer.
- 3. When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
- 4. All records that require retransmission are added to the payload, given `Send Epoch` has been reached.
-
-<!-- diagram -->
-
-<p align="center">
-    <img src="../assets/mvds/batch.png" />
-    <br />
-    Figure 2: Delivery without retransmissions in batch mode.
-</p>
-
-
-> ***NOTE:** Batch mode is higher bandwidth whereas interactive mode is higher latency.*
-
-<!-- Interactions with state, flow chart with retransmissions? -->
-
-### Retransmission
-
-The record of the type `Type` SHOULD be retransmitted every time `Send Epoch` is smaller than or equal to the current epoch.
-
-`Send Epoch` and `Send Count` MUST be increased every time a record is retransmitted. Although no function is defined on how to increase `Send Epoch`, it SHOULD be exponentially increased until reaching an upper bound where it then goes back to a lower epoch in order to prevent a record's `Send Epoch`'s from becoming too large.
-
-> ***NOTE:** We do not retransmission `ACK`s as we do not know when they have arrived, therefore we simply resend them every time we receive a `MESSAGE`.*
-
-## Formal Specification
-
-MVDS has been formally specified using TLA+: <https://github.com/vacp2p/formalities/tree/master/MVDS>.
-
-## Acknowledgments
- - Preston van Loon
- - Greg Markou
- - Rene Nayman
- - Jacek Sieka
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## Footnotes
-
-[^1]: akwizgran et al. [BSP](https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BSP.md). Briar.
-[^2]: <https://github.com/vacp2p/mvds>
+---
+slug: 2
+title: 2/MVDS
+name: Minimum Viable Data Synchronization
+status: stable
+editor: Sanaz Taheri <sanaz@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+In this specification, we describe a minimum viable protocol for
+data synchronization inspired by the Bramble Synchronization Protocol[^1].
+This protocol is designed to ensure reliable messaging
+between peers across an unreliable peer-to-peer (P2P) network where
+they may be unreachable or unresponsive.
+
+We present a reference implementation[^2]
+including a simulation to demonstrate its performance.
+
+## Definitions
+
+| Term       | Description                                                                         |
+|------------|-------------------------------------------------------------------------------------|
+| **Peer**   | The other nodes that a node is connected to.                                        |
+| **Record** | Defines a payload element of either the type `OFFER`, `REQUEST`, `MESSAGE` or `ACK` |
+| **Node**   | Some process that is able to store data, do processing and communicate for MVDS.    |
+
+## Wire Protocol
+
+### Secure Transport
+
+This specification does not define anything related to the transport of packets.
+It is assumed that this is abstracted in such a way that
+any secure transport protocol could be easily implemented.
+Likewise, properties such as confidentiality, integrity, authenticity and
+forward secrecy are assumed to be provided by a layer below.
+
+### Payloads
+
+Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
+
+```protobuf
+syntax = "proto3";
+
+package vac.mvds;
+
+message Payload {
+  repeated bytes acks = 5001;
+  repeated bytes offers = 5002;
+  repeated bytes requests = 5003;
+  repeated Message messages = 5004;
+}
+
+message Message {
+  bytes group_id = 6001;
+  int64 timestamp = 6002;
+  bytes body = 6003;
+}
+```
+
+*The payload field numbers are kept more "unique" to*
+*ensure no overlap with other protocol buffers.*
+
+Each payload contains the following fields:
+
+- **Acks:** This field contains a list (can be empty)
+of `message identifiers` informing the recipient that sender holds a specific message.
+- **Offers:** This field contains a list (can be empty)
+of `message identifiers` that the sender would like to give to the recipient.
+- **Requests:** This field contains a list (can be empty)
+of `message identifiers` that the sender would like to receive from the recipient.
+- **Messages:** This field contains a list of messages (can be empty).
+
+**Message Identifiers:** Each `message` has a message identifier calculated by
+hashing the `group_id`, `timestamp` and `body` fields as follows:
+
+```js
+HASH("MESSAGE_ID", group_id, timestamp, body);
+```
+
+**Group Identifiers:** Each `message` is assigned into a **group**
+using the `group_id` field,
+groups are independent synchronization contexts between peers.
+
+The current `HASH` function used is `sha256`.
+
+## Synchronization
+
+### State
+
+We refer to `state` as set of records for the types `OFFER`, `REQUEST` and
+`MESSAGE` that every node SHOULD store per peer.
+`state` MUST NOT contain `ACK` records as we do not retransmit those periodically.
+The following information is stored for records:
+
+- **Type** - Either `OFFER`, `REQUEST` or `MESSAGE`
+- **Send Count** - The amount of times a record has been sent to a peer.
+- **Send Epoch** - The next epoch at which a record can be sent to a peer.
+
+### Flow
+
+A maximum of one payload SHOULD be sent to peers per epoch,
+this payload contains all `ACK`, `OFFER`, `REQUEST` and
+`MESSAGE` records for the specific peer.
+Payloads are created every epoch,
+containing reactions to previously received records by peers or
+new records being sent out by nodes.
+
+Nodes MAY have two modes with which they can send records:
+`BATCH` and `INTERACTIVE` mode.
+The following rules dictate how nodes construct payloads
+every epoch for any given peer for both modes.
+
+> ***NOTE:** A node may send messages both in interactive and in batch mode.*
+
+#### Interactive Mode
+
+- A node initially offers a `MESSAGE` when attempting to send it to a peer.
+This means an `OFFER` is added to the next payload and state for the given peer.
+- When a node receives an `OFFER`, a `REQUEST` is added to the next payload and
+state for the given peer.
+- When a node receives a `REQUEST` for a previously sent `OFFER`,
+the `OFFER` is removed from the state and
+the corresponding `MESSAGE` is added to the next payload and
+state for the given peer.
+- When a node receives a `MESSAGE`, the `REQUEST` is removed from the state and
+an `ACK` is added to the next payload for the given peer.
+- When a node receives an `ACK`,
+the `MESSAGE` is removed from the state for the given peer.
+- All records that require retransmission are added to the payload,
+given `Send Epoch` has been reached.
+
+<p align="center">
+    <img src="./images/interactive.png" />
+    <br />
+    Figure 1: Delivery without retransmissions in interactive mode.
+</p>
+
+#### Batch Mode
+
+1. When a node sends a `MESSAGE`,
+it is added to the next payload and the state for the given peer.
+2. When a node receives a `MESSAGE`,
+an `ACK` is added to the next payload for the corresponding peer.
+3. When a node receives an `ACK`,
+the `MESSAGE` is removed from the state for the given peer.
+4. All records that require retransmission are added to the payload,
+given `Send Epoch` has been reached.
+
+<!-- diagram -->
+
+<p align="center">
+    <img src="./images/batch.png" />
+    <br />
+    Figure 2: Delivery without retransmissions in batch mode.
+</p>
+
+> ***NOTE:** Batch mode is higher bandwidth whereas interactive mode is higher latency.*
+
+<!-- Interactions with state, flow chart with retransmissions? -->
+
+### Retransmission
+
+The record of the type `Type` SHOULD be retransmitted
+every time `Send Epoch` is smaller than or equal to the current epoch.
+
+`Send Epoch` and `Send Count` MUST be increased every time a record is retransmitted.
+Although no function is defined on how to increase `Send Epoch`,
+it SHOULD be exponentially increased until reaching an upper bound
+where it then goes back to a lower epoch in order to
+prevent a record's `Send Epoch`'s from becoming too large.
+
+> ***NOTE:** We do not retransmission `ACK`s as we do not know when they have arrived, therefore we simply resend them every time we receive a `MESSAGE`.*
+
+## Formal Specification
+
+MVDS has been formally specified using TLA+: <https://github.com/vacp2p/formalities/tree/master/MVDS>.
+
+## Acknowledgments
+
+- Preston van Loon
+- Greg Markou
+- Rene Nayman
+- Jacek Sieka
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## Footnotes
+
+[^1]: akwizgran et al. [BSP](https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BSP.md). Briar.
+[^2]: <https://github.com/vacp2p/mvds>

vac/25/libp2p-dns-discovery.md

@@ -1,134 +1,153 @@
----
-slug: 25
-title: 25/LIBP2P-DNS-DISCOVERY
-name: Libp2p Peer Discovery via DNS
-status: deleted
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
----
-
-`25/LIBP2P-DNS-DISCOVERY` specifies a scheme to implement [`libp2p`](https://libp2p.io/) peer discovery via DNS for Waku v2.
-The generalised purpose is to retrieve an arbitrarily long, authenticated, updateable list of [`libp2p` peers](https://docs.libp2p.io/concepts/peer-id/) to bootstrap connection to a `libp2p` network.
-Since [`10/WAKU2`](https://rfc.vac.dev/spec/10/) currently specifies use of [`libp2p` peer identities](https://docs.libp2p.io/concepts/peer-id/),
-this method is suitable for a new Waku v2 node to discover other Waku v2 nodes to connect to.
-
-This specification is largely based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459),
-with the only deviation being the type of address being encoded (`multiaddr` vs `enr`).
-Also see [this earlier explainer](https://vac.dev/dns-based-discovery) for more background on the suitability of DNS based discovery for Waku v2.
-
-# List encoding
-
-The peer list MUST be encoded as a [Merkle tree](https://www.wikiwand.com/en/Merkle_tree).
-EIP-1459 specifies [the URL scheme](https://eips.ethereum.org/EIPS/eip-1459#specification) to refer to such a DNS node list.
-This specification uses the same approach, but with a `matree` scheme:
-
-```
-matree://<key>@<fqdn>
-```
-
-where
-- `matree` is the selected `multiaddr` Merkle tree scheme
-- `<fqdn>` is the fully qualified domain name on which the list can be found
-- `<key>` is the base32 encoding of the compressed 32-byte binary public key that signed the list.
-
-The example URL from EIP-1459, adapted to the above scheme becomes:
-
-```
-matree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@peers.example.org
-```
-
-Each entry within the Merkle tree MUST be contained within a [DNS TXT record](https://www.rfc-editor.org/rfc/rfc1035.txt)
-and stored in a subdomain (except for the base URL `matree` entry).
-The content of any TXT record MUST be small enough to fit into the 512-byte limit imposed on UDP DNS packets,
-which limits the number of hashes that can be contained within a branch entry.
-The subdomain name for each entry is the base32 encoding of the abbreviated keccak256 hash of its text content.
-See [this example](https://eips.ethereum.org/EIPS/eip-1459#dns-record-structure) of a fully populated tree for more information.
-
-# Entry types
-
-The following entry types are derived from [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
-and adapted for use with `multiaddrs`:
-
-## Root entry
-
-The tree root entry MUST use the following format:
-
-```
-matree-root:v1 m=<ma-root> l=<link-root> seq=<sequence number> sig=<signature>
-```
-
-where
-- `ma-root` and `link-root` refer to the root hashes of subtrees
-containing `multiaddrs` and links to other subtrees, respectively
-- `sequence-number` is the tree's update sequence number.
-This number SHOULD increase with each update to the tree.
-- `signature` is a 65-byte secp256k1 EC signature
-over the keccak256 hash of the root record content,
-excluding the `sig=` part,
-encoded as URL-safe base64
-
-## Branch entry
-
-Branch entries MUST take the format:
-
-```
-matree-branch:<h₁>,<h₂>,...,<hₙ>
-```
-
-where
-- `<h₁>,<h₂>,...,<hₙ>` are the hashes of other subtree entries
-
-## Leaf entries
-
-There are two types of leaf entries:
-
-### Link entries
-
-For the subtree pointed to by `link-root`,
-leaf entries MUST take the format:
-
-```
-matree://<key>@<fqdn>
-```
-
-which links to a different list located in another domain.
-
-### `multiaddr` entries
-
-For the subtree pointed to by `ma-root`,
-leaf entries MUST take the format:
-
-```
-ma:<multiaddr>
-```
-
-which contains the `multiaddr` of a `libp2p` peer.
-
-# Client protocol
-
-A client MUST adhere to the [client protocol](https://eips.ethereum.org/EIPS/eip-1459#client-protocol) as specified in EIP-1459,
-and adapted for usage with `multiaddr` entry types below:
-
-To find nodes at a given DNS name a client MUST perform the following steps:
-1. Resolve the TXT record of the DNS name and check whether it contains a valid `matree-root:v1` entry.
-2. Verify the signature on the root against the known public key
-and check whether the sequence number is larger than or equal to any previous number seen for that name.
-3. Resolve the TXT record of a hash subdomain indicated in the record
-and verify that the content matches the hash.
-4. If the resolved entry is of type:
-	- `matree-branch`: parse the list of hashes and continue resolving them (step 3).
-	- `ma`: import the `multiaddr` and add it to a local list of discovered nodes.
-
-# Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-# References
-
-1. [`10/WAKU2`](https://rfc.vac.dev/spec/10/)
-1. [EIP-1459: Client Protocol](https://eips.ethereum.org/EIPS/eip-1459#client-protocol)
-1. [EIP-1459: Node Discovery via DNS ](https://eips.ethereum.org/EIPS/eip-1459)
-1. [`libp2p`](https://libp2p.io/)
-1. [`libp2p` peer identity](https://docs.libp2p.io/concepts/peer-id/)
-1. [Merkle trees](https://www.wikiwand.com/en/Merkle_tree)
+---
+slug: 25
+title: 25/LIBP2P-DNS-DISCOVERY
+name: Libp2p Peer Discovery via DNS
+status: deleted
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+---
+
+`25/LIBP2P-DNS-DISCOVERY` specifies a scheme to implement [`libp2p`](https://libp2p.io/)
+peer discovery via DNS for Waku v2.
+The generalised purpose is to retrieve an arbitrarily long, authenticated,
+updateable list of [`libp2p` peers](https://docs.libp2p.io/concepts/peer-id/)
+to bootstrap connection to a `libp2p` network.
+Since [`10/WAKU2`](../../waku/standards/core/10/waku2.md)
+currently specifies use of [`libp2p` peer identities](https://docs.libp2p.io/concepts/peer-id/),
+this method is suitable for a new Waku v2 node
+to discover other Waku v2 nodes to connect to.
+
+This specification is largely based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459),
+with the only deviation being the type of address being encoded (`multiaddr` vs `enr`).
+Also see [this earlier explainer](https://vac.dev/dns-based-discovery)
+for more background on the suitability of DNS based discovery for Waku v2.
+
+## List encoding
+
+The peer list MUST be encoded as a [Merkle tree](https://www.wikiwand.com/en/Merkle_tree).
+EIP-1459 specifies [the URL scheme](https://eips.ethereum.org/EIPS/eip-1459#specification)
+to refer to such a DNS node list.
+This specification uses the same approach, but with a `matree` scheme:
+
+```yaml
+matree://<key>@<fqdn>
+```
+
+where
+
+- `matree` is the selected `multiaddr` Merkle tree scheme
+- `<fqdn>` is the fully qualified domain name on which the list can be found
+- `<key>` is the base32 encoding of the compressed 32-byte binary public key
+that signed the list.
+
+The example URL from EIP-1459, adapted to the above scheme becomes:
+
+```yaml
+matree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@peers.example.org
+```
+
+Each entry within the Merkle tree MUST be contained within a [DNS TXT record](https://www.rfc-editor.org/rfc/rfc1035.txt)
+and stored in a subdomain (except for the base URL `matree` entry).
+The content of any TXT record
+MUST be small enough to fit into the 512-byte limit imposed on UDP DNS packets,
+which limits the number of hashes that can be contained within a branch entry.
+The subdomain name for each entry
+is the base32 encoding of the abbreviated keccak256 hash of its text content.
+See [this example](https://eips.ethereum.org/EIPS/eip-1459#dns-record-structure)
+of a fully populated tree for more information.
+
+## Entry types
+
+The following entry types are derived from [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
+and adapted for use with `multiaddrs`:
+
+## Root entry
+
+The tree root entry MUST use the following format:
+
+```yaml
+matree-root:v1 m=<ma-root> l=<link-root> seq=<sequence number> sig=<signature>
+```
+
+where
+
+- `ma-root` and `link-root` refer to the root hashes of subtrees
+containing `multiaddrs` and links to other subtrees, respectively
+- `sequence-number` is the tree's update sequence number.
+This number SHOULD increase with each update to the tree.
+- `signature` is a 65-byte secp256k1 EC signature
+over the keccak256 hash of the root record content,
+excluding the `sig=` part,
+encoded as URL-safe base64
+
+## Branch entry
+
+Branch entries MUST take the format:
+
+```yaml
+matree-branch:<h₁>,<h₂>,...,<hₙ>
+```
+
+where
+
+- `<h₁>,<h₂>,...,<hₙ>` are the hashes of other subtree entries
+
+## Leaf entries
+
+There are two types of leaf entries:
+
+### Link entries
+
+For the subtree pointed to by `link-root`,
+leaf entries MUST take the format:
+
+```yaml
+matree://<key>@<fqdn>
+```
+
+which links to a different list located in another domain.
+
+### `multiaddr` entries
+
+For the subtree pointed to by `ma-root`,
+leaf entries MUST take the format:
+
+```yaml
+ma:<multiaddr>
+```
+
+which contains the `multiaddr` of a `libp2p` peer.
+
+## Client protocol
+
+A client MUST adhere to the [client protocol](https://eips.ethereum.org/EIPS/eip-1459#client-protocol)
+as specified in EIP-1459,
+and adapted for usage with `multiaddr` entry types below:
+
+To find nodes at a given DNS name a client MUST perform the following steps:
+
+1. Resolve the TXT record of the DNS name and
+check whether it contains a valid `matree-root:v1` entry.
+2. Verify the signature on the root against the known public key
+and check whether the sequence number is larger than or
+equal to any previous number seen for that name.
+3. Resolve the TXT record of a hash subdomain indicated in the record
+and verify that the content matches the hash.
+4. If the resolved entry is of type:
+
+- `matree-branch`: parse the list of hashes and continue resolving them (step 3).
+- `ma`: import the `multiaddr` and add it to a local list of discovered nodes.
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [`10/WAKU2`](../../waku/standards/core/10/waku2.md)
+1. [EIP-1459: Client Protocol](https://eips.ethereum.org/EIPS/eip-1459#client-protocol)
+1. [EIP-1459: Node Discovery via DNS](https://eips.ethereum.org/EIPS/eip-1459)
+1. [`libp2p`](https://libp2p.io/)
+1. [`libp2p` peer identity](https://docs.libp2p.io/concepts/peer-id/)
+1. [Merkle trees](https://www.wikiwand.com/en/Merkle_tree)

vac/3/images/remote-log.msc

@@ -1,21 +1,21 @@
-# Alice and Bob: remote log data sync
-msc {
-  hscale="2", wordwraparcs=on;
-
-  alice [label="Alice"],
-  cas [label="CAS"],
-  ns [label="NS"],
-  bob [label="Bob"];
-
-  --- [label="Alice replicates data to a remote log"];
-  alice => cas [label="Add content"];
-  cas => alice [label="Address"];
-  alice => ns [label="Update NameUpdate"];
-  ns => alice [label="Response"];
-
-  --- [label="Bob comes online"];
-  bob => ns [label="Fetch"];
-  ns => bob [label="Content"];
-  bob => cas [label="Fetch Query"];
-  cas => bob [label="Content"];
-}
+# Alice and Bob: remote log data sync
+msc {
+  hscale="2", wordwraparcs=on;
+
+  alice [label="Alice"],
+  cas [label="CAS"],
+  ns [label="NS"],
+  bob [label="Bob"];
+
+  --- [label="Alice replicates data to a remote log"];
+  alice => cas [label="Add content"];
+  cas => alice [label="Address"];
+  alice => ns [label="Update NameUpdate"];
+  ns => alice [label="Response"];
+
+  --- [label="Bob comes online"];
+  bob => ns [label="Fetch"];
+  ns => bob [label="Content"];
+  bob => cas [label="Fetch Query"];
+  cas => bob [label="Content"];
+}

vac/3/remote-log.md

@@ -1,219 +1,229 @@
----
-slug: 3
-title: 3/REMOTE-LOG
-name: Remote log specification
-status: draft
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributors:
-  - Dean Eigenmann <dean@status.im>
----
-
-A remote log is a replication of a local log. This means a node can read data that originally came from a node that is offline.
-
-This specification is complemented by a proof of concept implementation[^1].
-
-## Definitions
-
-| Term        | Definition                                                                                   |
-| ----------- | --------------------------------------------------------------------------------------       |
-| CAS         | Content-addressed storage. Stores data that can be addressed by its hash.                    |
-| NS          | Name system. Associates mutable data to a name.                                              |
-| Remote log  | Replication of a local log at a different location.                                          |
-
-## Wire Protocol
-
-### Secure Transport, storage, and name system
-
-This specification does not define anything related to: secure transport,
-content addressed storage, or the name system. It is assumed these capabilities
-are abstracted away in such a way that any such protocol can easily be
-implemented.
-
-<!-- TODO: Elaborate on properties required here. -->
-
-### Payloads
-
-Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
-
-**CAS service**:
-
-```protobuf
-syntax = "proto3";
-
-package vac.cas;
-
-service CAS {
-  rpc Add(Content) returns (Address) {}
-  rpc Get(Address) returns (Content) {}
-}
-
-message Address {
-  bytes id = 1;
-}
-
-message Content {
-  bytes data = 1;
-}
-```
-
-<!-- XXX/TODO: Can we get rid of the id/data complication and just use bytes? -->
-
-**NS service**:
-
-```protobuf
-syntax = "proto3";
-
-package vac.cas;
-
-service NS {
-  rpc Update(NameUpdate) returns (Response) {}
-  rpc Fetch(Query) returns (Content) {}
-}
-
-message NameUpdate {
-  string name = 1;
-  bytes content = 2;
-}
-
-message Query {
-  string name = 1;
-}
-
-message Content {
-  bytes data = 1;
-}
-
-message Response {
-  bytes data = 1;
-}
-```
-
-<!-- XXX: Response and data type a bit weird, Ok/Err enum? -->
-<!-- TODO: Do we want NameInit here? -->
-
-**Remote log:**
-
-```protobuf
-syntax = "proto3";
-
-package vac.cas;
-
-message RemoteLog {
-  repeated Pair pair = 1;
-  bytes tail = 2;
-
-  message Pair {
-    bytes remoteHash = 1;
-    bytes localHash = 2;
-    bytes data = 3;
-  }
-}
-```
-
-<!-- TODO: Better name for Pair, Mapping? -->
-
-<!-- TODO: Consider making more useful in conjunction with metadata field. It makes sense to explicitly list what sequence a message is <local hash, remote hash, data, seqid> this way I can easily sync a messages prior or after a specific number. To enable this to be dynamic it might make sense to add page info so that I am aware which page I can find seqid on -->
-
-## Synchronization
-
-### Roles
-
-There are four fundamental roles:
-
-1. Alice
-2. Bob
-2. Name system (NS)
-3. Content-addressed storage (CAS)
-
-The *remote log* protobuf is what is stored in the name system.
-
-"Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.
-
-<!-- TODO: Document random node as remote log -->
-<!-- TODO: Document how to find initial remote log (e.g. per sync contexts -->
-
-### Flow
-
-<!-- diagram -->
-
-<p align="center">
-    <img src="./images/remote-log.png" />
-    <br />
-    Figure 1: Remote log data synchronization.
-</p>
-
-<!-- Document the flow wrt operations -->
-
-### Remote log
-
-The remote log lets receiving nodes know what data they are missing. Depending
-on the specific requirements and capabilities of the nodes and name system, the
-information can be referred to differently. We distinguish between three rough
-modes:
-
-1. Fully replicated log
-2. Normal sized page with CAS mapping
-3. "Linked list" mode - minimally sized page with CAS mapping
-
-**Data format:**
-
-```
-| H1_3 | H2_3 |
-| H1_2 | H2_2 |
-| H1_1 | H2_1 |
-| ------------|
-| next_page   |
-```
-
-Here the upper section indicates a list of ordered pairs, and the lower section
-contains the address for the next page chunk. `H1` is the native hash function,
-and `H2` is the one used by the CAS. The numbers corresponds to the messages.
-
-To indicate which CAS is used, a remote log SHOULD use a multiaddr.
-
-**Embedded data:**
-
-A remote log MAY also choose to embed the wire payloads that corresponds to the
-native hash. This bypasses the need for a dedicated CAS and additional
-round-trips, with a trade-off in bandwidth usage.
-
-```
-| H1_3 | | C_3 |
-| H1_2 | | C_2 |
-| H1_1 | | C_1 |
-| -------------|
-| next_page    |
-```
-
-Here `C` stands for the content that would be stored at the CAS.
-
-Both patterns can be used in parallel, e,g. by storing the last `k` messages
-directly and use CAS pointers for the rest. Together with the `next_page` page
-semantics, this gives users flexibility in terms of bandwidth and
-latency/indirection, all the way from a simple linked list to a fully replicated
-log. The latter is useful for things like backups on durable storage.
-
-### Next page semantics
-
-The pointer to the 'next page' is another remote log entry, at a previous point
-in time.
-
-<!-- TODO: Determine requirement re overlapping, adjacent, and/or missing entries -->
-
-<!-- TODO: Document message ordering append only requirements -->
-
-### Interaction with MVDS
-
-[vac.mvds.Message](../2/mvds.md/#payloads) payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.
-
-## Acknowledgments
-
-TBD.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## Footnotes
-
-[^1]:  <https://github.com/vacp2p/research/tree/master/remote_log>
+---
+slug: 3
+title: 3/REMOTE-LOG
+name: Remote log specification
+status: draft
+editor: Oskar Thorén <oskarth@titanproxy.com>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+---
+
+A remote log is a replication of a local log.
+This means a node can read data that originally came from a node that is offline.
+
+This specification is complemented by a proof of concept implementation[^1].
+
+## Definitions
+
+| Term        | Definition                                                                                   |
+| ----------- | --------------------------------------------------------------------------------------       |
+| CAS         | Content-addressed storage. Stores data that can be addressed by its hash.                    |
+| NS          | Name system. Associates mutable data to a name.                                              |
+| Remote log  | Replication of a local log at a different location.                                          |
+
+## Wire Protocol
+
+### Secure Transport, storage, and name system
+
+This specification does not define anything related to: secure transport,
+content addressed storage, or the name system. It is assumed these capabilities
+are abstracted away in such a way that any such protocol can easily be
+implemented.
+
+<!-- TODO: Elaborate on properties required here. -->
+
+### Payloads
+
+Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
+
+**CAS service**:
+
+```protobuf
+syntax = "proto3";
+
+package vac.cas;
+
+service CAS {
+  rpc Add(Content) returns (Address) {}
+  rpc Get(Address) returns (Content) {}
+}
+
+message Address {
+  bytes id = 1;
+}
+
+message Content {
+  bytes data = 1;
+}
+```
+
+<!-- XXX/TODO: Can we get rid of the id/data complication and just use bytes? -->
+
+**NS service**:
+
+```protobuf
+syntax = "proto3";
+
+package vac.cas;
+
+service NS {
+  rpc Update(NameUpdate) returns (Response) {}
+  rpc Fetch(Query) returns (Content) {}
+}
+
+message NameUpdate {
+  string name = 1;
+  bytes content = 2;
+}
+
+message Query {
+  string name = 1;
+}
+
+message Content {
+  bytes data = 1;
+}
+
+message Response {
+  bytes data = 1;
+}
+```
+
+<!-- XXX: Response and data type a bit weird, Ok/Err enum? -->
+<!-- TODO: Do we want NameInit here? -->
+
+**Remote log:**
+
+```protobuf
+syntax = "proto3";
+
+package vac.cas;
+
+message RemoteLog {
+  repeated Pair pair = 1;
+  bytes tail = 2;
+
+  message Pair {
+    bytes remoteHash = 1;
+    bytes localHash = 2;
+    bytes data = 3;
+  }
+}
+```
+
+<!-- TODO: Better name for Pair, Mapping? -->
+
+<!-- TODO: Consider making more useful in conjunction with metadata field.
+It makes sense to explicitly list what sequence a message is <local hash,
+remote hash, data, seqid> this way I can easily sync a messages prior or
+after a specific number.
+To enable this to be dynamic it might make sense to add page info so
+that I am aware which page I can find seqid on -->
+
+## Synchronization
+
+### Roles
+
+There are four fundamental roles:
+
+1. Alice
+2. Bob
+3. Name system (NS)
+4. Content-addressed storage (CAS)
+
+The *remote log* protobuf is what is stored in the name system.
+
+"Bob" can represent anything from 0 to N participants. Unlike Alice,
+Bob only needs read-only access to NS and CAS.
+
+<!-- TODO: Document random node as remote log -->
+<!-- TODO: Document how to find initial remote log (e.g. per sync contexts -->
+
+### Flow
+
+<!-- diagram -->
+
+<p align="center">
+    <img src="./images/remote-log.png" />
+    <br />
+    Figure 1: Remote log data synchronization.
+</p>
+
+<!-- Document the flow wrt operations -->
+
+### Remote log
+
+The remote log lets receiving nodes know what data they are missing. Depending
+on the specific requirements and capabilities of the nodes and name system, the
+information can be referred to differently. We distinguish between three rough
+modes:
+
+1. Fully replicated log
+2. Normal sized page with CAS mapping
+3. "Linked list" mode - minimally sized page with CAS mapping
+
+**Data format:**
+
+```yaml
+| H1_3 | H2_3 |
+| H1_2 | H2_2 |
+| H1_1 | H2_1 |
+| ------------|
+| next_page   |
+```
+
+Here the upper section indicates a list of ordered pairs, and the lower section
+contains the address for the next page chunk. `H1` is the native hash function,
+and `H2` is the one used by the CAS. The numbers corresponds to the messages.
+
+To indicate which CAS is used, a remote log SHOULD use a multiaddr.
+
+**Embedded data:**
+
+A remote log MAY also choose to embed the wire payloads that corresponds to the
+native hash. This bypasses the need for a dedicated CAS and additional
+round-trips, with a trade-off in bandwidth usage.
+
+```yaml
+| H1_3 | | C_3 |
+| H1_2 | | C_2 |
+| H1_1 | | C_1 |
+| -------------|
+| next_page    |
+```
+
+Here `C` stands for the content that would be stored at the CAS.
+
+Both patterns can be used in parallel, e,g. by storing the last `k` messages
+directly and use CAS pointers for the rest. Together with the `next_page` page
+semantics, this gives users flexibility in terms of bandwidth and
+latency/indirection, all the way from a simple linked list to a fully replicated
+log. The latter is useful for things like backups on durable storage.
+
+### Next page semantics
+
+The pointer to the 'next page' is another remote log entry, at a previous point
+in time.
+
+<!-- TODO: Determine requirement re overlapping, adjacent, 
+and/or missing entries -->
+
+<!-- TODO: Document message ordering append only requirements -->
+
+### Interaction with MVDS
+
+[vac.mvds.Message](../2/mvds.md/#payloads) payloads are the only payloads
+that MUST be uploaded.
+Other messages types MAY be uploaded, depending on the implementation.
+
+## Acknowledgments
+
+TBD.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## Footnotes
+
+[^1]:  <https://github.com/vacp2p/research/tree/master/remote_log>

vac/4/mvds-meta.md

@@ -1,80 +1,104 @@
----
-slug: 4
-title: 4/MVDS-META
-name: MVDS Metadata Field
-status: draft
-editor: Sanaz Taheri <sanaz@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Andrea Maria Piana <andreap@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
----
-
-In this specification, we describe a method to construct message history that will aid the consistency guarantees of [2/MVDS](../2/mvds.md). Additionally, we explain how data sync can be used for more lightweight messages that do not require full synchronization.
-
-## Motivation
-
-In order for more efficient synchronization of conversational messages, information should be provided allowing a node to more effectively synchronize the dependencies for any given message.
-
-## Format
-
-We introduce the metadata message which is used to convey information about a message and how it SHOULD be handled.
-
-```protobuf
-package vac.mvds;
-
-message Metadata {
-  repeated bytes parents = 1;
-  bool ephemeral = 2;
-}
-```
-
-Nodes MAY transmit a `Metadata` message by extending the MVDS [message](../2/mvds.md/#payloads) with a `metadata` field.
-
-```diff
-message Message {
-  bytes group_id = 6001;
-  int64 timestamp = 6002;
-  bytes body = 6003;
-+ Metadata metadata = 6004;
-}
-```
-
-### Fields
-
-| Name                   |   Description                                                                                                                    |
-| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
-| `parents`               |   list of parent [`message identifier`s](../2/mvds.md/#payloads) for the specific message. |            
-| `ephemeral`         |   indicates whether a message is ephemeral or not.                                                             |
-
-## Usage
-
-### `parents`
-
-This field contains a list of parent [`message identifier`s](../2/mvds.md/#payloads) for the specific message. It MUST NOT contain any messages as parent whose `ack` flag was set to `false`. This establishes a directed acyclic graph (DAG)[^2] of persistent messages.
-
-Nodes MAY buffer messages until dependencies are satisfied for causal consistency[^3], they MAY also pass the messages straight away for eventual consistency[^4].
-
-A parent is any message before a new message that a node is aware of that has no children.
-
-The number of parents for a given message is bound by [0, N], where N is the number of nodes participating in the conversation, therefore the space requirements for the `parents` field is O(N).
-
-If a message has no parents it is considered a root. There can be multiple roots, which might be disconnected, giving rise to multiple DAGs.
-
-### `ephemeral`
-
-When the `ephemeral` flag is set to `false`, a node MUST send an acknowledgment when they have received and processed a message. If it is set to `true`, it SHOULD NOT send any acknowledgment. The flag is `false` by default.
-
-Nodes MAY decide to not persist ephemeral messages, however they MUST NOT be shared as part of the message history.
-
-Nodes SHOULD send ephemeral messages in batch mode. As their delivery is not needed to be guaranteed.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## Footnotes
-[^1]: [2/MVDS](../2/mvds.md)
-[^2]: <https://en.wikipedia.org/wiki/Directed_acyclic_graph>
-[^3]: Jepsen. [Causal Consistency](https://jepsen.io/consistency/models/causal). Jepsen, LLC.
-[^4]: <https://en.wikipedia.org/wiki/Eventual_consistency>
+---
+slug: 4
+title: 4/MVDS-META
+name: MVDS Metadata Field
+status: draft
+editor: Sanaz Taheri <sanaz@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Andrea Maria Piana <andreap@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+In this specification, we describe a method to construct message history that
+will aid the consistency guarantees of [2/MVDS](../2/mvds.md).
+Additionally,
+we explain how data sync can be used for more lightweight messages that
+do not require full synchronization.
+
+## Motivation
+
+In order for more efficient synchronization of conversational messages,
+information should be provided allowing a node to more effectively synchronize
+the dependencies for any given message.
+
+## Format
+
+We introduce the metadata message which is used to convey information about a message
+and how it SHOULD be handled.
+
+```protobuf
+package vac.mvds;
+
+message Metadata {
+  repeated bytes parents = 1;
+  bool ephemeral = 2;
+}
+```
+
+Nodes MAY transmit a `Metadata` message by extending the MVDS [message](../2/mvds.md/#payloads)
+with a `metadata` field.
+
+```diff
+message Message {
+  bytes group_id = 6001;
+  int64 timestamp = 6002;
+  bytes body = 6003;
++ Metadata metadata = 6004;
+}
+```
+
+### Fields
+
+| Name                   |   Description                                                                                                                    |
+| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| `parents`               |   list of parent [`message identifier`s](../2/mvds.md/#payloads) for the specific message. |
+| `ephemeral`         |   indicates whether a message is ephemeral or not.                                                             |
+
+## Usage
+
+### `parents`
+
+This field contains a list of parent [`message identifier`s](../2/mvds.md/#payloads)
+for the specific message.
+It MUST NOT contain any messages as parent whose `ack` flag was set to `false`.
+This establishes a directed acyclic graph (DAG)[^2] of persistent messages.
+
+Nodes MAY buffer messages until dependencies are satisfied for causal consistency[^3],
+they MAY also pass the messages straight away for eventual consistency[^4].
+
+A parent is any message before a new message that
+a node is aware of that has no children.
+
+The number of parents for a given message is bound by [0, N],
+where N is the number of nodes participating in the conversation,
+therefore the space requirements for the `parents` field is O(N).
+
+If a message has no parents it is considered a root.
+There can be multiple roots, which might be disconnected,
+giving rise to multiple DAGs.
+
+### `ephemeral`
+
+When the `ephemeral` flag is set to `false`,
+a node MUST send an acknowledgment when they have received and processed a message.
+If it is set to `true`, it SHOULD NOT send any acknowledgment.
+The flag is `false` by default.
+
+Nodes MAY decide to not persist ephemeral messages,
+however they MUST NOT be shared as part of the message history.
+
+Nodes SHOULD send ephemeral messages in batch mode.
+As their delivery is not needed to be guaranteed.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## Footnotes
+
+1: [2/MVDS](../2/mvds.md)
+2: [directed_acyclic_graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph)
+3: Jepsen. [Causal Consistency](https://jepsen.io/consistency/models/causal)
+Jepsen, LLC.
+4: <https://en.wikipedia.org/wiki/Eventual_consistency>

vac/48/rln-interep-spec.md

@@ -1,109 +0,0 @@
----
-slug: 48
-title: 48/RLN-INTEREP-SPEC
-name: Interep as group management for RLN
-status: raw
-category:  
-tags: rln
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
----
-
-## Abstract
-
-This spec integrates [Interep](https://interep.link) into the [RLN](../32/rln-v1.md) spec. 
-Interep is a group management protocol that allows for the creation of groups of users and the management of their membership. 
-It is used to manage the membership of the RLN group.
-
-Interep ties in web2 identities with reputation, and sorts the users into groups based on their reputation score.
-For example, a GitHub user with over 100 followers is considered to have "gold" reputation.
-
-Interep uses [Semaphore](https://semaphore.appliedzkp.org/) under the hood to allow anonymous signaling of membership in a group.
-Therefore, a user with a "gold" reputation can prove the existence of their membership without revealing their identity.
-
-RLN is used for spam prevention, and Interep is used for group management.
-
-By using Interep with RLN, we allow users to join RLN membership groups without the need for on-chain financial stake.
-
-## Motivation
-
-To have Sybil-Resistant group management, there are [implementations](https://github.com/vacp2p/rln-contract) of RLN which make use of financial stake on-chain.
-However, this is not ideal because it reduces the barrier of entry for honest participants.
-
-In this case, honest participants will most likely have a web2 identity accessible to them, which can be used for joining an Interep reputation group.
-By modifying the RLN spec to use Interep, we can have Sybil-Resistant group management without the need for on-chain financial stake.
-
-Since RLN and Interep both use Semaphore-style credentials, it is possible to use the same set of credentials for both.
-
-## Functional Operation
-
-Using Interep with RLN involves the following steps -
-
-1. Generate Semaphore credentials 
-2. Verify reputation and join Interep group
-3. Join RLN membership group via interaction with Smart Contract, by passing a proof of membership to the Interep group
-
-### 1. Generate Semaphore credentials
-
-Semaphore credentials are generated in a standard way, depicted in the [Semaphore documentation](https://semaphore.appliedzkp.org/docs/guides/identities#create-deterministic-identities).
-
-### 2. Verify reputation and join Interep group
-
-Using the Interep app deployed on [Goerli](https://goerli.interep.link/), the user can check their reputation tier and join the corresponding group.
-This results in a transaction to the Interep contract, which adds them to the group.
-
-### 3. Join RLN membership group
-
-Instead of sending funds to the RLN contract to join the membership group, the user can send a proof of membership to the Interep group.
-This proof is generated by the user, and is verified by the contract.
-The contract ensures that the user is a member of the Interep group, and then adds them to the RLN membership group.
-
-Following is the modified signature of the register function in the RLN contract -
-
-```solidity
-    /// @param groupId: Id of the group.
-    /// @param signal: Semaphore signal.
-    /// @param nullifierHash: Nullifier hash.
-    /// @param externalNullifier: External nullifier.
-    /// @param proof: Zero-knowledge proof.
-    /// @param idCommitment: ID Commitment of the member.
-    function register(
-        uint256 groupId,
-        bytes32 signal,
-        uint256 nullifierHash,
-        uint256 externalNullifier,
-        uint256[8] calldata proof,
-        uint256 idCommitment
-    )
-```
-
-## Verification of messages
-
-Messages are verified the same way as in the [RLN spec](../32/rln-v1.md/#verification).
-
-## Slashing
-
-The slashing mechanism is the same as in the [RLN spec](../32/rln-v1.md/#slashing).
-It is important to note that the slashing may not have the intended effect on the user, since the only consequence is that they cannot send messages.
-This is due to the fact that the user can send a identity commitment in the registration to the RLN contract, which is different than the one used in the Interep group.
-
-## Proof of Concept
-
-A proof of concept is available at [vacp2p/rln-interp-contract](https://github.com/vacp2p/rln-interep-contract) which integrates Interep with RLN.
-
-## Security Considerations
-
-1. As mentioned in [Slashing](#slashing), the slashing mechanism may not have the intended effect on the user.
-2. This spec inherits the security considerations of the [RLN spec](../32/rln-v1.md/#security-considerations).
-3. This spec inherits the security considerations of [Interep](https://docs.interep.link/).
-4. A user may make multiple registrations using the same Interep proofs but different identity commitments. The way to mitigate this is to check if the nullifier hash has been detected previously in proof verification.
-
-## References
-
-1. [RLN spec](../32/rln-v1.md)
-2. [Interep](https://interep.link)
-3. [Semaphore](https://semaphore.appliedzkp.org/)
-4. [Decentralized cloudflare using Interep](https://ethresear.ch/t/decentralised-cloudflare-using-rln-and-rich-user-identities/10774)
-5. [Interep contracts](https://github.com/interep-project/contracts)
-6. [RLN contract](https://github.com/vacp2p/rln-contract)
-7. [RLNP2P](https://rlnp2p.vac.dev/)

vac/70/eth-secpm.md

@@ -1,810 +0,0 @@
----
-slug: 70
-title: 70/ETH-SECPM
-name: Secure channel setup using Ethereum accounts
-status: raw
-category: Standards Track
-tags:
-editor: Ramses Fernandez <ramses@status.im>
-contributors:
----
-
-## Motivation
-The need for secure communications has become paramount. 
-Traditional centralized messaging protocols are susceptible to various security threats, 
-including unauthorized access, data breaches, and single points of failure. 
-Therefore a decentralized approach to secure communication becomes increasingly relevant, 
-offering a robust solution to address these challenges.
-
-This specification outlines a private messaging service using the Ethereum blockchain as authentication service. 
-Rooted in the existing [model](https://rfc.vac.dev/spec/20/), 
-this proposal addresses the deficiencies related to forward privacy and authentication inherent in the current framework.
-The specification is divided into 3 sections:
-
-- Private 1-to-1 communications protocol, based on [Signal's double ratchet](https://signal.org/docs/specifications/doubleratchet/).
-- Private group messaging protocol, based on the [MLS protocol](https://datatracker.ietf.org/doc/rfc9420/).
-- Description of an Ethereum-based authentication protocol, based on [SIWE](https://eips.ethereum.org/EIPS/eip-4361).
-
-
-## Private 1-to-1 communications protocol
-
-### Theory
-The specification is based on the noise protocol framework.
-It corresponds to the double ratchet scheme combined with the X3DH algorithm, which will be used to initialize the former. 
-We chose to express the protocol in noise to be be able to use the noise streamlined implementation and proving features.
-The X3DH algorithm provides both authentication and forward secrecy, as stated in the [X3DH specification](https://signal.org/docs/specifications/x3dh/).
-
-This protocol will consist of several stages:
-
-1.  Key setting for X3DH: this step will produce prekey bundles for Bob which will be fed into X3DH. It will also allow Alice to generate the keys required to run the X3DH algorithm correctly.
-2.  Execution of X3DH: This step will output a common secret key `SK` together with an additional data vector `AD`. Both will be used in the double ratchet algorithm initialization.
-3.  Execution of the double ratchet algorithm for forward secure, authenticated communications, using the common secret key `SK`, obtained from X3DH, as a root key.
-
-The protocol assumes the following requirements:
--   Alice knows Bob’s Ethereum address.
--   Bob is willing to participate in the protocol, and publishes his public key.
--   Bob’s ownership of his public key is verifiable,
--   Alice wants to send message M to Bob.
--   An eavesdropper cannot read M’s content even if she is storing it or relaying it.
-
-### Syntax
-#### Cryptographic suite
-The following cryptographic functions MUST be used:
-- `X488` as Diffie-Hellman function `DH`.
-- `SHA256` as KDF.
-- `AES256-GCM` as AEAD algorithm.
-- `SHA512` as hash function.
-- `XEd448` for digital signatures.
-
-#### X3DH initialization
-This scheme MUST work on the curve curve448.
-The X3DH algorithm corresponds to the IX pattern in Noise.
-
-Bob and Alice MUST define personal key pairs `(ik_B, IK_B)` and `(ik_A, IK_A)` respectively where:
--   The key `ik` must be kept secret,
--   and the key `IK` is public.
-
-Bob MUST generate new keys using `(ik_B, IK_B) = GENERATE_KEYPAIR(curve = curve448)`.
-
-Bob MUST also generate a public key pair `(spk_B, SPK_B) = GENERATE_KEYPAIR(curve = curve448)`.
-
-`SPK` is a public key generated and stored at medium-term. 
-Both signed prekey and the certificate MUST undergo periodic replacement. 
-After replacing the key, 
-Bob keeps the old private key of `SPK` for some interval, dependant on the implementation.
-This allows Bob to decrypt delayed messages. 
-
-Bob MUST sign `SPK` for authentication: `SigSPK = XEd448(ik, Encode(SPK))`
-
-A final step requires the definition of `prekey_bundle = (IK, SPK, SigSPK, OPK_i)`
-
-One-time keys `OPK` MUST be generated as `(opk_B, OPK_B) = GENERATE_KEYPAIR(curve = curve448)`.
-
-Before sending an initial message to Bob, Alice MUST generate an AD: `AD = Encode(IK_A) || Encode(IK_B)`.
-
-Alice MUST generate ephemeral key pairs `(ek, EK) = GENERATE_KEYPAIR(curve = curve448)`.
-
-The function `Encode()` transforms an curve448 public key into a byte sequence. 
-This is specified in the [RFC 7748](http://www.ietf.org/rfc/rfc7748.txt) on elliptic curves for security.
-
-One MUST consider `q = 2^446 - 13818066809895115352007386748515426880336692474882178609894547503885` for digital signatures with `(XEd448_sign, XEd448_verify)`:
-```
-XEd448_sign((ik, IK), message):
-    Z = randbytes(64)  
-    r = SHA512(2^456 - 2 || ik || message || Z )
-    R = (r * convert_mont(5)) % q
-    h = SHA512(R || IK || M)
-    s = (r + h * ik) % q
-    return (R || s)
-```
-```
-XEd448_verify(u, message, (R || s)):
-    if (R.y >= 2^448) or (s >= 2^446): return FALSE
-    h = (SHA512(R || 156326 || message)) % q
-    R_check = s * convert_mont(5) - h * 156326
-    if R == R_check: return TRUE
-    return FALSE 
-```
-```
-convert_mont(u):
-    u_masked = u % mod 2^448
-    inv = ((1 - u_masked)^(2^448 - 2^224 - 3)) % (2^448 - 2^224 - 1)
-    P.y = ((1 + u_masked) * inv)) % (2^448 - 2^224 - 1)
-    P.s = 0
-    return P
-```
-
-#### Use of X3DH
-This specification combines the double ratchet with X3DH using the following data as initialization for the former:
-
--   The `SK` output from X3DH becomes the `SK` input of the double ratchet. See section 3.3 of [Signal Specification](https://signal.org/docs/specifications/doubleratchet/) for a detailed description.
--   The `AD` output from X3DH becomes the `AD` input of the double ratchet. See sections 3.4 and 3.5 of  [Signal Specification](https://signal.org/docs/specifications/doubleratchet/)  for a detailed description.
--   Bob’s signed prekey `SigSPKB` from X3DH is used as Bob’s initial ratchet public key of the double ratchet.
-
-X3DH has three phases:
-
-1.  Bob publishes his identity key and prekeys to a server, a network, or dedicated smart contract.
-2.  Alice fetches a prekey bundle from the server, and uses it to send an initial message to Bob.
-3.  Bob receives and processes Alice's initial message.
-
-Alice MUST perform the following computations:
-```
-dh1 = DH(IK_A, SPK_B, curve = curve448)
-dh2 = DH(EK_A, IK_B, curve = curve448)
-dh3 = DH(EK_A, SPK_B)
-SK = KDF(dh1 || dh2 || dh3)
-```
-Alice MUST send to Bob a message containing: 
-
--  `IK_A, EK_A`.
--  An identifier to Bob's prekeys used.
--  A message encrypted with AES256-GCM using `AD` and `SK`.
-
-Upon reception of the initial message, Bob MUST:
-1. Perform the same computations above with the `DH()` function.
-2. Derive `SK` and construct `AD`.
-3. Decrypt the initial message encrypted with `AES256-GCM`.
-4. If decryption fails, abort the protocol.
-
-#### Initialization of the double datchet
-In this stage Bob and Alice have generated key pairs and agreed a shared secret `SK` using X3DH.
-
-Alice calls `RatchetInitAlice()` defined below:
-```
-RatchetInitAlice(SK, IK_B):
-    state.DHs = GENERATE_KEYPAIR(curve = curve448)
-    state.DHr = IK_B
-    state.RK, state.CKs = HKDF(SK, DH(state.DHs, state.DHr)) 
-    state.CKr = None
-    state.Ns, state.Nr, state.PN = 0
-    state.MKSKIPPED = {}
-```
-The HKDF function MUST be the proposal by [Krawczyk and Eronen](http://www.ietf.org/rfc/rfc5869.txt).
-In this proposal `chaining_key` and `input_key_material` MUST be replaced with `SK` and the output of `DH` respectively.
-
-Similarly, Bob calls the function `RatchetInitBob()` defined below:
-```
-RatchetInitBob(SK, (ik_B,IK_B)):
-    state.DHs = (ik_B, IK_B)
-    state.Dhr = None
-    state.RK = SK
-    state.CKs, state.CKr = None
-    state.Ns, state.Nr, state.PN = 0
-    state.MKSKIPPED = {}
-```
-#### Encryption
-This function performs the symmetric key ratchet.
-
-```
-RatchetEncrypt(state, plaintext, AD):
-    state.CKs, mk = HMAC-SHA256(state.CKs)
-    header = HEADER(state.DHs, state.PN, state.Ns)
-    state.Ns = state.Ns + 1
-	return header, AES256-GCM_Enc(mk, plaintext, AD || header)
-```
-The `HEADER` function creates a new message header containing the public key from the key pair output of the `DH`function.  
-It outputs the previous chain length `pn`, and the message number `n`. 
-The returned header object contains ratchet public key `dh` and integers `pn` and `n`.
-
-#### Decryption
-The function `RatchetDecrypt()` decrypts incoming messages:
-```
-RatchetDecrypt(state, header, ciphertext, AD):
-    plaintext = TrySkippedMessageKeys(state, header, ciphertext, AD)
-    if plaintext != None:
-        return plaintext
-    if header.dh != state.DHr:
-        SkipMessageKeys(state, header.pn)
-        DHRatchet(state, header)
-    SkipMessageKeys(state, header.n)
-    state.CKr, mk = HMAC-SHA256(state.CKr)
-    state.Nr = state.Nr + 1
-    return AES256-GCM_Dec(mk, ciphertext, AD || header)
-```
-Auxiliary functions follow:
-
-```
-DHRatchet(state, header):
-    state.PN = state.Ns
-    state.Ns = state.Nr = 0
-    state.DHr = header.dh
-    state.RK, state.CKr = HKDF(state.RK, DH(state.DHs, state.DHr))
-    state.DHs = GENERATE_KEYPAIR(curve = curve448)
-    state.RK, state.CKs = HKDF(state.RK, DH(state.DHs, state.DHr))
-```
-```
-SkipMessageKeys(state, until):
-    if state.NR + MAX_SKIP < until:
-        raise Error
-    if state.CKr != none:
-        while state.Nr < until:
-            state.CKr, mk = HMAC-SHA256(state.CKr)
-            state.MKSKIPPED[state.DHr, state.Nr] = mk
-            state.Nr = state.Nr + 1
-```
-```
-TrySkippedMessageKey(state, header, ciphertext, AD):
-    if (header.dh, header.n) in state.MKSKIPPED:
-        mk = state.MKSKIPPED[header.dh, header.n]
-        delete state.MKSKIPPED[header.dh, header.n]
-        return AES256-GCM_Dec(mk, ciphertext, AD || header)
-    else: return None
-```
-
-## Information retrieval
-
-### Static data
-Some data, such as the key pairs `(ik, IK)` for Alice and Bob, MAY NOT be regenerated after a period of time. 
-Therefore the prekey bundle MAY be stored in long-term storage solutions, such as a dedicated smart contract which outputs such a key pair when receiving an Ethereum wallet address.
-
-Storing static data is done using a dedicated smart contract `PublicKeyStorage` which associates the Ethereum wallet address of a user with his public key.
-This mapping is done by `PublicKeyStorage` using a `publicKeys` function, or a `setPublicKey` function.
-This mapping is done if the user passed an authorization process.
-A user who wants to retrieve a public key associated with a specific wallet address calls a function `getPublicKey`.
-The user provides the wallet address as the only input parameter for `getPublicKey`.
-The function outputs the associated public key from the smart contract.
-
-### Ephemeral data
-Storing ephemeral data on Ethereum MAY be done using a combination of on-chain and off-chain solutions. 
-This approach provides an efficient solution to the problem of storing updatable data in Ethereum.
-1. Ethereum stores a reference or a hash that points to the off-chain data.
-2. Off-chain solutions can include systems like IPFS, traditional cloud storage solutions, or decentralized storage networks such as a [Swarm](https://www.ethswarm.org). 
-In any case, the user stores the associated IPFS hash, URL or reference in Ethereum.
-
-The fact of a user not updating the ephemeral information can be understood as Bob not willing to participate in any communication.
-
-This applies to `KeyPackage`, which in the MLS specification are meant to be stored in a directory provided by the delivery service.
-If such an element does not exist, `KeyPackage` MUST be stored according to one of the two options outlined above.
-
-## Private group messaging protocol
-### Theory
-The [Messaging Layer Security](https://datatracker.ietf.org/doc/rfc9420/)(MLS) protocol aims at providing a group of users with end-to-end encryption in an authenticated and asynchronous way. 
-The main security characteristics of the protocol are: Message confidentiality and authentication, sender authentication, 
-membership agreement, post-remove and post-update security, and forward secrecy and post-compromise security.
-The MLS protocol achieves: low-complexity, group integrity, synchronization and extensibility.
-
-The extension to group chat described in forthcoming sections is built upon the [MLS](https://datatracker.ietf.org/doc/rfc9420/) protocol.
-
-### Syntax
-Each MLS session uses a single cipher suite that specifies the primitives to be used in group key computations. The cipher suite MUST use:
-- `X488` as Diffie-Hellman function.
-- `SHA256` as KDF.
-- `AES256-GCM` as AEAD algorithm.
-- `SHA512` as hash function.
-- `XEd448` for digital signatures.
-
-Formats for public keys, signatures and public-key encryption MUST follow Section 5.1 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Hash-based identifiers
-Some MLS messages refer to other MLS objects by hash.
-These identifiers MUST be computed according to Section 5.2 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Credentials
-Each member of a group presents a credential that provides one or more identities for the member and associates them with the member's signing key. 
-The identities and signing key are verified by the Authentication Service in use for a group.
-
-Credentials MUST follow the specifications of section 5.3 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).  
-
-### Message framing
-Handshake and application messages use a common framing structure providing encryption to ensure confidentiality within the group, and signing to authenticate the sender.
-
-The structure is:
-- `PublicMessage`: represents a message that is only signed, and not encrypted.
-The definition and the encoding/decoding of a `PublicMessage` MUST follow the specification in section 6.2 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-- `PrivateMessage`: represents a signed and encrypted message, with protections for both the content of the message and related metadata.
-The definition, and the encoding/decoding of a `PrivateMessage` MUST follow the specification in section 6.3 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-Applications MUST use `PrivateMessage` to encrypt application messages. 
-
-Applications SHOULD use `PrivateMessage` to encode handshake messages.
-
-Each encrypted MLS message carries a "generation" number which is a per-sender incrementing counter. 
-If a group member observes a gap in the generation sequence for a sender, 
-then they know that they have missed a message from that sender.
-
-### Nodes contents
-The nodes of a ratchet tree contain several types of data:
-
-- Leaf nodes describe individual members.
-- Parent nodes describe subgroups.
-
-Contents of each kind of node, and its structure MUST follow the indications described in sections 7.1 and 7.2 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Leaf node validation
-`KeyPackage` objects describe the client's capabilities and provides keys that can be used to add the client to a group.
-
-The validity of a leaf node needs to be verified at the following stages:
-- When a leaf node is downloaded in a `KeyPackage`, before it is used to add the client to the group.
-- When a leaf node is received by a group member in an Add, Update, or Commit message.
-- When a client validates a ratchet tree.
-
-A client MUST verify the validity of a leaf node following the instructions of section 7.3 in [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Ratchet tree evolution
-Whenever a member initiates an epoch change, they MAY need to refresh the key pairs of their leaf and of the nodes on their direct path. This is done to keep forward secrecy and post-compromise security.
-The member initiating the epoch change MUST follow this procedure procedure.
-A member updates the nodes along its direct path as follows:
-- Blank all the nodes on the direct path from the leaf to the root.
-- Generate a fresh HPKE key pair for the leaf.
-- Generate a sequence of path secrets, one for each node on the leaf's filtered direct path.
-It MUST follow the procedure described in section 7.4 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-- Compute the sequence of HPKE key pairs `(node_priv,node_pub)`, one for each node on the leaf's direct path.
-It MUST follow the procedure described in section 7.4 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Views of the tree synchronization
-After generating fresh key material and applying it to update their local tree state, the generator broadcasts this update to other members of the group.
-This operation MUST be done according to section 7.5 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Leaf synchronization
-Changes to group memberships MUST be represented by adding and removing leaves of the tree.
-This corresponds to increasing or decreasing the depth of the tree, resulting in the number of leaves being doubled or halved.
-These operations MUST be done as described in section 7.7 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Tree and parent hashing
-Group members can agree on the cryptographic state of the group by generating a hash value that represents the contents of the group ratchet tree and the member’s credentials. 
-The hash of the tree is the hash of its root node, defined recursively from the leaves.
-Tree hashes summarize the state of a tree at point in time.
-The hash of a leaf is the hash of the `LeafNodeHashInput` object. 
-At the same time, the hash of a parent node including the root, is the hash of a `ParentNodeHashInput` object.
-Parent hashes capture information about how keys in the tree were populated.
-
-Tree and parent hashing MUST follow the directions in Sections 7.8 and 7.9 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Key schedule
-Group keys are derived using the `Extract` and `Expand` functions from the KDF for the group's cipher suite, as well as the functions defined below:
-
-```
-ExpandWithLabel(Secret, Label, Context, Length) = KDF.Expand(Secret, KDFLabel, Length)
-DeriveSecret(Secret, Label) = ExpandWithLabel(Secret, Label, "", KDF.Nh)
-```
-`KDFLabel` MUST be specified as:
-```
-struct {
-    uint16 length;
-    opaque label<V>;
-    opaque context<V>;
-} KDFLabel;
-```
-The fields of `KDFLabel` MUST be:
-```
-length = Length;
-label = "MLS 1.0 " + Label;
-context = Context;
-```
-
-Each member of the group MUST maintaint a `GroupContext` object summarizing the state of the group. 
-The sturcture of such object MUST be:
-
-```
-struct {
-ProtocolVersion version = mls10;
-CipherSuite cipher_suite;
-opaque group_id<V>;
-uint64 epoch;
-opaque tree_hash<V>;
-opaque confirmed_trasncript_hash<V>;
-Extension extension<V>;
-} GroupContext;
-```
-
-The use of key scheduling MUST follow the indications in sections 8.1 - 8.7 in [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Secret trees
-For the generation of encryption keys and nonces, the key schedule begins with the `encryption_secret` at the root and derives a tree of secrets with the same structure as the group's ratchet tree.
-Each leaf in the secret tree is associated with the same group member as the corresponding leaf in the ratchet tree.
-
-If `N` is a parent node in the secret tree, the secrets of the children of `N` MUST be defined following section 9 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-#### Encryption keys
-
-MLS encrypts three different types of information:
-- Metadata (sender information).
-- Handshake messages (Proposal and Commit).
-- Application messages.
-
-For handshake and application messages, a sequence of keys is derived via a sender ratchet. 
-Each sender has their own sender ratchet, and each step along the ratchet is called a generation. These procedures MUST follow section 9.1 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-#### Deletion schedule
-
-All security-sensitive values MUST be deleted as soon as they are consumed. 
-A sensitive value S is consumed if:
-
-- S was used to encrypt or (successfully) decrypt a message. 
-- A key, nonce, or secret derived from S has been consumed.
-
-The deletion procedure MUST follow the instruction described in section 9.2 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Key packages
-KeyPackage objects are used to ease the addition of clients to a group asynchronously.
-A KeyPackage object specifies:
-
-- Protocol version and cipher suite supported by the client.
-- Public keys that can be used to encrypt Welcome messages. Welcome messages provide new members with the information to initialize their state for the epoch in which they were added or in which they want to add themselves to the group
-- The content of the leaf node that should be added to the tree to represent this client.
-
-KeyPackages are intended to be used only once and SHOULD NOT be reused. 
-
-Clients MAY generate and publish multiple KeyPackages to support multiple cipher suites.
-
-The structure of the object MUST be:
-```
-struct {
-ProtocolVersion version;
-CipherSuite cipher_suite;
-HPKEPublicKey init_key;
-LeafNode leaf_node;
-Extension extensions<V>;
-/* SignWithLabel(., "KeyPackageTBS", KeyPackageTBS) */
-opaque signature<V>;
-}
-```
-```
-struct {
-ProtocolVersion version;
-CipheSuite cipher_suite;
-HPKEPublicKey init_key;
-LeafNode leaf_node;
-Extension extensions<V>;
-}
-```
-`KeyPackage` object MUST be verified when:
-- A `KeyPackage` is downloaded by a group member, before it is used to add the client to the group.
-- When a `KeyPackage` is received by a group member in an `Add` message.
-
-Verification MUST be done as follows:
-- Verify that the cipher suite and protocol version of the `KeyPackage` match those in the `GroupContext`.
-- Verify that the `leaf_node` of the `KeyPackage` is valid for a `KeyPackage`.
-- Verify that the signature on the `KeyPackage` is valid.
-- Verify that the value of `leaf_node.encryption_key` is different from the value of the `init_key field`.
-
-HPKE public keys are opaque values in a format defined by Section 4 of [RFC9180](https://datatracker.ietf.org/doc/rfc9180/).
-
-Signature public keys are represented as opaque values in a format defined by the cipher suite's signature scheme.
-
-### Group creation
-A group is always created with a single member. 
-Other members are then added to the group using the usual Add/Commit mechanism.
-The creator of a group MUST set:
-- the group ID. 
-- cipher suite.
-- initial extensions for the group. 
-
-If the creator intends to add other members at the time of creation, then it SHOULD fetch `KeyPackages` for those members, and select a cipher suite and extensions according to their capabilities. 
-
-The creator MUST use the capabilities information in these `KeyPackages` to verify that the chosen version and cipher suite is the best option supported by all members.
-
-Group IDs SHOULD be constructed so they are unique with high probability. 
-
-To initialize a group, the creator of the group MUST initialize a one-member group with the following initial values:
-- Ratchet tree: A tree with a single node, a leaf node containing an HPKE public key and credential for the creator.
-- Group ID: A value set by the creator.
-- Epoch: `0`.
-- Tree hash: The root hash of the above ratchet tree.
-- Confirmed transcript hash: The zero-length octet string.
-- Epoch secret: A fresh random value of size `KDF.Nh`.
-- Extensions: Any values of the creator's choosing.
-
-The creator MUST also calculate the interim transcript hash:
-- Derive the `confirmation_key` for the epoch according to Section 8 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-- Compute a `confirmation_tag` over the empty `confirmed_transcript_hash` using the `confirmation_key` as described in Section 8.1 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-- Compute the updated `interim_transcript_hash` from the `confirmed_transcript_hash` and the `confirmation_tag` as described in Section 8.2 [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-All members of a group MUST support the cipher suite and protocol version in use. Additional requirements MAY be imposed by including a `required_capabilities` extension in the `GroupContext`.
-
-```
-struct {
-ExtensionType extension_types<V>;
-ProposalType proposal_types<V>;
-CredentialType credential_types<V>;
-}
-```
-
-### Group evolution
-Group membership can change, and existing members can change their keys in order to achieve post-compromise security. 
-In MLS, each such change is accomplished by a two-step process:
-- A proposal to make the change is broadcast to the group in a Proposal message.
-- A member of the group or a new member broadcasts a Commit message that causes one or more proposed changes to enter into effect.
-
-The group evolves from one cryptographic state to another each time a Commit message is sent and processed. 
-These states are called epochs and are uniquely identified among states of the group by eight-octet epoch values. 
-
-Proposals are included in a `FramedContent` by way of a `Proposal` structure that indicates their type:
-
-```
-struct {
-ProposalType proposal_type;
-select (Proposal.proposal_type) {
-case add:			Add:
-case update:			Update;
-case remove:			Remove;
-case psk:			PreSharedKey;
-case reinit:			ReInit;
-case external_init:		ExternalInit;
-case group_context_extensions:	GroupContextExtensions;
-}
-```
-On receiving a `FramedContent` containing a `Proposal`, a client MUST verify the signature inside `FramedContentAuthData` and that the epoch field of the enclosing FramedContent is equal to the epoch field of the current GroupContext object. 
-If the verification is successful, then the Proposal SHOULD be cached in such a way that it can be retrieved by hash in a later Commit message.
-
-Proposals are organized as follows:
-- `Add`: requests that a client with a specified KeyPackage be added to the group.
-- `Update`: similar to Add, it replaces the sender's LeafNode in the tree instead of adding a new leaf to the tree.
-- `Remove`: requests that the member with the leaf index removed be removed from the group.
-- `ReInit`: requests to reinitialize the group with different parameters.
-- `ExternalInit`: used by new members that want to join a group by using an external commit.
-- `GroupContentExtensions`: it is used to update the list of extensions in the GroupContext for the group.
-
-Proposals structure and semantics MUST follow sections 12.1.1 - 12.1.7 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-Any list of commited proposals MUST be validated either by a the group member who created the commit, or any group member processing such commit.
-The validation MUST be done according to one of the procedures described in Section 12.2 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-When creating or processing a Commit, a client applies a list of proposals to the ratchet tree and `GroupContext`. 
-The client MUST apply the proposals in the list in the order described in Section 12.3 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Commit messages
-Commit messages initiate new group epochs. 
-It informs group members to update their representation of the state of the group by applying the proposals and advancing the key schedule.
-
-Each proposal covered by the Commit is included by a `ProposalOrRef` value.
-`ProposalOrRef` identify the proposal to be applied by value or by reference. 
-Commits that refer to new Proposals from the committer can be included by value. 
-Commits for previously sent proposals from anyone can be sent by reference. 
-Proposals sent by reference are specified by including the hash of the `AuthenticatedContent`.
-
-Group members that have observed one or more valid proposals within an epoch MUST send a Commit message before sending application data.
-A sender and a receiver of a Commit MUST verify that the committed list of proposals is valid.
-The sender of a Commit SHOULD include all valid proposals received during the current epoch.
-
-Functioning of commits MUST follow the instructions of Section 12.4 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Application messages
-Handshake messages provide an authenticated group key exchange to clients. 
-To protect application messages sent among the members of a group, the `encryption_secret` provided by the key schedule is used to derive a sequence of nonces and keys for message encryption.
-
-Each client MUST maintain their local copy of the key schedule for each epoch during which they are a group member. 
-They derive new keys, nonces, and secrets as needed. This data MUST be deleted as soon as they have been used.
-
-Group members MUST use the AEAD algorithm associated with the negotiated MLS ciphersuite to encrypt and decrypt Application messages according to the Message Framing section.
-The group identifier and epoch allow a device to know which group secrets should be used and from which Epoch secret to start computing other secrets and keys. 
-Application messages SHOULD be padded to provide resistance against traffic analysis techniques. 
-This avoids additional information to be provided to an attacker in order to guess the length of the encrypted message.
-Padding SHOULD be used on messages with zero-valued bytes before AEAD encryption.
-
-Functioning of application messages MUST follow the instructions of Section 15 of [RFC9420](https://datatracker.ietf.org/doc/rfc9420/).
-
-### Considerations with respect to decentralization
-The MLS protocol assumes the existence on a (central, untrusted) *delivery service*, whose responsabilites include:
-
-- Acting as a directory service providing the initial keying material for clients to use.
-- Routing MLS messages among clients.
-
-The central delivery service can be avoided in protocols using the publish/gossip approach, such as [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub).
-
-Concerning keys, each node can generate and disseminate their encryption key among the other nodes, so they can create a local version of the tree that allows for the generation of the group key.
-
-Another important component is the *authentication service*, which is replaced with SIWE in this specification. 
-
-## Ethereum-based authentication protocol
-### Theory
-Sign-in with Ethereum describes how Ethereum accounts authenticate with off-chain services by signing a standard message format 
-parameterized by scope, session details, and security mechanisms.
-Sign-in with Ethereum (SIWE), which is described in the [EIP 4361](https://eips.ethereum.org/EIPS/eip-4361), MUST be the authentication method required. 
-
-### Syntax
-#### Message format (ABNF)
-A SIWE Message MUST conform with the following Augmented Backus–Naur Form ([RFC 5234](https://datatracker.ietf.org/doc/html/rfc5234)) expression.
-
-```
-sign-in-with-ethereum =
-    [ scheme "://" ] domain %s" wants you to sign in with your Ethereum account:" LF
-    address LF
-    LF
-    [ statement LF ]
-    LF
-    %s"URI: " uri LF
-    %s"Version: " version LF
-    %s"Chain ID: " chain-id LF
-    %s"Nonce: " nonce LF
-    %s"Issued At: " issued-at
-    [ LF %s"Expiration Time: " expiration-time ]
-    [ LF %s"Not Before: " not-before ]
-    [ LF %s"Request ID: " request-id ]
-    [ LF %s"Resources:"
-    resources ]
-
-scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
-    ; See RFC 3986 for the fully contextualized
-    ; definition of "scheme".
-
-domain = authority
-    ; From RFC 3986:
-    ;     authority     = [ userinfo "@" ] host [ ":" port ]
-    ; See RFC 3986 for the fully contextualized
-    ; definition of "authority".
-
-address = "0x" 40*40HEXDIG
-    ; Must also conform to captilization
-    ; checksum encoding specified in EIP-55
-    ; where applicable (EOAs).
-
-statement = *( reserved / unreserved / " " )
-    ; See RFC 3986 for the definition
-    ; of "reserved" and "unreserved".
-    ; The purpose is to exclude LF (line break).
-
-uri = URI
-    ; See RFC 3986 for the definition of "URI".
-
-version = "1"
-
-chain-id = 1*DIGIT
-    ; See EIP-155 for valid CHAIN_IDs.
-
-nonce = 8*( ALPHA / DIGIT )
-    ; See RFC 5234 for the definition
-    ; of "ALPHA" and "DIGIT".
-
-issued-at = date-time
-expiration-time = date-time
-not-before = date-time
-    ; See RFC 3339 (ISO 8601) for the
-    ; definition of "date-time".
-
-request-id = *pchar
-    ; See RFC 3986 for the definition of "pchar".
-
-resources = *( LF resource )
-
-resource = "- " URI
-```
-
-This specification defines the following SIWE Message fields that can be parsed from a SIWE Message by following the rules in ABNF Message Format:
-
-- `scheme` OPTIONAL. The URI scheme of the origin of the request. 
-Its value MUST be a [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986) URI scheme.
-
-- `domain` REQUIRED. The domain that is requesting the signing. 
-Its value MUST be a [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986) authority. The authority includes an OPTIONAL port. 
-If the port is not specified, the default port for the provided scheme is assumed. 
-
-If scheme is not specified, HTTPS is assumed by default.
-- `address` REQUIRED. The Ethereum address performing the signing. 
-Its value SHOULD be conformant to mixed-case checksum address encoding specified in ERC-55 where applicable.
-
-- `statement` OPTIONAL. A human-readable ASCII assertion that the user will sign which MUST NOT include '\n' (the byte 0x0a).
-- `uri` REQUIRED. An [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986) URI referring to the resource that is the subject of the signing.
-
-- `version` REQUIRED. The current version of the SIWE Message, which MUST be 1 for this specification.
-
-- `chain-id` REQUIRED. The EIP-155 Chain ID to which the session is bound, and the network where Contract Accounts MUST be resolved.
-
-- `nonce` REQUIRED. A random string (minimum 8 alphanumeric characters) chosen by the relying party and used to prevent replay attacks.
-
-- `issued-at` REQUIRED. The time when the message was generated, typically the current time. 
-Its value MUST be an ISO 8601 datetime string.
-
-- `expiration-time` OPTIONAL. The time when the signed authentication message is no longer valid. 
-Its value MUST be an ISO 8601 datetime string.
-
-- `not-before` OPTIONAL. The time when the signed authentication message will become valid. 
-Its value MUST be an ISO 8601 datetime string.
-
-- `request-id` OPTIONAL. An system-specific identifier that MAY be used to uniquely refer to the sign-in request.
-
-- `resources` OPTIONAL. A list of information or references to information the user wishes to have resolved as part of authentication by the relying party. 
-Every resource MUST be a RFC 3986 URI separated by "\n- " where \n is the byte 0x0a.
-
-#### Signing and Verifying Messages with Ethereum Accounts
-- For Externally Owned Accounts, the verification method specified in [ERC-191](https://eips.ethereum.org/EIPS/eip-191) MUST be used.
-
-- For Contract Accounts,
-
-	- The verification method specified in [ERC-1271](https://eips.ethereum.org/EIPS/eip-1271) SHOULD be used. 
-Otherwise, the implementer MUST clearly define the verification method to attain security and interoperability for both wallets and relying parties.
-
-	- When performing [ERC-1271](https://eips.ethereum.org/EIPS/eip-1271) signature verification, the contract performing the verification MUST be resolved from the specified `chain-id`.
-
-	- Implementers SHOULD take into consideration that [ERC-1271](https://eips.ethereum.org/EIPS/eip-1271) implementations are not required to be pure functions. 
-They can return different results for the same inputs depending on blockchain state. 
-This can affect the security model and session validation rules.
-
-#### Resolving Ethereum Name Service (ENS) Data
-- The relying party or wallet MAY additionally perform resolution of ENS data, as this can improve the user experience by displaying human-friendly information that is related to the `address`. 
-Resolvable ENS data include:
-	- The primary ENS name.
-	- The ENS avatar.
-	- Any other resolvable resources specified in the ENS documentation.
-
-- If resolution of ENS data is performed, implementers SHOULD take precautions to preserve user privacy and consent. 
-Their `address` could be forwarded to third party services as part of the resolution process.
-
-#### Implementer steps: specifying the request origin
-The `domain` and, if present, the `scheme`, in the SIWE Message MUST correspond to the origin from where the signing request was made. 
-
-#### Implementer steps: verifying a signed message
-The SIWE Message MUST be checked for conformance to the ABNF Message Format and its signature MUST be checked as defined in Signing and Verifying Messages with Ethereum Accounts.
-
-#### Implementer steps: creating sessions
-Sessions MUST be bound to the address and not to further resolved resources that can change.
-
-#### Implementer steps: interpreting and resolving resources
-Implementers SHOULD ensure that that URIs in the listed resources are human-friendly when expressed in plaintext form.
-
-#### Wallet implementer steps: verifying the message format
-The full SIWE message MUST be checked for conformance to the ABNF defined in ABNF Message Format.
-
-Wallet implementers SHOULD warn users if the substring `"wants you to sign in with your Ethereum account"` appears anywhere in an [ERC-191](https://eips.ethereum.org/EIPS/eip-191) message signing request unless the message fully conforms to the format defined ABNF Message Format.
-
-#### Wallet implementer steps: verifying the request origin
-Wallet implementers MUST prevent phishing attacks by verifying the origin of the request against the `scheme` and `domain` fields in the SIWE Message. 
-
-The origin SHOULD be read from a trusted data source such as the browser window or over WalletConnect [ERC-1328](https://eips.ethereum.org/EIPS/eip-1328) sessions for comparison against the signing message contents.
-
-Wallet implementers MAY warn instead of rejecting the verification if the origin is pointing to localhost.
-
-The following is a RECOMMENDED algorithm for Wallets to conform with the requirements on request origin verification defined by this specification.
-
-The algorithm takes the following input variables:
-
-- fields from the SIWE message.
-- `origin` of the signing request: the origin of the page which requested the signin via the provider.
-- `allowedSchemes`: a list of schemes allowed by the Wallet.
-- `defaultScheme`: a scheme to assume when none was provided. Wallet implementers in the browser SHOULD use https.
-- developer mode indication: a setting deciding if certain risks should be a warning instead of rejection. Can be manually configured or derived from `origin` being localhost.
-
-The algorithm is described as follows:
-
-- If `scheme` was not provided, then assign `defaultScheme` as scheme.
-- If `scheme` is not contained in `allowedSchemes`, then the `scheme` is not expected and the Wallet MUST reject the request. 
-Wallet implementers in the browser SHOULD limit the list of allowedSchemes to just 'https' unless a developer mode is activated.
-- If `scheme` does not match the scheme of origin, the Wallet SHOULD reject the request. 
-Wallet implementers MAY show a warning instead of rejecting the request if a developer mode is activated. 
-In that case the Wallet continues processing the request.
-- If the `host` part of the `domain` and `origin` do not match, the Wallet MUST reject the request unless the Wallet is in developer mode. 
-In developer mode the Wallet MAY show a warning instead and continues procesing the request.
-- If `domain` and `origin` have mismatching subdomains, the Wallet SHOULD reject the request unless the Wallet is in developer mode. 
-In developer mode the Wallet MAY show a warning instead and continues procesing the request.
-- Let `port` be the port component of `domain`, and if no port is contained in domain, assign port the default port specified for the scheme.
-- If `port` is not empty, then the Wallet SHOULD show a warning if the `port` does not match the port of `origin`.
-- If `port` is empty, then the Wallet MAY show a warning if `origin` contains a specific port.
-- Return request origin verification completed.
-
-#### Wallet implementer steps: creating SIWE interfaces
-Wallet implementers MUST display to the user the following fields from the SIWE Message request by default and prior to signing, if they are present: `scheme`, `domain`, `address`, `statement`, and `resources`. 
-Other present fields MUST also be made available to the user prior to signing either by default or through an extended interface.
-
-Wallet implementers displaying a plaintext SIWE Message to the user SHOULD require the user to scroll to the bottom of the text area prior to signing.
-
-Wallet implementers MAY construct a custom SIWE user interface by parsing the ABNF terms into data elements for use in the interface. 
-The display rules above still apply to custom interfaces.
-
-#### Wallet implementer steps: supporting internationalization (i18n)
-After successfully parsing the message into ABNF terms, translation MAY happen at the UX level per human language.
-
-## Privacy and Security Considerations
-- The double ratchet "recommends" using AES in CBC mode. Since encryption must be with an AEAD encryption scheme, we will use AES in GCM mode instead (supported by Noise).
-- For the information retrieval, the algorithm MUST include a access control mechanisms to restrict who can call the set and get functions.
-- One SHOULD include event logs to track changes in public keys.
-- The curve vurve448 MUST be chosen due to its higher security level: 224-bit security instead of the 128-bit security provided by X25519.
-- It is important that Bob MUST NOT reuse `SPK`.
-
-## Copyright
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-# References
-- [Augmented BNF for Syntax Specifications](https://datatracker.ietf.org/doc/html/rfc5234)
-- [Gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub)
-- [HMAC-based Extract-and-Expand Key Derivation Function](https://www.ietf.org/rfc/rfc5869.txt)
-- [Hybrid Public Key Encryption](https://datatracker.ietf.org/doc/rfc9180/)
-- [Security Analysis and Improvements for the IETF MLS Standard for Group Messaging](https://eprint.iacr.org/2019/1189.pdf)
-- [Signed Data Standard](https://eips.ethereum.org/EIPS/eip-191)
-- [Sign-In with Ethereum](https://eips.ethereum.org/EIPS/eip-4361)
-- [Standard Signature Validation Method for Contracts](https://eips.ethereum.org/EIPS/eip-1271)
-- [The Double Ratchet Algorithm](https://signal.org/docs/specifications/doubleratchet/)
-- [The Messaging Layer Security Protocol](https://datatracker.ietf.org/doc/rfc9420/)
-- [The X3DH Key Agreement Protocol](https://signal.org/docs/specifications/x3dh/)
-- [Toy Ethereum Private Messaging Protocol](https://rfc.vac.dev/spec/20/)
-- [Uniform Resource Identifier](https://datatracker.ietf.org/doc/html/rfc3986)
-- [WalletConnect URI Format](https://eips.ethereum.org/EIPS/eip-1328)

vac/README.md

@@ -1,9 +1,9 @@
-# Vac RFCs
-
-Vac builds public good protocols for the decentralise web.
-Vac acts as a custodian for the protocols that live in the RFC-Index repository.
-With the goal of widespread adoption,
-Vac will make sure the protocols adhere to the set of principles, 
-including but not limited to liberty, security, privacy, decentralisation, and inclusivity.
-
-To learn more, visit [Vac Research](https://vac.dev/)
+# Vac RFCs
+
+Vac builds public good protocols for the decentralise web.
+Vac acts as a custodian for the protocols that live in the RFC-Index repository.
+With the goal of widespread adoption,
+Vac will make sure the protocols adhere to the set of principles, 
+including but not limited to liberty, security, privacy, decentralisation, and inclusivity.
+
+To learn more, visit [Vac Research](https://vac.dev/)

vac/raw/README.md

@@ -1,4 +1,4 @@
-# Vac Raw Specifications
-
-All Vac specifications that have not reached **draft** status will live in this repository.
-To learn more about **raw** specifications, take a look at [1/COSS](../1/coss.md).
+# Vac Raw Specifications
+
+All Vac specifications that have not reached **draft** status will live in this repository.
+To learn more about **raw** specifications, take a look at [1/COSS](../1/coss.md).

vac/raw/decentralized-messaging-ethereum.md

@@ -0,0 +1,858 @@
+---
+title: VAC-DECENTRALIZED-MESSAGING-ETHEREUM
+name: Decentralized Key and Session Setup for Secure Messaging over Ethereum
+status: raw
+category: informational
+editor: Ramses Fernandez-Valencia <ramses@status.im>
+contributors:
+---
+
+## Abstract
+
+This document introduces a decentralized group messaging protocol
+using Ethereum adresses as identifiers.
+It is based in the proposal
+[DCGKA](https://eprint.iacr.org/2020/1281) by Weidner et al.
+It includes also approximations to overcome limitations related to using PKI and
+the multi-device setting.
+
+## Motivation
+
+The need for secure communications has become paramount.
+Traditional centralized messaging protocols are susceptible to various security threats,
+including unauthorized access, data breaches, and single points of failure.
+Therefore a decentralized approach to secure communication becomes increasingly relevant,
+offering a robust solution to address these challenges.
+
+Secure messaging protocols used should have the following key features:
+
+1. **Asynchronous Messaging:** Users can send messages even if
+the recipients are not online at the moment.
+2. **Resilience to Compromise:** If a user's security is compromised,
+the protocol ensures that previous messages remain secure
+through forward secrecy (FS).
+This means that messages sent before the compromise cannot be decrypted by adversaries.
+Additionally, the protocol maintains post-compromise security (PCS)
+by regularly updating keys,
+making it difficult for adversaries to decrypt future communication.
+3. **Dynamic Group Management:** Users can easily add or
+remove group members at any time,
+reflecting the flexible nature of communication within the app.
+
+In this field, there exists a *trilemma*, similar to what one observes in blockchain,
+involving three key aspects:
+
+1. security,
+2. scalability, and
+3. decentralization.
+
+For instance, protocols like the [MLS](https://messaginglayersecurity.rocks)
+perform well in terms of scalability and security.
+However, they falls short in decentralization.
+
+Newer studies such as [CoCoa](https://eprint.iacr.org/2022/251)
+improve features related to security and scalability,
+but they still rely on servers, which may not be fully trusted though they are necessary.
+
+On the other hand,
+older studies like [Causal TreeKEM](https://mattweidner.com/assets/pdf/acs-dissertation.pdf)
+exhibit decent scalability (logarithmic)
+but lack forward secrecy and have weak post-compromise security (PCS).
+
+The creators of [DCGKA](https://eprint.iacr.org/2020/1281) introduce a decentralized,
+asynchronous secure group messaging protocol that supports dynamic groups.
+This protocol operates effectively on various underlying networks
+without strict requirements on message ordering or latency.
+It can be implemented in peer-to-peer or anonymity networks,
+accommodating network partitions, high latency links, and
+disconnected operation seamlessly.
+Notably, the protocol doesn't rely on servers or
+a consensus protocol for its functionality.
+
+This proposal provides end-to-end encryption with forward secrecy and
+post-compromise security,
+even when multiple users concurrently modify the group state.
+
+## Theory
+
+### Protocol overview
+
+This protocol makes use of ratchets to provide FS
+by encrypting each message with a different key.
+
+In the figure one can see the ratchet for encrypting a sequence of messages.
+The sender requires an initial update secret `I_1`, which is introduced in a PRG.
+The PRG will produce two outputs, namely a symmetric key for AEAD encryption, and
+a seed for the next ratchet state.
+The associated data needed in the AEAD encryption includes the message index `i`.
+The ciphertext `c_i` associated to message `m_i`
+is then broadcasted to all group members.
+The next step requires deleting `I_1`, `k_i` and any old ratchet state.
+
+After a period of time the sender may replace the ratchet state with new update secrets
+`I_2`, `I_3`, and so on.
+
+To start a post-compromise security update,
+a user creates a new random value known as a seed secret and
+shares it with every other group member through a secure two-party channel.
+Upon receiving the seed secret,
+each group member uses it to calculate an update secret for both the sender's ratchet
+and their own.
+Additionally, the recipient sends an unencrypted acknowledgment to the group
+confirming the update.
+Every member who receives the acknowledgment updates
+not only the ratchet for the original sender but
+also the ratchet for the sender of the acknowledgment.
+Consequently, after sharing the seed secret through `n - 1` two-party messages and
+confirming it with `n - 1` broadcast acknowledgments,
+every group member has derived an update secret and updated their ratchet accordingly.
+
+When removing a group member,
+the user who initiates the removal conducts a post-compromise security update
+by sending the update secret to all group members except the one being removed.
+To add a new group member,
+each existing group member shares the necessary state with the new user,
+enabling them to derive their future update secrets.
+
+Since group members may receive messages in various orders,
+it's important to ensure that each sender's ratchet is updated consistently
+with the same sequence of update secrets at each group member.
+
+The network protocol used in this scheme ensures that messages from the same sender
+are processed in the order they were sent.
+
+### Components of the protocol
+
+This protocol relies in 3 components:
+authenticated causal broadcast (ACB),
+decentralized group membership (DGM) and
+2-party secure messaging (2SM).
+
+#### Authenticated causal broadcast
+
+A causal order is a partial order relation `<` on messages.
+Two messages `m_1` and `m_2` are causally ordered, or
+`m_1` causally precedes `m_2`
+(denoted by `m_1 < m_2`), if one of the following contiditions hold:
+
+1. `m_1` and `m_2` were sent by the same group member, and
+`m_1` was sent before `m_2`.
+2. `m_2` was sent by a group member U, and `m_1` was received and
+processed by `U` before sending `m_2`.
+3. There exists `m_3` such that `m_1 < m_3` and `m_3 < m_2`.
+
+Causal broadcast requires that before processing `m`,
+a group member must process all preceding messages `{m' | m' < m}`.
+
+The causal broadcast module used in this protocol
+authenticates the sender of each message,
+as well as its causal ordering metadata,
+using a digital signature under the sender’s identity key.
+This prevents a passive adversary from impersonating users or
+affecting causally ordered delivery.
+
+#### Decentralized group membership
+
+This protocol assumes the existence of a decentralized group membership function
+(denoted as DGM) that takes a set of membership change messages and
+their causal order relantionships,
+and returns the current set of group members’ IDs.
+It needs to be deterministic and depend only on causal order, and not exact order.
+
+#### 2-party secure messaging (2SM)
+
+This protocol makes use of bidirectional 2-party secure messaging schemes,
+which consist of 3 algorithms: `2SM-Init`, `2SM-Send` and `2SM-Receive`.
+
+##### 2SM-Init
+
+This function takes two IDs as inputs:
+`ID1` representing the local user and `ID2` representing the other party.
+It returns an initial protocol state `sigma`.
+The 2SM protocol relies on a Public Key Infrastructure (PKI) or
+a key server to map these IDs to their corresponding public keys.
+In practice, the PKI should incorporate ephemeral prekeys.
+This allows users to send messages to a new group member,
+even if that member is currently offline.
+
+##### 2SM-Send
+
+This function takes a state `sigma` and a plaintext `m` as inputs, and
+returns a new state `sigma’` and a ciphertext `c`.
+
+##### 2SM-Receive
+
+This function takes a state `sigma` and a ciphertext `c`, and
+returns a new state `sigma’` and a plaintext `m`.
+
+#### 2SM Syntax
+
+The variable `sigma` denotes the state consisting in the variables below:
+
+```rust
+sigma.mySks[0] = sk
+sigma.nextIndex = 1 
+sigma.receivedSk = empty_string
+sigma.otherPk = pk`<br> 
+sigma.otherPksender = “other”
+sigma.otherPkIndex = 0
+```
+
+#### The 2SM-Init
+
+On input a key pair `(sk, pk)`, this functions otuputs a state `sigma`.
+
+#### The 2SM-Send
+
+This function encrypts the message `m` using `sigma.otherPk`,
+which represents the other party’s current public key.
+This key is determined based on the last public key generated
+for the other party or the last public key received from the other party,
+whichever is more recent.
+`sigma.otherPkSender` is set to `me` in the former case and
+`other` in the latter case.
+
+Metadata including `otherPkSender` and
+`otherPkIndex` are included in the message to indicate
+which of the recipient’s public keys is being utilized.
+
+Additionally, this function generates a new key pair for the local user,
+storing the secret key in `sigma.mySks` and sending the public key.
+Similarly, it generates a new key pair for the other party,
+sending the secret key (encrypted) and storing the public key in `sigma.otherPk`.
+
+```go
+sigma.mySks[sigma.nextIndex], myNewPk) = PKE-Gen()
+(otherNewSk, otherNewPk) = PKE-Gen()
+plaintext = (m, otherNewSk, sigma`.nextIndex, myNewPk)
+msg = (PKE-Enc(sigma.otherPk, plaintext), sigma.otherPkSender, sigma.otherPkIndex)
+sigma.nextIndex++
+(sigma.otherPk, sigma.otherPkSender, sigma.otherPkIndex) = (otherNewPk, "me", empty_string)
+return (sigma`, msg)
+```
+
+#### The 2SM-Receive
+
+This function utilizes the metadata of the message `c`
+to determine which secret key to utilize for decryption,
+assigning it to `sk`.
+If the secret key corresponds to one generated by ourselves,
+that secret key along with all keys with lower index are deleted.
+This deletion is indicated by `sigma.mySks[≤ keyIndex] = empty_string`.
+Subsequently, the new public and secret keys contained in the message are stored.
+
+```go
+(ciphertext, keySender, keyIndex) = c
+if keySender = "other" then 
+sk = sigma.mySks[keyIndex] 
+sigma.mySks[≤ keyIndex] = empty_string
+else sk = sigma.receivedSk
+(m, sigma.receivedSk, sigma.otherPkIndex, sigma.otherPk) = PKE-Dec(sk, ciphertext)
+sigma.otherPkSender = "other"
+return (sigma, m)
+```
+
+### PKE Syntax
+
+The required PKE that MUST be used is ElGamal with a 2048-bit modulus `p`.
+
+#### Parameters
+
+The following parameters must be used:
+
+```go
+p = 308920927247127345254346920820166145569
+g = 2
+```
+
+#### PKE-KGen
+
+Each user `u` MUST do the following:
+
+```go
+PKE-KGen():
+a = randint(2, p-2)
+pk = (p, g, g^a)
+sk = a
+return (pk, sk)
+```
+
+#### PKE-Enc
+
+A user `v` encrypting a message `m` for `u` MUST follow these steps:
+
+```go
+PKE-Enc(pk):
+k = randint(2, p-2)
+eta = g^k % p
+delta = m * (g^a)^k % p
+return ((eta, delta))
+```
+
+#### PKE-Dec
+
+The user `u` recovers a message `m` from a ciphertext `c`
+by performing the following operations:
+
+```go
+PKE-Dec(sk):
+mu = eta^(p-1-sk) % p
+return ((mu * delta) % p)
+```
+
+### DCGKA Syntax
+
+#### Auxiliary functions
+
+There exist 6 functions that are auxiliary for the rest of components of the protocol,
+namely:
+
+#### init
+
+This function takes an `ID` as input and returns its associated initial state,
+denoted by `gamma`:
+
+```go
+gamma.myId = ID
+gamma.mySeq = 0
+gamma.history = empty
+gamma.nextSeed = empty_string
+gamma.2sm[·] = empty_string
+gamma.memberSecret[·, ·, ·] = empty_string
+gamma.ratchet[·] = empty_string
+return (gamma)
+```
+
+#### encrypt-to
+
+Upon reception of the recipient’s `ID` and a plaintext,
+it encrypts a direct message for another group member.
+Should it be the first message for a particular `ID`,
+then the `2SM` protocol state is initialized and stored in `gamma.2sm[recipient.ID]`.
+One then uses `2SM_Send` to encrypt the message and
+store the updated protocol in `gamma`.
+
+```go
+if gamma.2sm[recipient_ID] = empty_string then
+ gamma.2sm[recipient_ID] = 2SM_Init(gamma.myID, recipient_ID)
+(gamma.2sm[recipient_ID], ciphertext) = 2SM_Send(gamma.2sm[recipient_ID], plaintext)
+return (gamma, ciphertext)
+```
+
+#### decrypt-from
+
+After receiving the sender’s `ID` and a ciphertext,
+it behaves as the reverse function of `encrypt-to` and has a similar initialization:
+
+```go
+if gamma.2sm[sender_ID] = empty_string then
+gamma.2sm[sender_ID] = 2SM_Init(gamma.myID, sender_ID)
+(gamma.2sm[sender_ID], plaintext) = 2SM_Receive(gamma.2sm[sender_ID], ciphertext)
+return (gamma, plaintext)
+```
+
+#### update-ratchet
+
+This function generates the next update secret `I_update` for the group member `ID`.
+The ratchet state is stored in `gamma.ratchet[ID]`.
+It is required to use a HMAC-based key derivation function HKDF
+to combine the ratchet state with an input,
+returning an update secret and a new ratchet state.
+
+```go
+(updateSecret, gamma.ratchet[ID]) = HKDF(gamma.ratchet[ID], input)
+return (gamma, updateSecret)
+```
+
+#### member-view
+
+This function calculates the set of group members
+based on the most recent control message sent by the specified user `ID`.
+It filters the group membership operations
+to include only those observed by the specified `ID`, and
+then invokes the DGM function to generate the group membership.
+
+```go
+ops = {m in gamma.history st. m was sent or acknowledged by ID}
+return DGM(ops)
+```
+
+#### generate-seed
+
+This functions generates a random bit string and
+sends it encrypted to each member of the group using the `2SM` mechanism.
+It returns the updated protocol state and
+the set of direct messages (denoted as `dmsgs`) to send.
+
+```go
+gamma.nextSeed = random.randbytes()
+dmsgs = empty
+for each ID in recipients:
+(gamma, msg) = encrypt-to(gamma, ID, gamma.nextSeed)
+dmsgs = dmsgs + (ID, msg)
+return (gamma, dmsgs)
+```
+
+### Creation of a group
+
+A group is generated in a 3 steps procedure:
+
+1. A user calls the `create` function and broadcasts a control message of type *create*.
+2. Each receiver of the message processes the message and
+broadcasts an *ack* control message.
+3. Each member processes the *ack* message received.
+
+#### create
+
+This function generates a *create* control message and
+calls `generate-seed` to define the set of direct messages that need to be sent.
+Then it calls `process-create` to process the control message for this user.
+The function `process-create` returns a tuple including an updated state gamma and
+an update secret `I`.
+
+```go
+control = (“create”, gamma.mySeq, IDs)
+(gamma, dmsgs) = generate-seed(gamma, IDs)
+(gamma, _, _, I, _) = process-create(gamma, gamma.myId, gamma.mySeq, IDs, empty_string)
+return (gamma, control, dmsgs, I)
+```
+
+#### process-seed
+
+This function initially employs `member-view` to
+identify the users who were part of the group when the control message was dispatched.
+Then, it attempts to acquire the seed secret through the following steps:
+
+1. If the control message was dispatched by the local user,
+it uses the most recent invocation of `generate-seed` stored the seed secret in `gamma.nextSeed`.
+2. If the `control` message was dispatched by another user, and
+the local user is among its recipients,
+the function utilizes `decrypt-from` to decrypt the direct message
+that includes the seed secret.
+3. Otherwise, it returns an `ack` message without deriving an update secret.
+
+Afterwards, `process-seed` generates separate member secrets
+for each group member from the seed secret by combining the seed secret and
+each user ID using HKDF.
+The secret for the sender of the message is stored in `senderSecret`,
+while those for the other group members are stored in `gamma.memberSecret`.
+The sender's member secret is immediately utilized to update their KDF ratchet and
+compute their update secret `I_sender` using `update-ratchet`.
+If the local user is the sender of the control message,
+the process is completed, and the update secret is returned.
+However, if the seed secret is received from another user,
+an `ack` control message is constructed for broadcast,
+including the sender ID and sequence number of the message being acknowledged.
+
+The final step computes an update secret `I_me`
+for the local user invoking the `process-ack` function.
+
+```go
+recipients = member-view(gamma, sender) - {sender}
+if sender =  gamma.myId then seed = gamma.nextSeed; gamma.nextSeed = empty_string
+else if  gamma.myId in recipients then (gamma, seed) = decrypt-from(gamma, sender, dmsg)
+else
+return (gamma, (ack, ++gamma.mySeq, (sender, seq)), empty_string , empty_string , empty_string)
+
+for ID in recipients do gamma.memberSecret[sender, seq, ID] = HKDF(seed, ID)
+
+senderSecret = HKDF(seed, sender)
+(gamma, I_sender) = update-ratchet(gamma, sender, senderSecret)
+if sender = gamma.myId then return (gamma, empty_string , empty_string , I_sender, empty_string)
+control = (ack, ++gamma.mySeq, (sender, seq))
+members = member-view(gamma, gamma.myId)
+forward = empty
+for ID in {members - (recipients + {sender})}
+ s = gamma.memberSecret[sender, seq, gamma.myId]
+ (gamma, msg) = encrypt-to(gamma, ID, s)
+ forward = forward + {(ID, msg)}
+
+(gamma, _, _, I_me, _) = process-ack(gamma, gamma.myId, gamma.mySeq, (sender, seq), empty_string)
+return (gamma, control, forward, I_sender, I_me)
+```
+
+#### process-create
+
+This function is called by the sender and
+each of the receivers of the `create` control message.
+First, it records the information from the create message
+in the `gamma.history+ {op}`, which is used to track group membership changes.
+Then, it proceeds to call `process-seed`.
+
+```go
+op = (”create”, sender, seq, IDs)
+gamma.history = gamma.history + {op}
+return (process-seed(gamma, sender, seq, dmsg))
+```
+
+#### process-ack
+
+This function is called by those group members once they receive an ack message.
+In `process-ack`, `ackID` and `ackSeq` are the sender and
+sequence number of the acknowledged message.
+Firstly, if the acknowledged message is a group membership operation,
+it records the acknowledgement in `gamma.history`.
+
+Following this, the function retrieves the relevant member secret from `gamma.memberSecret`,
+which was previously obtained from the seed secret
+contained in the acknowledged message.
+
+Finally, it updates the ratchet for the sender of the `ack` and
+returns the resulting update secret.
+
+```go
+if (ackID, ackSeq) was a create / add / remove then
+op = ("ack", sender, seq, ackID, ackSeq)
+gamma.history = gamma.history + {op}`
+s = gamma.memberSecret[ackID, ackSeq, sender]
+gamma.memberSecret[ackID, ackSeq, sender] = empty_string
+if (s = empty_string) & (dmsg = empty_string) then return (gamma, empty_string, empty_string, empty_string, empty_string)
+if (s = empty_string) then (gamma, s) = decrypt-from(gamma, sender, dmsg)
+(gamma, I) = update-ratchet(gamma, sender, s)
+return (gamma, empty_string, empty_string, I, empty_string)
+```
+
+The HKDF function MUST follow RFC 5869 using the hash function SHA256.
+
+### Post-compromise security updates and group member removal
+
+The functions `update` and `remove` share similarities with `create`:
+they both call the function `generate-seed`
+to encrypt a new seed secret for each group member.
+The distinction lies in the determination of the group members using `member-view`.
+In the case of `remove`,
+the user being removed is excluded from the recipients of the seed secret.
+Additionally, the control message they construct is designated with type `update`
+ or `remove` respectively.
+
+Likewise, `process-update` and `process-remove` are akin to `process-create`.
+The function `process-update` skips the update of `gamma.history`,
+whereas `process-remove` includes a removal operation in the history.
+
+#### update
+
+```go
+control = ("update", ++gamma.mySeq, empty_string)
+recipients = member-view(gamma, gamma.myId) - {gamma.myId}
+(gamma, dmsgs) = generate-seed(gamma, recipients)
+(gamma, _, _, I , _) = process-update(gamma, gamma.myId, gamma.mySeq, empty_string, empty_string)
+return (gamma, control, dmsgs, I)
+```
+
+#### remove
+
+```go
+control = ("remove", ++gamma.mySeq, empty)
+recipients = member-view(gamma, gamma.myId) - {ID, gamma.myId}
+(gamma, dmsgs) = generate-seed(gamma, recipients)
+(gamma, _, _, I , _) = process-update(gamma, gamma.myId, gamma.mySeq, ID, empty_string)
+return (gamma, control, dmsgs, I)
+```
+
+#### process-update
+
+`return process-seed(gamma, sender, seq, dmsg)`
+
+#### process-remove
+
+```go
+op = ("remove", sender, seq, removed)
+gamma.history = gamma.history + {op}
+return process-seed(gamma, sender, seq, dmsg)
+```
+
+### Group member addition
+
+#### add
+
+When adding a new group member,
+an existing member initiates the process by invoking the `add` function and
+providing the ID of the user to be added.
+This function prepares a control message marked as `add` for broadcast to the group.
+Simultaneously, it creates a welcome message intended for the new member
+as a direct message.
+This `welcome` message includes the current state of the sender's KDF ratchet,
+encrypted using `2SM`,
+along with the history of group membership operations conducted so far.
+
+```go
+control = ("add", ++gamma.mySeq, ID)
+(gamma, c) = encrypt-to(gamma, ID, gamma.ratchet[gamma.myId])
+op = ("add", gamma.myId, gamma.mySeq, ID)
+welcome = (gamma.history + {op}, c)
+(gamma, _, _, I, _) = process-add(gamma, gamma.myId, gamma.mySeq, ID, empty_string)
+return (gamma, control, (ID, welcome), I)
+```
+
+#### process-add
+
+This function is invoked by both the sender and
+each recipient of an `add` message, which includes the new group member.
+If the local user is the newly added member,
+the function proceeds to call `process-welcome` and then exits.
+Otherwise, it extends `gamma.history` with the `add` operation.
+
+Line 5 determines whether the local user was already a group member
+at the time the `add` message was sent;
+this condition is typically true but may be false if multiple users were added concurrently.
+
+On lines 6 to 8, the ratchet for the sender of the *add* message is updated twice.
+In both calls to `update-ratchet`,
+a constant string is used as the ratchet input instead of a random seed secret.
+
+The value returned by the first ratchet update is stored in `gamma.memberSecret`
+as the added user’s initial member secret.
+The result of the second ratchet update becomes `I_sender`,
+the update secret for the sender of the `add` message.
+On line 10, if the local user is the sender, the update secret is returned.
+
+If the local user is not the sender, an acknowledgment for the `add` message is required.
+Therefore, on line 11, a control message of type `add-ack` is constructed for broadcast.
+Subsequently, in line 12 the current ratchet state is encrypted using `2SM`
+to generate a direct message intended for the added user,
+allowing them to decrypt subsequent messages sent by the sender.
+Finally, in lines 13 to 15,
+`process-add-ack` is called to calculate the local user’s update secret (`I_me`),
+which is then returned along with `I_sender`.
+
+```go
+if added = gamma.myId then return process-welcome(gamma, sender, seq, dmsg)
+op = ("add", sender, seq, added)
+gamma.history = gamma.history + {op}
+if gamma.myId in member-view(gamma, sender) then
+ (gamma, s) = update-ratchet(gamma, sender, "welcome")
+ gamma.memberSecret[sender, seq, added] = s
+ (gamma, I_sender) = update-ratchet(gamma, sender, "add")
+else I_sender = empty_string
+if sender = gamma.myId then return (gamma, empty_string, empty_string, I_sender, empty_string)
+control = ("add-ack", ++gamma.mySeq, (sender, seq))
+(gamma, c) = encrypt-to(gamma, added, ratchet[gamma.myId])
+(gamma, _, _, I_me, _) = process-add-ack(gamma, gamma.myId, gamma.mySeq, (sender, seq), empty_string)
+return (gamma, control, {(added, c)}, I_sender, I_me)
+```
+
+#### process-add-ack
+
+This function is invoked by both the sender and each recipient of an `add-ack` message,
+including the new group member.
+Upon lines 1–2, the acknowledgment is added to `gamma.history`,
+mirroring the process in `process-ack`.
+If the current user is the new group member,
+the `add-ack` message includes the direct message constructed in `process-add`;
+this direct message contains the encrypted ratchet state of the sender of the `add-ack`,
+then it is decrypted on lines 3–5.
+
+Upon line 6,
+a check is performed to check if the local user
+was already a group member at the time the `add-ack` was sent.
+If affirmative, a new update secret `I` for the sender of the `add-ack`
+is computed on line 7 by invoking `update-ratchet` with the constant string `add`.
+
+In the scenario involving the new member,
+the ratchet state was recently initialized on line 5.
+This ratchet update facilitates all group members, including the new addition,
+to derive each member’s update by obtaining any update secret from before their inclusion.
+
+```go
+op = ("ack", sender, seq, ackID, ackSeq)
+gamma$.history = gamma.history + {op}
+if dmsg != empty_string then
+ (gamma, s) = decrypt-from(gamma, sender, dmsg)
+ gamma.ratchet[sender] = s
+if gamma.myId in member-view(gamma, sender) then
+ (gamma, I) = update-ratchet(gamma, sender, "add")
+ return (gamma, empty_string, empty_string, I, empty_string)
+else return (gamma, empty_string, empty_string, empty_string, empty_string)
+```
+
+#### process-welcome
+
+This function serves as the second step called by a newly added group member.
+In this context, `adderHistory` represents the adding user’s copy of `gamma.history`
+sent in their welcome message,
+which is utilized to initialize the added user’s history.
+Here, `c` denotes the ciphertext of the adding user’s ratchet state,
+which is decrypted on line 2 using `decrypt-from`.
+
+Once `gamma.ratchet[sender]` is initialized,
+`update-ratchet` is invoked twice on lines 3 to 5 with the constant strings `welcome`
+and `add` respectively.
+These operations mirror the ratchet operations
+performed by every other group member in `process-add`.
+The outcome of the first `update-ratchet` call
+becomes the first member secret for the added user,
+while the second call returns `I_sender`,
+the update secret for the sender of the add operation.
+
+Subsequently, the new group member constructs an *ack* control message
+to broadcast on line 6 and
+calls `process-ack` to compute their initial update secret I_me.
+The function `process-ack` reads from `gamma.memberSecret` and
+passes it to `update-ratchet`.
+The previous ratchet state for the new member is the empty string `empty`,
+as established by `init`,
+thereby initializing the new member’s ratchet.
+Upon receiving the new member’s `ack`,
+every other group member initializes their copy of the new member’s ratchet
+in a similar manner.
+
+By the conclusion of `process-welcome`,
+the new group member has acquired update secrets for themselves and
+the user who added them.
+The ratchets for other group members are initialized by `process-add-ack`.
+
+```go
+gamma.history = adderHistory
+(gamma, gamma.ratchet[sender]) = decrypt-from(gamma, sender, c)
+(gamma, s) = update-ratchet(gamma, sender, "welcome")
+gamma.memberSecret[sender, seq, gamma.myId] = s
+(gamma, I_sender) = update-ratchet(gamma, sender, "add")
+control = ("ack", ++gamma.mySeq, (sender, seq))
+(gamma, _, _, I_me, _) = process-ack(gamma, gamma.myId, gamma.mySeq, (sender, seq), empty_string)
+return (gamma, control, empty_string , I_sender, I_me)
+```
+
+## Privacy Considerations
+
+### Dependency on PKI
+
+The [DCGKA](https://eprint.iacr.org/2020/1281)
+proposal presents some limitations highlighted by the authors.
+Among these limitations one finds the requirement of a PKI
+(or a key server) mapping IDs to public keys.
+
+One method to overcome this limitation is adapting the protocol SIWE
+(Sign in with Ethereum) so a user `u_1` who wants to start a communication
+with a user `u_2` can interact with latter’s wallet to request a public key
+using an Ethereum address as `ID`.
+
+#### SIWE
+
+The [SIWE](https://docs.login.xyz/general-information/siwe-overview)
+(Sign In With Ethereum) proposal was a suggested standard for leveraging Ethereum
+to authenticate and authorize users on web3 applications.
+Its goal is to establish a standardized method for users
+to sign in to web3 applications using their Ethereum address and private key,
+mirroring the process by which users currently sign in to web2 applications
+using their email and password.
+Below follows the required steps:
+
+1. A server generates a unique Nonce for each user intending to sign in.
+2. A user initiates a request to connect to a website using their wallet.
+3. The user is presented with a distinctive message that includes the Nonce and
+details about the website.
+4. The user authenticates their identity by signing in with their wallet.
+5. Upon successful authentication, the user's identity is confirmed or approved.
+6. The website grants access to data specific to the authenticated user.
+
+#### Our approach
+
+The idea in the [DCGKA](https://eprint.iacr.org/2020/1281)
+setting closely resembles the procedure outlined in SIWE. Here:
+
+1. The server corresponds to user D1,
+who initiates a request (instead of generating a nonce)
+to obtain the public key of user D2.
+2. Upon receiving the request, the wallet of D2 send the request to the user,
+3. User D2 receives the request from the wallet, and decides whether accepts or rejects.
+4. The wallet and responds with a message containing the requested public key
+in case of acceptance by D2.
+
+This message may be signed,
+allowing D1 to verify that the owner of the received public key is indeed D2.
+
+### Multi-device setting
+
+One may see the set of devices as a group and create a group key for internal communications.
+One may use treeKEM for instance,
+since it provides interesting properties like forward secrecy and
+post-compromise security.
+All devices share the same `ID`,
+which is held by one of them, and from other user’s point of view,
+they would look as a single user.
+
+Using servers,
+like in the paper [Multi-Device for Signal](https://eprint.iacr.org/2019/1363),
+should be avoided; but this would imply using a particular device as receiver and
+broadcaster within the group.
+There is an obvious drawback which is having a single device working as a “server”.
+Should this device be attacked or without connection,
+there should be a mechanism for its revocation and replacement.
+
+Another approach for communications between devices
+could be using the keypair of each device.
+This could open the door to use UPKE, since keypairs should be regenerated frequently.
+
+Each time a device sends a message, either an internal message or an external message,
+it needs to replicate and broadcast it to all devices in the group.
+
+The mechanism for the substitution of misbehaving leader devices follows:
+
+1. Each device within a group knows the details of other leader devices.
+This information may come from metadata in received messages, and
+is replicated by the leader device.
+2. To replace a leader,
+the user should select any other device within its group and
+use it to send a signed message to all other users.
+3. To get the ability to sign messages,
+this new leader should request the keypair associated to the ID to the wallet.
+4. Once the leader has been changed,
+it revocates access from DCGKA to the former leader using the DCGKA protocol.
+5. The new leader starts a key update in DCGKA.
+
+Not all devices in a group should be able to send messages to other users.
+Only the leader device should be in charge of sending and receiving messages.
+To prevent other devices from sending messages outside their group,
+a requirement should be signing each message.
+The keys associated to the `ID` should only be in control of the leader device.
+
+The leader device is in charge of setting the keys involved in the DCGKA.
+This information must be replicated within the group to make sure it is updated.
+
+To detect missing messages or potential misbehavior, messages must include a counter.
+
+### Using UPKE
+
+Managing the group of devices of a user can be done either
+using a group key protocol such as treeKEM or
+using the keypair of each device.
+Setting a common key for a group of devices
+under the control of the same actor might be excessive,
+furthermore it may imply some of the problems one can find
+in the usual setting of a group of different users;
+for example: one of the devices may not participate in the required updating processes,
+representing a threat for the group.
+
+The other approach to managing the group of devices is using each device’s keypair,
+but it would require each device updating these materia frequently,
+something that may not happens.
+
+[UPKE](https://eprint.iacr.org/2022/068) is a form of asymetric cryptography
+where any user can update any other user’s key pair
+by running an update algorithm with (high-entropy) private coins.
+Any sender can initiate a *key update* by sending a special update ciphertext.
+This ciphertext updates the receiver’s public key and also,
+once processed by the receiver, will update their secret key.
+
+To the best of my knowledge,
+there exists several efficient constructions both [UPKE from ElGamal](https://eprint.iacr.org/2019/1189)
+(based in the DH assumption) and
+[UPKE from Lattices]((https://eprint.iacr.org/2023/1400)) (based in lattices).
+None of them have been implemented in a secure messaging protocol, and
+this opens the door to some novel research.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [DCGKA](https://eprint.iacr.org/2020/1281)
+- [MLS](https://messaginglayersecurity.rocks)
+- [CoCoa](https://eprint.iacr.org/2022/251)
+- [Causal TreeKEM](https://mattweidner.com/assets/pdf/acs-dissertation.pdf)
+- [SIWE](https://docs.login.xyz/general-information/siwe-overview)
+- [Multi-device for Signal](https://eprint.iacr.org/2019/1363)
+- [UPKE](https://eprint.iacr.org/2022/068)
+- [UPKE from ElGamal](https://eprint.iacr.org/2019/1189)
+- [UPKE from Lattices](https://eprint.iacr.org/2023/1400)

vac/raw/gossipsub-tor-push.md

@@ -1,185 +1,226 @@
----
-slug: 46
-title: 46/GOSSIPSUB-TOR-PUSH
-name: Gossipsub Tor Push
-status: raw
-category: Standards Track
-tags:
-editor: Daniel Kaiser <danielkaiser@status.im>
-contributors:
----
-
-## Abstract
-
-This document extends the [libp2p gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
-specifying gossipsub Tor Push,
-a gossipsub-internal way of pushing messages into a gossipsub network via Tor.
-Tor Push adds sender identity protection to gossipsub.
-
-**Protocol identifier**: /meshsub/1.1.0
-
-Note: Gossipsub Tor Push does not have a dedicated protocol identifier.
-It uses the same identifier as gossipsub and works with all [pubsub](https://github.com/libp2p/specs/tree/master/pubsub) based protocols.
-This allows nodes that are oblivious to Tor Push to process messages received via Tor Push.
-
-## Background
-
-Without extensions, [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
-does not protect sender identities.
-
-A possible design of an anonymity extension to gossipsub is pushing messages through an anonymization network before they enter the gossipsub network.
-[Tor](https://www.torproject.org/) is currently the largest anonymization network.
-It is well researched and works reliably.
-Basing our solution on Tor both inherits existing security research, as well as allows for a quick deployment.
-
-Using the anonymization network approach, even the first gossipsub node that relays a given message cannot link the message to its sender (within a relatively strong adversarial model).
-Taking the low bandwidth overhead and the low latency overhead into consideration, Tor offers very good anonymity properties.
-
-## Functional Operation
-
-Tor Push allows nodes to push messages over Tor into the gossipsub network.
-The approach specified in this document is fully backwards compatible.
-Gossipsub nodes that do not support Tor Push can receive and relay Tor Push messages,
-because Tor Push uses the same Protocol ID as gossipsub.
-
-Messages are sent over Tor via [SOCKS5](https://www.rfc-editor.org/rfc/rfc1928).
-Tor Push uses a dedicated libp2p context to prevent information leakage.
-To significantly increase resilience and mitigate circuit failures,
-Tor Push establishes several connections, each to a different randomly selected gossipsub node.
-
-## Specification
-
-This section specifies the format of Tor Push messages, as well as how Tor Push messages are received and sent, respectively.
-
-### Wire Format
-
-The wire format of a Tor Push message corresponds verbatim to a typical [libp2p pubsub message](https://github.com/libp2p/specs/tree/master/pubsub#the-message).
-
-```
-message Message {
-  optional string from = 1;
-  optional bytes data = 2;
-  optional bytes seqno = 3;
-  required string topic = 4;
-  optional bytes signature = 5;
-  optional bytes key = 6;
-}
-```
-
-### Receiving Tor Push Messages
-
-Any node supporting a protocol with ID `/meshsub/1.1.0` (e.g. gossipsub), can receive Tor Push messages.
-Receiving nodes are oblivious to Tor Push and will process incoming messages according to the respective `meshsub/1.1.0` specification.
-
-### Sending Tor Push Messages
-
-In the following, we refer to nodes sending Tor Push messages as Tp-nodes (Tor Push nodes).
-
-Tp-nodes MUST setup a separate libp2p context, i.e. [libp2p switch](https://docs.libp2p.io/concepts/multiplex/switch/),
-which MUST NOT be used for any purpose other than Tor Push.
-We refer to this context as Tp-context.
-The Tp-context MUST NOT share any data, e.g. peer lists, with the default context.
-
-Tp-peers are peers a Tp-node plans to send Tp-messages to.
-Tp-peers MUST support `/meshsub/1.1.0`.
-For retrieving Tp-peers, Tp-nodes SHOULD use an ambient peer discovery method that retrieves a random peer sample (from the set of all peers), e.g. [33/WAKU2-DISCV5](../../waku/standards/core/33/discv5.md).
-
-Tp-nodes MUST establish a connection as described in sub-section [Tor Push Connection Establishment](#connection-establishment) to at least one Tp-peer.
-To significantly increase resilience, Tp-nodes SHOULD establish Tp-connections to `D` peers,
-where `D` is the [desired gossipsub out-degree](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters),
-with a default value of `8`.
-
-Each Tp-message MUST be sent via the Tp-context over at least one Tp-connection.
-To increase resilience, Tp-messages SHOULD be sent via the Tp-context over all available Tp-connections.
-
-Control messages of any kind, e.g. gossipsub graft, MUST NOT be sent via Tor Push.
-
-#### Connection Establishment
-
-Tp-nodes establish a `/meshsub/1.1.0` connection to tp-peers via [SOCKS5](https://www.rfc-editor.org/rfc/rfc1928) over [Tor](https://www.torproject.org/).
-
-Establishing connections, which in turn establishes the respective Tor circuits, can be done ahead of time.
-
-#### Epochs
-
-Tor Push introduces epochs.
-The default epoch duration is 10 minutes.
-(We might adjust this default value based on experiments and evaluation in future versions of this document.
-It seems a good trade-off between traceablity and circuit building overhead.)
-
-For each epoch, the Tp-context SHOULD be refreshed, which includes
-
-* libp2p peer-ID
-* Tp-peer list
-* connections to Tp-peers
-
-Both Tp-peer selection for the next epoch and establishing connections to the newly selected peers SHOULD be done during the current epoch
-and be completed before the new epoch starts.
-This avoids adding latency to message transmission.
-
-## Security/Privacy Considerations
-
-### Fingerprinting Attacks
-
-Protocols that feature distinct patterns are prone to fingerprinting attacks when using them over Tor Push.
-Both malicious guards and exit nodes could detect these patterns
-and link the sender and receiver, respectively, to transmitted traffic.
-As a mitigation, such protocols can introduce dummy messages and/or padding to hide patterns.
-
-### DoS
-
-#### General DoS against Tor
-
-Using untargeted DoS to prevent Tor Push messages from entering the gossipsub network would cost vast resources,
-because Tor Push transmits messages over several circuits and the Tor network is well established.
-
-#### Targeting the Guard
-
-Denying the service of a specific guard node blocks Tp-nodes using the respective guard.
-Tor guard selection will replace this guard [TODO elaborate].
-Still, messages might be delayed during this window which might be critical to certain applications.
-
-#### Targeting the Gossipsub Network
-
-Without sophisticated rate limiting (for example using [17/WAKU2-RLN-RELAY](../../waku/standards/core/17/rln-relay.md)),
-attackers can spam the gossipsub network.
-It is not enough to just block peers that send too many messages,
-because these messages might actually come from a Tor exit node that many honest Tp-nodes use.
-Without Tor Push, protocols on top of gossipsub could block peers if they exceed a certain message rate.
-With Tor Push, this would allow the reputation-based DoS attack described in
-[Bitcoin over Tor isn't a Good Idea](https://ieeexplore.ieee.org/abstract/document/7163022).
-
-#### Peer Discovery
-
-The discovery mechanism could be abused to link requesting nodes to their Tor connections to discovered nodes.
-An attacker that controls both the node that responds to a discovery query,
-and the node who’s ENR the response contains,
-can link the requester to a Tor connection that is expected to be opened to the node represented by the returned ENR soon after.
-
-Further, the discovery mechanism (e.g. discv5) could be abused to distribute disproportionately many malicious nodes.
-For instance if p% of the nodes in the network are malicious,
-an attacker could manipulate the discovery to return malicious nodes with 2p% probability.
-The discovery mechanism needs to be resilient against this attack.
-
-### Roll-out Phase
-
-During the roll-out phase of Tor Push, during which only a few nodes use Tor Push,
-attackers can narrow down the senders of Tor messages to the set of gossipsub nodes that do not originate messages.
-Nodes who want anonymity guarantees even during the roll-out phase can use separate network interfaces for their default context and Tp-context, respectively.
-For the best protection, these contexts should run on separate physical machines.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-* [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
-* [libp2p pubsub](https://github.com/libp2p/specs/tree/master/pubsub)
-* [libp2p pubsub message](https://github.com/libp2p/specs/tree/master/pubsub#the-message)
-* [libp2p switch](https://docs.libp2p.io/concepts/multiplex/switch)
-* [SOCKS5](https://www.rfc-editor.org/rfc/rfc1928)
-* [Tor](https://www.torproject.org/)
-* [33/WAKU2-DISCV5](../../waku/standards/core/33/discv5.md)
-* [Bitcoin over Tor isn't a Good Idea](https://ieeexplore.ieee.org/abstract/document/7163022)
-* [17/WAKU2-RLN-RELAY](../../waku/standards/core/17/rln-relay.md)
+---
+title: GOSSIPSUB-TOR-PUSH
+name: Gossipsub Tor Push
+status: raw
+category: Standards Track
+tags:
+editor: Daniel Kaiser <danielkaiser@status.im>
+contributors:
+---
+
+## Abstract
+
+This document extends the [libp2p gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+specifying gossipsub Tor Push,
+a gossipsub-internal way of pushing messages into a gossipsub network via Tor.
+Tor Push adds sender identity protection to gossipsub.
+
+**Protocol identifier**: /meshsub/1.1.0
+
+Note: Gossipsub Tor Push does not have a dedicated protocol identifier.
+It uses the same identifier as gossipsub and
+works with all [pubsub](https://github.com/libp2p/specs/tree/master/pubsub)
+based protocols.
+This allows nodes that are oblivious to Tor Push to process messages received via
+Tor Push.
+
+## Background
+
+Without extensions, [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+does not protect sender identities.
+
+A possible design of an anonymity extension to gossipsub
+is pushing messages through an anonymization network
+before they enter the gossipsub network.
+[Tor](https://www.torproject.org/) is currently the largest anonymization network.
+It is well researched and works reliably.
+Basing our solution on Tor both inherits existing security research,
+as well as allows for a quick deployment.
+
+Using the anonymization network approach,
+even the first gossipsub node that relays a given message
+cannot link the message to its sender
+(within a relatively strong adversarial model).
+Taking the low bandwidth overhead and the low latency overhead into consideration,
+Tor offers very good anonymity properties.
+
+## Functional Operation
+
+Tor Push allows nodes to push messages over Tor into the gossipsub network.
+The approach specified in this document is fully backwards compatible.
+Gossipsub nodes that do not support Tor Push can receive and relay Tor Push messages,
+because Tor Push uses the same Protocol ID as gossipsub.
+
+Messages are sent over Tor via [SOCKS5](https://www.rfc-editor.org/rfc/rfc1928).
+Tor Push uses a dedicated libp2p context to prevent information leakage.
+To significantly increase resilience and mitigate circuit failures,
+Tor Push establishes several connections,
+each to a different randomly selected gossipsub node.
+
+## Specification
+
+This section specifies the format of Tor Push messages,
+as well as how Tor Push messages are received and sent, respectively.
+
+### Wire Format
+
+The wire format of a Tor Push message corresponds verbatim to a typical
+[libp2p pubsub message](https://github.com/libp2p/specs/tree/master/pubsub#the-message).
+
+```protobuf
+message Message {
+  optional string from = 1;
+  optional bytes data = 2;
+  optional bytes seqno = 3;
+  required string topic = 4;
+  optional bytes signature = 5;
+  optional bytes key = 6;
+}
+```
+
+### Receiving Tor Push Messages
+
+Any node supporting a protocol with ID `/meshsub/1.1.0` (e.g. gossipsub),
+can receive Tor Push messages.
+Receiving nodes are oblivious to Tor Push and
+will process incoming messages according to the respective `meshsub/1.1.0` specification.
+
+### Sending Tor Push Messages
+
+In the following, we refer to nodes sending Tor Push messages as Tp-nodes
+(Tor Push nodes).
+
+Tp-nodes MUST setup a separate libp2p context, i.e. [libp2p switch](https://docs.libp2p.io/concepts/multiplex/switch/),
+which MUST NOT be used for any purpose other than Tor Push.
+We refer to this context as Tp-context.
+The Tp-context MUST NOT share any data, e.g. peer lists, with the default context.
+
+Tp-peers are peers a Tp-node plans to send Tp-messages to.
+Tp-peers MUST support `/meshsub/1.1.0`.
+For retrieving Tp-peers,
+Tp-nodes SHOULD use an ambient peer discovery method
+that retrieves a random peer sample (from the set of all peers),
+e.g. [33/WAKU2-DISCV5](../../waku/standards/core/33/discv5.md).
+
+Tp-nodes MUST establish a connection as described in sub-section
+[Tor Push Connection Establishment](#connection-establishment) to at least one Tp-peer.
+To significantly increase resilience,
+Tp-nodes SHOULD establish Tp-connections to `D` peers,
+where `D` is the [desired gossipsub out-degree](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters),
+with a default value of `8`.
+
+Each Tp-message MUST be sent via the Tp-context over at least one Tp-connection.
+To increase resilience,
+Tp-messages SHOULD be sent via the Tp-context over all available Tp-connections.
+
+Control messages of any kind, e.g. gossipsub graft, MUST NOT be sent via Tor Push.
+
+#### Connection Establishment
+
+Tp-nodes establish a `/meshsub/1.1.0` connection to tp-peers via
+[SOCKS5](https://www.rfc-editor.org/rfc/rfc1928) over [Tor](https://www.torproject.org/).
+
+Establishing connections, which in turn establishes the respective Tor circuits,
+can be done ahead of time.
+
+#### Epochs
+
+Tor Push introduces epochs.
+The default epoch duration is 10 minutes.
+(We might adjust this default value based on experiments and
+evaluation in future versions of this document.
+It seems a good trade-off between traceablity and circuit building overhead.)
+
+For each epoch, the Tp-context SHOULD be refreshed, which includes
+
+* libp2p peer-ID
+* Tp-peer list
+* connections to Tp-peers
+
+Both Tp-peer selection for the next epoch and
+establishing connections to the newly selected peers
+SHOULD be done during the current epoch
+and be completed before the new epoch starts.
+This avoids adding latency to message transmission.
+
+## Security/Privacy Considerations
+
+### Fingerprinting Attacks
+
+Protocols that feature distinct patterns are prone to fingerprinting attacks
+when using them over Tor Push.
+Both malicious guards and exit nodes could detect these patterns
+and link the sender and receiver, respectively, to transmitted traffic.
+As a mitigation, such protocols can introduce dummy messages and/or
+padding to hide patterns.
+
+### DoS
+
+#### General DoS against Tor
+
+Using untargeted DoS to prevent Tor Push messages
+from entering the gossipsub network would cost vast resources,
+because Tor Push transmits messages over several circuits and
+the Tor network is well established.
+
+#### Targeting the Guard
+
+Denying the service of a specific guard node
+blocks Tp-nodes using the respective guard.
+Tor guard selection will replace this guard [TODO elaborate].
+Still, messages might be delayed during this window
+which might be critical to certain applications.
+
+#### Targeting the Gossipsub Network
+
+Without sophisticated rate limiting (for example using [17/WAKU2-RLN-RELAY](../../waku/standards/core/17/rln-relay.md)),
+attackers can spam the gossipsub network.
+It is not enough to just block peers that send too many messages,
+because these messages might actually come from a Tor exit node
+that many honest Tp-nodes use.
+Without Tor Push,
+protocols on top of gossipsub could block peers
+if they exceed a certain message rate.
+With Tor Push, this would allow the reputation-based DoS attack described in
+[Bitcoin over Tor isn't a Good Idea](https://ieeexplore.ieee.org/abstract/document/7163022).
+
+#### Peer Discovery
+
+The discovery mechanism could be abused to link requesting nodes
+to their Tor connections to discovered nodes.
+An attacker that controls both the node that responds to a discovery query,
+and the node who’s ENR the response contains,
+can link the requester to a Tor connection
+that is expected to be opened to the node represented by the returned ENR soon after.
+
+Further, the discovery mechanism (e.g. discv5)
+could be abused to distribute disproportionately many malicious nodes.
+For instance if p% of the nodes in the network are malicious,
+an attacker could manipulate the discovery to return malicious nodes with 2p% probability.
+The discovery mechanism needs to be resilient against this attack.
+
+### Roll-out Phase
+
+During the roll-out phase of Tor Push, during which only a few nodes use Tor Push,
+attackers can narrow down the senders of Tor messages
+to the set of gossipsub nodes that do not originate messages.
+Nodes who want anonymity guarantees even during the roll-out phase
+can use separate network interfaces for their default context and
+Tp-context, respectively.
+For the best protection, these contexts should run on separate physical machines.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+* [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+* [libp2p pubsub](https://github.com/libp2p/specs/tree/master/pubsub)
+* [libp2p pubsub message](https://github.com/libp2p/specs/tree/master/pubsub#the-message)
+* [libp2p switch](https://docs.libp2p.io/concepts/multiplex/switch)
+* [SOCKS5](https://www.rfc-editor.org/rfc/rfc1928)
+* [Tor](https://www.torproject.org/)
+* [33/WAKU2-DISCV5](../../waku/standards/core/33/discv5.md)
+* [Bitcoin over Tor isn't a Good Idea](https://ieeexplore.ieee.org/abstract/document/7163022)
+* [17/WAKU2-RLN-RELAY](../../waku/standards/core/17/rln-relay.md)

vac/raw/images/test.txt

@@ -0,0 +1 @@
+

vac/raw/rln-interep-spec.md

@@ -0,0 +1,140 @@
+---
+title: RLN-INTEREP-SPEC
+name: Interep as group management for RLN
+status: raw
+category:  
+tags: rln
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+---
+
+## Abstract
+
+This spec integrates [Interep](https://interep.link)
+into the [RLN](../32/rln-v1.md) spec.
+Interep is a group management protocol
+that allows for the creation of groups of users and
+the management of their membership.
+It is used to manage the membership of the RLN group.
+
+Interep ties in web2 identities with reputation, and
+sorts the users into groups based on their reputation score.
+For example, a GitHub user with over 100 followers is considered to have "gold" reputation.
+
+Interep uses [Semaphore](https://semaphore.appliedzkp.org/)
+under the hood to allow anonymous signaling of membership in a group.
+Therefore, a user with a "gold" reputation can prove the existence
+of their membership without revealing their identity.
+
+RLN is used for spam prevention, and Interep is used for group management.
+
+By using Interep with RLN,
+we allow users to join RLN membership groups
+without the need for on-chain financial stake.
+
+## Motivation
+
+To have Sybil-Resistant group management,
+there are [implementations](https://github.com/vacp2p/rln-contract)
+of RLN which make use of financial stake on-chain.
+However, this is not ideal because it reduces the barrier of entry for honest participants.
+
+In this case,
+honest participants will most likely have a web2 identity accessible to them,
+which can be used for joining an Interep reputation group.
+By modifying the RLN spec to use Interep,
+we can have Sybil-Resistant group management
+without the need for on-chain financial stake.
+
+Since RLN and Interep both use Semaphore-style credentials,
+it is possible to use the same set of credentials for both.
+
+## Functional Operation
+
+Using Interep with RLN involves the following steps -
+
+1. Generate Semaphore credentials
+2. Verify reputation and join Interep group
+3. Join RLN membership group via interaction with Smart Contract,
+by passing a proof of membership to the Interep group
+
+### 1. Generate Semaphore credentials
+
+Semaphore credentials are generated in a standard way,
+depicted in the [Semaphore documentation](https://semaphore.appliedzkp.org/docs/guides/identities#create-deterministic-identities).
+
+### 2. Verify reputation and join Interep group
+
+Using the Interep app deployed on [Goerli](https://goerli.interep.link/),
+the user can check their reputation tier and join the corresponding group.
+This results in a transaction to the Interep contract, which adds them to the group.
+
+### 3. Join RLN membership group
+
+Instead of sending funds to the RLN contract to join the membership group,
+the user can send a proof of membership to the Interep group.
+This proof is generated by the user, and
+is verified by the contract.
+The contract ensures that the user is a member of the Interep group, and
+then adds them to the RLN membership group.
+
+Following is the modified signature of the register function
+in the RLN contract -
+
+```solidity
+    /// @param groupId: Id of the group.
+    /// @param signal: Semaphore signal.
+    /// @param nullifierHash: Nullifier hash.
+    /// @param externalNullifier: External nullifier.
+    /// @param proof: Zero-knowledge proof.
+    /// @param idCommitment: ID Commitment of the member.
+    function register(
+        uint256 groupId,
+        bytes32 signal,
+        uint256 nullifierHash,
+        uint256 externalNullifier,
+        uint256[8] calldata proof,
+        uint256 idCommitment
+    )
+```
+
+## Verification of messages
+
+Messages are verified the same way as in the [RLN spec](../32/rln-v1.md/#verification).
+
+## Slashing
+
+The slashing mechanism is the same as in the [RLN spec](../32/rln-v1.md/#slashing).
+It is important to note that the slashing
+may not have the intended effect on the user,
+since the only consequence is that they cannot send messages.
+This is due to the fact that the user
+can send a identity commitment in the registration to the RLN contract,
+which is different than the one used in the Interep group.
+
+## Proof of Concept
+
+A proof of concept is available at
+[vacp2p/rln-interp-contract](https://github.com/vacp2p/rln-interep-contract)
+which integrates Interep with RLN.
+
+## Security Considerations
+
+1. As mentioned in [Slashing](#slashing),
+the slashing mechanism may not have the intended effect on the user.
+2. This spec inherits the security considerations of the [RLN spec](../32/rln-v1.md/#security-considerations).
+3. This spec inherits the security considerations of [Interep](https://docs.interep.link/).
+4. A user may make multiple registrations using the same Interep proofs but
+different identity commitments.
+The way to mitigate this is to check if the nullifier hash has been detected
+previously in proof verification.
+
+## References
+
+1. [RLN spec](../32/rln-v1.md)
+2. [Interep](https://interep.link)
+3. [Semaphore](https://semaphore.appliedzkp.org/)
+4. [Decentralized cloudflare using Interep](https://ethresear.ch/t/decentralised-cloudflare-using-rln-and-rich-user-identities/10774)
+5. [Interep contracts](https://github.com/interep-project/contracts)
+6. [RLN contract](https://github.com/vacp2p/rln-contract)
+7. [RLNP2P](https://rlnp2p.vac.dev/)

vac/raw/rln-stealth-commitments.md

@@ -0,0 +1,126 @@
+---
+title: RLN-STEALTH-COMMITMENTS
+name: RLN Stealth Commitment Usage
+category: Standards Track
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Jimmy Debe <jimmy@status.im>
+---
+
+## Abstract
+
+This specification describes the usage of stealth commitments
+to add prospective users to a network-governed
+[32/RLN-V1](./32/rln-v1.md) membership set.
+
+## Motivation
+
+When [32/RLN-V1](./32/rln-v1.md) is enforced in [10/Waku2](../waku/standards/core/10/waku2.md),
+all users are required to register to a membership set.
+The membership set will store user identities
+allowing the secure interaction within an application.
+Forcing a user to do an on-chain transaction
+to join a membership set is an onboarding friction,
+and some projects may be opposed to this method.
+To improve the user experience,
+stealth commitments can be used by a counterparty
+to register identities on the user's behalf,
+while maintaining the user's anonymity.
+
+This document specifies a privacy-preserving mechanism,
+allowing a counterparty to utilize [32/RLN-V1](./32/rln-v1.md)
+to register an `identityCommitment` on-chain.
+Counterparties will be able to register members
+to a RLN membership set without exposing the user's private keys.
+
+## Background
+
+The [32/RLN-V1](./32/rln-v1.md) protocol,
+consists of a smart contract that stores a `idenitityCommitment`
+in a membership set.
+In order for a user to join the membership set,
+the user is required to make a transaction on the blockchain.
+A set of public keys is used to compute a stealth commitment for a user,
+as described in [ERC-5564](https://eips.ethereum.org/EIPS/eip-5564).
+This specification is an implementation of the
+[ERC-5564](https://eips.ethereum.org/EIPS/eip-5564) scheme,
+tailored to the curve that is used in the [32/RLN-V1](./32/rln-v1.md) protocol.
+
+This can be used in a couple of ways in applications:
+
+1. Applications can add users
+to the [32/RLN-V1](./32/rln-v1.md) membership set in a batch.
+2. Users of the application
+can register other users to the [32/RLN-V1](./32/rln-v1.md) membership set.
+
+This is useful when the prospective user does not have access to funds
+on the network that [32/RLN-V1](./32/rln-v1.md) is deployed on.
+
+## Wire Format Specification
+
+The two parties, the requester and the receiver,
+MUST exchange the following information:
+
+```protobuf
+
+message Request {
+  // The spending public key of the requester
+  bytes spending_public_key = 1;
+
+  // The viewing public key of the requester
+  bytes viewing_public_key = 2;
+}
+```
+
+### Generate Stealth Commitment
+
+The application or user SHOULD generate a `stealth_commitment`
+after a request to do so is received.
+This commitment MAY be inserted into the corresponding application membership set.
+
+Once the membership set is updated,
+the receiver SHOULD exchange the following as a response to the request:
+
+```protobuf
+
+message Response {
+  
+  // The used to check if the stealth_commitment belongs to the requester
+  bytes view_tag = 2;
+
+  // The stealth commitment for the requester
+  bytes stealth_commitment = 3;
+
+  // The ephemeral public key used to generate the commitment
+  bytes ephemeral_public_key = 4;
+
+}
+
+```
+
+The receiver MUST generate an `ephemeral_public_key`,
+`view_tag` and `stealth_commitment`.
+This will be used to check the stealth commitment
+used to register to the membership set,
+and the user MUST be able to check ownership with their `viewing_public_key`.
+
+## Implementation Suggestions
+
+An implementation of the Stealth Address scheme is available in the
+[erc-5564-bn254](https://github.com/rymnc/erc-5564-bn254) repository,
+which also includes a test to generate a stealth commitment for a given user.
+
+## Security/Privacy Considerations
+
+This specification inherits the security and privacy considerations of the
+[Stealth Address](https://eips.ethereum.org/EIPS/eip-5564) scheme.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [10/Waku2](../waku/standards/core/10/waku2.md)
+- [32/RLN-V1](./32/rln-v1.md)
+- [ERC-5564](https://eips.ethereum.org/EIPS/eip-5564)

vac/raw/rln-v2.md

@@ -1,202 +1,237 @@
----
-slug: 58
-title: 58/RLN-V2
-name: Rate Limit Nullifier V2
-status: raw
-editor: Rasul Ibragimov <curryrasul@gmail.com>
-contributors: 
-- Lev Soukhanov <0xdeadfae@gmail.com>
----
-
-## Abstract
-
-The protocol specified in this document is an improvement of [32/RLN-V1](../32/rln-v1.md), being more general construct, that allows to set various limits for an epoch (it's 1 message per epoch in [32/RLN-V1](../32/rln-v1.md)) while remaining almost as simple as it predecessor. 
-Moreover, it allows to set different rate-limits for different RLN app users based on some public data, e.g. stake or reputation.
-
-## Motivation
-
-The main goal of this RFC is to generalize [32/RLN-V1](../32/rln-v1.md) and expand its applications. 
-There are two different subprotocols based on this protocol:
-* RLN-Same - RLN with the same rate-limit for all users;
-* RLN-Diff - RLN that allows to set different rate-limits for different users.
-
-It is important to note that by using a large epoch limit value, users will be able to remain anonymous, because their `internal_nullifiers` will not be repeated until they exceed the limit.
-
-## Flow
-
-As in [32/RLN-V1](../32/rln-v1.md), the general flow can be described by three steps:
-
-1. Registration
-2. Signaling
-3. Verification and slashing
-
-The two sub-protocols have different flows, and hence are defined separately.
-
-### Important note
-
-All terms and parameters used remain the same as in [32/RLN-V1](../32/rln-v1.md), more details [here](../32/rln-v1.md/#technical-overview)
-
-## RLN-Same flow
-
-### Registration
-
-The registration process in the RLN-Same subprotocol does not differ from [32/RLN-V1](../32/rln-v1.md).
-
-### Signalling
-
-#### Proof generation
-
-For proof generation, the user needs to submit the following fields to the circuit:
-
-```
-{
-    identity_secret: identity_secret_hash,
-    path_elements: Merkle_proof.path_elements,
-    identity_path_index: Merkle_proof.indices,
-    x: signal_hash,
-    message_id: message_id,
-    external_nullifier: external_nullifier,
-    message_limit: message_limit
-}
-```
-
-#### Calculating output
-
-The following fields are needed for proof output calculation:
-
-```
-{
-    identity_secret_hash: bigint, 
-    external_nullifier: bigint,
-    message_id: bigint,
-    x: bigint, 
-}
-```
-
-The output `[y, internal_nullifier]` is calculated in the following way:
-
-```
-a_0 = identity_secret_hash
-a_1 = poseidonHash([a0, external_nullifier, message_id])
-
-y = a_0 + x * a_1
-
-internal_nullifier = poseidonHash([a_1])
-```
-
-## RLN-Diff flow
-
-### Registration
-
-**id_commitment** in [32/RLN-V1](../32/rln-v1.md) is equal to `poseidonHash(identity_secret)`. 
-The goal of RLN-Diff is to set different rate-limits for different users. 
-It follows that **id_commitment** must somehow depend on the `user_message_limit` parameter, where 0 <= `user_message_limit` <= `message_limit`. 
-There are few ways to do that:
-1. Sending `identity_secret_hash` = `poseidonHash(identity_secret, userMessageLimit)` and zk proof that `user_message_limit` is valid (is in the right range). 
-This approach requires zkSNARK verification, which is an expensive operation on the blockchain.
-2. Sending the same `identity_secret_hash` as in [32/RLN-V1](../32/rln-v1.md) (`poseidonHash(identity_secret)`) and a user_message_limit publicly to a server or smart-contract where `rate_commitment` = `poseidonHash(identity_secret_hash, userMessageLimit)` is calculated. 
-The leaves in the membership Merkle tree would be the rate_commitments of the users. 
-This approach requires additional hashing in the Circuit, but it eliminates the need for zk proof verification for the registration.
-
-Both methods are correct, and the choice of the method is left to the implementer. 
-It is recommended to use second method for the reasons already described.
-The following flow description will also be based on the second method.
-
-### Signalling
-
-#### Proof generation
-
-For proof generation, the user need to submit the following fields to the circuit:
-
-```
-{
-    identity_secret: identity_secret_hash,
-    path_elements: Merkle_proof.path_elements,
-    identity_path_index: Merkle_proof.indices,
-    x: signal_hash,
-    message_id: message_id,
-    external_nullifier: external_nullifier,
-    user_message_limit: message_limit
-}
-```
-
-#### Calculating output
-
-The Output is calculated in the same way as the RLN-Same sub-protocol.
-
-### Verification and slashing
-
-Verification and slashing in both subprotocols remain the same as in [32/RLN-V1](../32/rln-v1.md).
-The only difference that may arise is the `message_limit` check in RLN-Same, since it is now a public input of the Circuit.
-
-### ZK Circuits specification
-
-The design of the [32/RLN-V1](../32/rln-v1.md) circuits is different from the circuits of this protocol. 
-RLN-v2 requires additional algebraic constraints.
-The membership proof and Shamir's Secret Sharing constraints remain unchanged.
-
-The ZK Circuit is implemented using a [Groth-16 ZK-SNARK](https://eprint.iacr.org/2016/260.pdf),
-using the [circomlib](https://docs.circom.io/) library. 
-Both schemes contain compile-time constants/system parameters:
-* DEPTH - depth of membership Merkle tree
-* LIMIT_BIT_SIZE - bit size of `limit` numbers, e.g. for the 16 - maximum `limit` number is 65535.
-
-The main difference of the protocol is that instead of a new polynomial (a new value `a_1`) for a new epoch, a new polynomial is generated for each message. 
-The user assigns an identifier to each message; the main requirement is that this identifier be in the range from 1 to `limit`. 
-This is proven using range constraints.
-
-### RLN-Same circuit
-
-#### Circuit parameters
-
-**Public Inputs**
-- `x`
-- `external_nullifier`
-- `message_limit` - limit per epoch
-
-**Private Inputs**
-- `identity_secret_hash`
-- `path_elements` 
-- `identity_path_index`
-- `message_id`
-
-**Outputs**
-- `y`
-- `root`
-- `internal_nullifier`
-
-### RLN-Diff circuit
-
-In the RLN-Diff scheme, instead of the public parameter `message_limit`, a parameter is used that is set for each user during registration (`user_message_limit`); the `message_id` value is compared to it in the same way as it is compared to `message_limit` in the case of RLN-Same.
-
-#### Circuit parameters
-
-**Public Inputs**
-- `x`
-- `external_nullifier`
-
-**Private Inputs**
-- `identity_secret_hash`
-- `path_elements`
-- `identity_path_index`
-- `message_id`
-- `user_message_limit`
-
-**Outputs**
-- `y`
-- `root`
-- `internal_nullifier`
-
-## Appendix A: Security considerations
-
-Although there are changes in the circuits, this spec inherits all the security considerations of [32/RLN-V1](../32/rln-v1.md).
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [1](https://zkresear.ch/t/rate-limit-nullifier-v2-circuits/102)
-- [2](https://github.com/Rate-Limiting-Nullifier/rln-circuits-v2)
-- [3](../32/rln-v1.md/#technical-overview)
+---
+title: RLN-V2
+name: Rate Limit Nullifier V2
+status: raw
+editor: Rasul Ibragimov <curryrasul@gmail.com>
+contributors: 
+- Lev Soukhanov <0xdeadfae@gmail.com>
+---
+
+## Abstract
+
+The protocol specified in this document is an improvement of [32/RLN-V1](../32/rln-v1.md),
+being more general construct, that allows to set various limits for an epoch
+(it's 1 message per epoch in [32/RLN-V1](../32/rln-v1.md))
+while remaining almost as simple as it predecessor.
+Moreover, it allows to set different rate-limits
+for different RLN app users based on some public data,
+e.g. stake or reputation.
+
+## Motivation
+
+The main goal of this RFC is to generalize [32/RLN-V1](../32/rln-v1.md) and
+expand its applications.
+There are two different subprotocols based on this protocol:
+
+* RLN-Same - RLN with the same rate-limit for all users;
+* RLN-Diff - RLN that allows to set different rate-limits for different users.
+
+It is important to note that by using a large epoch limit value,
+users will be able to remain anonymous,
+because their `internal_nullifiers` will not be repeated until they exceed the limit.
+
+## Flow
+
+As in [32/RLN-V1](../32/rln-v1.md), the general flow can be described by three steps:
+
+1. Registration
+2. Signaling
+3. Verification and slashing
+
+The two sub-protocols have different flows, and
+hence are defined separately.
+
+### Important note
+
+All terms and parameters used remain the same as in [32/RLN-V1](../32/rln-v1.md),
+more details [here](../32/rln-v1.md/#technical-overview)
+
+## RLN-Same flow
+
+### Registration
+
+The registration process in the RLN-Same subprotocol does not differ from [32/RLN-V1](../32/rln-v1.md).
+
+### Signalling
+
+#### Proof generation
+
+For proof generation, the user needs to submit the following fields to the circuit:
+
+```js
+{
+    identity_secret: identity_secret_hash,
+    path_elements: Merkle_proof.path_elements,
+    identity_path_index: Merkle_proof.indices,
+    x: signal_hash,
+    message_id: message_id,
+    external_nullifier: external_nullifier,
+    message_limit: message_limit
+}
+```
+
+#### Calculating output
+
+The following fields are needed for proof output calculation:
+
+```js
+{
+    identity_secret_hash: bigint, 
+    external_nullifier: bigint,
+    message_id: bigint,
+    x: bigint, 
+}
+```
+
+The output `[y, internal_nullifier]` is calculated in the following way:
+
+```js
+a_0 = identity_secret_hash
+a_1 = poseidonHash([a0, external_nullifier, message_id])
+
+y = a_0 + x * a_1
+
+internal_nullifier = poseidonHash([a_1])
+```
+
+## RLN-Diff flow
+
+Registration
+
+**id_commitment** in [32/RLN-V1](../32/rln-v1.md) is equal to `poseidonHash(identity_secret)`.
+The goal of RLN-Diff is to set different rate-limits for different users.
+It follows that **id_commitment** must somehow depend
+on the `user_message_limit` parameter,
+where 0 <= `user_message_limit` <= `message_limit`.
+There are few ways to do that:
+
+1. Sending `identity_secret_hash` = `poseidonHash(identity_secret, userMessageLimit)`
+and zk proof that `user_message_limit` is valid (is in the right range).
+This approach requires zkSNARK verification,
+which is an expensive operation on the blockchain.
+2. Sending the same `identity_secret_hash` as in [32/RLN-V1](../32/rln-v1.md)
+(`poseidonHash(identity_secret)`) and a user_message_limit publicly to a server
+ or smart-contract where
+`rate_commitment` = `poseidonHash(identity_secret_hash, userMessageLimit)` is calculated.
+The leaves in the membership Merkle tree would be the rate_commitments of the users.
+This approach requires additional hashing in the Circuit, but
+it eliminates the need for zk proof verification for the registration.
+
+Both methods are correct, and the choice of the method is left to the implementer.
+It is recommended to use second method for the reasons already described.
+The following flow description will also be based on the second method.
+
+### Signalling
+
+#### Proof generation
+
+For proof generation, the user need to submit the following fields to the circuit:
+
+```js
+{
+    identity_secret: identity_secret_hash,
+    path_elements: Merkle_proof.path_elements,
+    identity_path_index: Merkle_proof.indices,
+    x: signal_hash,
+    message_id: message_id,
+    external_nullifier: external_nullifier,
+    user_message_limit: message_limit
+}
+```
+
+#### Calculating output
+
+The Output is calculated in the same way as the RLN-Same sub-protocol.
+
+### Verification and slashing
+
+Verification and slashing in both subprotocols remain the same as in [32/RLN-V1](../32/rln-v1.md).
+The only difference that may arise is the `message_limit` check in RLN-Same,
+since it is now a public input of the Circuit.
+
+### ZK Circuits specification
+
+The design of the [32/RLN-V1](../32/rln-v1.md) circuits
+is different from the circuits of this protocol.
+RLN-v2 requires additional algebraic constraints.
+The membership proof and Shamir's Secret Sharing constraints remain unchanged.
+
+The ZK Circuit is implemented using a [Groth-16 ZK-SNARK](https://eprint.iacr.org/2016/260.pdf),
+using the [circomlib](https://docs.circom.io/) library.
+Both schemes contain compile-time constants/system parameters:
+
+* DEPTH - depth of membership Merkle tree
+* LIMIT_BIT_SIZE - bit size of `limit` numbers,
+e.g. for the 16 - maximum `limit` number is 65535.
+
+The main difference of the protocol is that instead of a new polynomial
+(a new value `a_1`) for a new epoch, a new polynomial is generated for each message.
+The user assigns an identifier to each message;
+the main requirement is that this identifier be in the range from 1 to `limit`.
+This is proven using range constraints.
+
+### RLN-Same circuit
+
+#### Circuit parameters
+
+**Public Inputs**
+
+* `x`
+* `external_nullifier`
+* `message_limit` - limit per epoch
+
+**Private Inputs**
+
+* `identity_secret_hash`
+* `path_elements`
+* `identity_path_index`
+* `message_id`
+
+**Outputs**
+
+* `y`
+* `root`
+* `internal_nullifier`
+
+### RLN-Diff circuit
+
+In the RLN-Diff scheme, instead of the public parameter `message_limit`,
+a parameter is used that is set for each user during registration (`user_message_limit`);
+the `message_id` value is compared to it in the same way
+as it is compared to `message_limit` in the case of RLN-Same.
+
+#### Circuit parameters
+
+**Public Inputs**
+
+* `x`
+* `external_nullifier`
+
+**Private Inputs**
+
+* `identity_secret_hash`
+* `path_elements`
+* `identity_path_index`
+* `message_id`
+* `user_message_limit`
+
+**Outputs**
+
+* `y`
+* `root`
+* `internal_nullifier`
+
+## Appendix A: Security considerations
+
+Although there are changes in the circuits,
+this spec inherits all the security considerations of [32/RLN-V1](../32/rln-v1.md).
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+* [1](https://zkresear.ch/t/rate-limit-nullifier-v2-circuits/102)
+* [2](https://github.com/Rate-Limiting-Nullifier/rln-circuits-v2)
+* [3](../32/rln-v1.md/#technical-overview)

vac/raw/url-scheme.md

@@ -0,0 +1,72 @@
+---
+title: STATUS-URL-SCHEME
+name: Status URL Scheme
+status: raw
+category: Standards Track
+tags:
+editor: Felicio Mununga <felicio@status.im>
+contributors:
+---
+
+## Abstract
+
+This document describes URL scheme for previewing and
+deep linking content as well as for triggering actions.
+
+## Background / Rationale / Motivation
+
+### Requirements
+
+#### Related scope
+
+##### Features
+
+- Onboarding website
+- Link preview
+- Link sharing
+- Deep linking
+- Routing and navigation
+- Payment requests
+- Chat creation
+
+## Wire Format Specification / Syntax
+
+### Schemes
+
+- Internal `status-app://`
+- External `https://` (i.e. univers/deep links)
+
+### Paths
+
+| Name | Url | Description |
+| ----- | ---- | ---- |
+| User profile | `/u/<encoded_data>#<user_chat_key>` | Preview/Open user profile |
+| | `/u#<user_chat_key>` | |
+| | `/u#<ens_name>` | |
+| Community | `/c/<encoded_data>#<community_chat_key>` | Preview/Open community |
+| | `/c#<community_chat_key>` | |
+| Community channel | `/cc/<encoded_data>#<community_chat_key >`| Preview/Open community channel |
+| | `/cc/<channel_uuid>#<community_chat_key>` | |
+
+<!-- # Security/Privacy Considerations
+
+A standard track RFC in `stable` status MUST feature this section.
+A standard track RFC in `raw` or `draft` status SHOULD feature this section.
+Informational RFCs (in any state) may feature this section.
+If there are none, this section MUST explicitly state that fact.
+This section MAY contain additional relevant information,
+e.g. an explanation as to why there are no security consideration
+for the respective document. -->
+
+## Discussions
+
+- See <https://github.com/status-im/specs/pull/159>
+- See <https://github.com/status-im/status-web/issues/327>
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [STATUS-URL-DATA](./url-data.md)

vac/template.md

@@ -1,82 +1,93 @@
----
-slug: XX
-title: XX/(WAKU2|LOGOS|CODEX|*)-TEMPLATE
-name: (Waku v2 | Logos | Codex) RFC Template
-status: (raw|draft|stable)
-category: (Standards Track|Informational|Best Current Practice)
-tags: an optional list of tags, not standard
-editor: Daniel Kaiser <danielkaiser@status.im>
-contributors:
----
-
-# (Info, remove this section)
-
-This section contains meta info about writing RFCs.
-This section (including its subsections) MUST be removed.
-
-[COSS](https://rfc.vac.dev/spec/1/) explains the Vac RFC process.
-
-## Tags
-
-The `tags` metadata SHOULD contain a list of tags if applicable.
-
-Currently identified tags comprise
-
-* `waku/core-protocol` for Waku protocol definitions (e.g. store, relay, light push),
-* `waku/application` for applications built on top of Waku protocol (e.g. eth-dm, toy-chat),
-
-
-# Abstract
-
-
-# Background / Rationale / Motivation
-
-This section serves as an introduction providing background information and a motivation/rationale for why the specified protocol is useful.
-
-# Theory / Semantics
-
-A standard track RFC in `stable` status MUST feature this section.
-A standard track RFC in `raw` or `draft` status SHOULD feature this section.
-This section SHOULD explain in detail how the proposed protocol works.
-It may touch on the wire format where necessary for the explanation.
-This section MAY also specify endpoint behaviour when receiving specific messages, e.g. the behaviour of certain caches etc.
-
-# Wire Format Specification / Syntax
-
-A standard track RFC in `stable` status MUST feature this section.
-A standard track RFC in `raw` or `draft` status SHOULD feature this section.
-This section SHOULD not contain explanations of semantics and focus on concisely defining the wire format.
-Implementations MUST adhere to these exact formats to interoperate with other implementations.
-It is fine, if parts of the previous section that touch on the wire format are repeated.
-The purpose of this section is having a concise definition of what an implementation sends and accepts.
-Parts that are not specified here are considered implementation details. Implementors are free to decide on how to implement these details.
-An optional *implementation suggestions* section may provide suggestions on how to approach implementation details, and, if available, point to existing implementations for reference.
-
-# Implementation Suggestions (optional)
-
-
-# (Further Optional Sections)
-
-
-# Security/Privacy Considerations
-
-A standard track RFC in `stable` status MUST feature this section.
-A standard track RFC in `raw` or `draft` status SHOULD feature this section.
-Informational RFCs (in any state) may feature this section.
-If there are none, this section MUST explicitly state that fact.
-This section MAY contain additional relevant information, e.g. an explanation as to why there are no security consideration for the respective document.
-
-# Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-# References
-
-References MAY be subdivided into normative and informative.
-
-## normative
-A list of references that MUST be read to fully understand and/or implement this protocol.
-See [RFC3967 Section 1.1](https://datatracker.ietf.org/doc/html/rfc3967#section-1.1).
-
-## informative
-A list of additional references.
+---
+slug: XX
+title: XX/(WAKU2|LOGOS|CODEX|*)-TEMPLATE
+name: (Waku v2 | Logos | Codex) RFC Template
+status: (raw|draft|stable)
+category: (Standards Track|Informational|Best Current Practice)
+tags: an optional list of tags, not standard
+editor: Daniel Kaiser <danielkaiser@status.im>
+contributors:
+---
+
+## (Info, remove this section)
+
+This section contains meta info about writing RFCs.
+This section (including its subsections) MUST be removed.
+
+[COSS](https://rfc.vac.dev/spec/1/) explains the Vac RFC process.
+
+## Tags
+
+The `tags` metadata SHOULD contain a list of tags if applicable.
+
+Currently identified tags comprise
+
+* `waku/core-protocol` for Waku protocol definitions (e.g. store, relay, light push),
+* `waku/application` for applications built on top of Waku protocol
+(e.g. eth-dm, toy-chat),
+
+## Abstract
+
+## Background / Rationale / Motivation
+
+This section serves as an introduction providing background information and
+a motivation/rationale for why the specified protocol is useful.
+
+## Theory / Semantics
+
+A standard track RFC in `stable` status MUST feature this section.
+A standard track RFC in `raw` or `draft` status SHOULD feature this section.
+This section SHOULD explain in detail how the proposed protocol works.
+It may touch on the wire format where necessary for the explanation.
+This section MAY also specify endpoint behaviour when receiving specific messages,
+e.g. the behaviour of certain caches etc.
+
+## Wire Format Specification / Syntax
+
+A standard track RFC in `stable` status MUST feature this section.
+A standard track RFC in `raw` or `draft` status SHOULD feature this section.
+This section SHOULD not contain explanations of semantics and
+focus on concisely defining the wire format.
+Implementations MUST adhere to these exact formats to interoperate with other implementations.
+It is fine, if parts of the previous section that touch on the wire format are repeated.
+The purpose of this section is having a concise definition
+of what an implementation sends and accepts.
+Parts that are not specified here are considered implementation details.
+Implementors are free to decide on how to implement these details.
+An optional *implementation suggestions* section may provide suggestions
+on how to approach implementation details, and,
+if available, point to existing implementations for reference.
+
+## Implementation Suggestions (optional)
+
+
+## (Further Optional Sections)
+
+
+## Security/Privacy Considerations
+
+A standard track RFC in `stable` status MUST feature this section.
+A standard track RFC in `raw` or `draft` status SHOULD feature this section.
+Informational RFCs (in any state) may feature this section.
+If there are none, this section MUST explicitly state that fact.
+This section MAY contain additional relevant information,
+e.g. an explanation as to why there are no security consideration
+for the respective document.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+References MAY be subdivided into normative and informative.
+
+## normative
+
+A list of references that MUST be read to fully understand and/or
+implement this protocol.
+See [RFC3967 Section 1.1](https://datatracker.ietf.org/doc/html/rfc3967#section-1.1).
+
+## informative
+
+A list of additional references.

waku/README.md

@@ -1,5 +1,5 @@
-## Waku RFCs
-
-Waku builds a family of privacy-preserving, censorship-resistant communication protocols for web3 applications.
-
-Contributors can visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.
+# Waku RFCs
+
+Waku builds a family of privacy-preserving, censorship-resistant communication protocols for web3 applications.
+
+Contributors can visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.

waku/deprecated/18/swap.md

@@ -0,0 +1,263 @@
+---
+slug: 18
+title: 18/WAKU2-SWAP
+name: Waku SWAP Accounting
+status: deprecated
+editor: Oskar Thorén <oskarth@titanproxy.com>
+contributor: Ebube Ud <ebube@status.im>
+---
+
+## Abstract
+
+This specification outlines how we do accounting and settlement based on the provision
+and usage of resources, most immediately bandwidth usage and/or
+storing and retrieving of Waku message.
+This enables nodes to cooperate and efficiently share resources,
+and in the case of unequal nodes to settle the difference
+through a relaxed payment mechanism in the form of sending cheques.
+
+**Protocol identifier***: `/vac/waku/swap/2.0.0-beta1`
+
+## Motivation
+
+The Waku network makes up a service network, and
+some nodes provide a useful service to other nodes.
+We want to account for that, and when imbalances arise, settle this.
+The core of this approach has some theoretical backing in game theory, and
+variants of it have practically been proven to work in systems such as Bittorrent.
+The specific model use was developed by the Swarm project
+(previously part of Ethereum), and
+we re-use contracts that were written for this purpose.
+
+By using a delayed payment mechanism in the form of cheques,
+a barter-like mechanism can arise, and
+nodes can decide on their own policy
+as opposed to be strictly tied to a specific payment scheme.
+Additionally, this delayed settlement eases requirements
+on the underlying network in terms of transaction speed or costs.
+
+Theoretically, nodes providing and using resources over a long,
+indefinite, period of time can be seen as an iterated form of
+[Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma).
+Specifically, and more intuitively,
+since we have a cost and benefit profile for each provision/usage
+(of Waku Message's, e.g.), and
+the pricing can be set such that mutual cooperation is incentivized,
+this can be analyzed as a form of donations game.
+
+## Game Theory - Iterated prisoner's dilemma / donation game
+
+What follows is a sketch of what the game looks like between two nodes.
+We can look at it as a special case of iterated prisoner's dilemma called a
+[Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game)
+where each node can cooperate with some benefit `b` at a personal cost `c`,
+where `b>c`.
+
+From A's point of view:
+
+A/B | Cooperate | Defect
+-----|----------|-------
+Cooperate | b-c | -c
+Defect | b | 0
+
+What this means is that if A and B cooperates,
+A gets some benefit `b` minus a cost `c`.
+If A cooperates and B defects she only gets the cost,
+and if she defects and B cooperates A only gets the benefit.
+If both defect they get neither benefit nor cost.
+
+The generalized form of PD is:
+
+A/B | Cooperate | Defect
+-----|----------|-------
+Cooperate | R | S
+Defect | T | P
+
+With R=reward, S=Sucker's payoff, T=temptation, P=punishment
+
+And the following holds:
+
+- `T>R>P>S`
+- `2R>T+S`
+
+In our case, this means `b>b-c>0>-c` and `2(b-c)> b-c` which is trivially true.
+
+As this is an iterated game with no clear finishing point in most circumstances,
+a tit-for-tat strategy is simple, elegant and functional.
+To be more theoretically precise,
+this also requires reasonable assumptions on error rate and discount parameter.
+This captures notions such as
+"does the perceived action reflect the intended action" and
+"how much do you value future (uncertain) actions compared to previous actions".
+See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
+for more details.
+In specific circumstances,
+nodes can choose slightly different policies if there's a strong need for it.
+A policy is simply how a node chooses to act given a set of circumstances.
+
+A tit-for-tat strategy basically means:
+
+- cooperate first (perform service/beneficial action to other node)
+- defect when node stops cooperating (disconnect and similar actions),
+i.e. when it stops performing according to set parameters re settlement
+- resume cooperation if other node does so
+
+This can be complemented with node selection mechanisms.
+
+## SWAP protocol overview
+
+We use SWAP for accounting and
+settlement in conjunction with other request/reply protocols in Waku v2,
+where accounting is done in a pairwise manner.
+It is an acronym with several possible meanings (as defined in the Book
+of Swarm), for example:
+
+- service wanted and provided
+- settle with automated payments
+- send waiver as payment
+- start without a penny
+
+This approach is based on communicating payment thresholds and
+sending cheques as indications of later payments.
+Communicating payment thresholds MAY be done out-of-band or as part of the handshake.
+Sending cheques is done once payment threshold is hit.
+
+See [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth)
+section 3.2. on Peer-to-peer accounting etc., for more context and details.
+
+### Accounting
+
+Nodes perform their own accounting for each relevant peer
+based on some "volume"/bandwidth metric.
+For now we take this to mean the number of `WakuMessage`s exchanged.
+
+Additionally, a price is attached to each unit.
+Currently, this is simply a "karma counter" and equal to 1 per message.
+
+Each accounting balance SHOULD be w.r.t. to a given protocol it is accounting for.
+
+NOTE: This may later be complemented with other metrics,
+either as part of SWAP or more likely outside of it.
+For example, online time can be communicated and
+attested to as a form of enhanced quality of service to inform peer selection.
+
+### Flow
+
+Assuming we have two store nodes,
+one operating mostly as a client (A) and another as server (B).
+
+1. Node A performs a handshake with B node.
+B node responds and both nodes communicate their payment threshold.
+2. Node A and B creates an accounting entry for the other peer,
+keep track of peer and current balance.
+3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`.
+Based on the number of WakuMessages in the response,
+both nodes update their accounting records.
+4. When payment threshold is reached,
+Node A sends over a cheque to reach a neutral balance.
+Settlement of this is currently out of scope,
+but would occur through a SWAP contract (to be specified).
+(mock and hard phase).
+5. If disconnect threshold is reached, Node B disconnects Node A (mock and hard phase).
+
+Note that not all of these steps are mandatory in initial stages,
+see below for more details.
+For example, the payment threshold MAY initially be set out of bounds,
+and policy is only activated in the mock and hard phase.
+
+### Protobufs
+
+We use protobuf to specify the handshake and signature.
+This current protobuf is a work in progress.
+This is needed for mock and hard phase.
+
+A handshake gives initial information about payment thresholds and
+possibly other information.
+A cheque is best thought of as a promise to pay at a later date.
+
+```protobuf
+
+message Handshake {
+    bytes payment_threshold = 1;
+}
+
+// TODO Signature?
+// Should probably be over the whole Cheque type
+message Cheque {
+    bytes beneficiary = 1;
+    // TODO epoch time or block time?
+    uint32 date = 2;
+    // TODO ERC20 extension?
+    // For now karma counter
+    uint32 amount = 3;
+}
+
+```
+
+## Incremental integration and roll-out
+
+To incrementally integrate this into Waku v2,
+we have divided up the roll-out into three phases:
+
+- Soft - accounting only
+- Mock - send mock cheques and take word for it
+- Hard Test - blockchain integration and deployed to public testnet
+(Goerli, Optimism testnet or similar)
+- Hard Main - deployed to a public mainnet
+
+An implementation MAY support any of these phases.
+
+### Soft phase
+
+In the soft phase only accounting is performed, without consequence for the
+peers. No disconnect or sending of cheques is performed at this tage.
+
+SWAP protocol is performed in conjunction with another request-reply protocol
+to account for its usage.
+It SHOULD be done for [13/WAKU2-STORE](../../core/13/store.md)
+and it MAY be done for other request/reply protocols.
+
+A client SHOULD log accounting state per peer
+and SHOULD indicate when a peer is out of bounds (either of its thresholds met).
+
+### Mock phase
+
+In the mock phase, we send mock cheques and send cheques/disconnect peers as appropriate.
+
+- If a node reaches a disconnect threshold,
+which MUST be outside the payment threshold, it SHOULD disconnect the other peer.
+- If a node is within payment balance, the other node SHOULD stay connected to it.
+- If a node receives a valid Cheque it SHOULD update its internal accounting records.
+- If any node behaves badly, the other node is free to disconnect and
+pick another node.
+  - Peer rating is out of scope of this specification.
+
+### Hard phase
+
+In the hard phase, in addition to sending cheques and activating policy, this is
+done with blockchain integration on a public testnet. More details TBD.
+
+This also includes settlements where cheques can be redeemed.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [Prisoner's Dilemma](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma)
+2. [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
+3. [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth)
+4. [13/WAKU2-STORE](../../core/13/store.md)
+
+<!--
+
+General TODOs:
+
+- Find new link for book of swarm
+- Illustrate payment and disconnection thresholds (mscgen not great for this?)
+- Elaborate on how accounting works with amount in the context of e.g. store
+- Illustrate flow
+- Specify chequeboo
+
+-->

waku/deprecated/README.md

@@ -1,6 +1,6 @@
-## Deprecated RFCs 
-
-Deprecated specifications are no longer used in Waku products.
-This subdirectory is for achrive purpose and 
-should not be used in production ready implementations.
-Visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.
+# Deprecated RFCs 
+
+Deprecated specifications are no longer used in Waku products.
+This subdirectory is for achrive purpose and 
+should not be used in production ready implementations.
+Visit [Waku RFCs](https://github.com/waku-org/specs) for new Waku specifications under discussion.

waku/informational/22/toy-chat.md

@@ -1,53 +1,55 @@
----
-slug: 22
-title: 22/TOY-CHAT
-name: Waku v2 Toy Chat
-status: draft
-tags: waku/application
-editor: Franck Royer <franck@status.im>
-contributors:
-   - Hanno Cornelius <hanno@status.im>
----
-
-**Content Topic**: `/toy-chat/2/huilong/proto`.
-
-This specification explains a toy chat example using Waku v2.
-This protocol is mainly used to:
-
-1. Dogfood Waku v2,
-2. Show an example of how to use Waku v2.
-
-Currently, all main Waku v2 implementations support the toy chat protocol:
-[nim-waku](https://github.com/status-im/nim-waku/blob/master/examples/v2/chat2.nim),
-js-waku ([NodeJS](https://github.com/status-im/js-waku/tree/main/examples/cli-chat) and [web](https://github.com/status-im/js-waku/tree/main/examples/web-chat))
-and [go-waku](https://github.com/status-im/go-waku/tree/master/examples/chat2).
-
-Note that this is completely separate from the protocol the Status app is using for its chat functionality.
-
-# Design
-
-The chat protocol enables sending and receiving messages in a chat room.
-There is currently only one chat room, which is tied to the content topic.
-The messages SHOULD NOT be encrypted.
-
-The `contentTopic` MUST be set to `/toy-chat/2/huilong/proto`.
-
-# Payloads
-
-```protobuf
-syntax = "proto3";
-
-message Chat2Message {
-   uint64 timestamp = 1;
-   string nick = 2;
-   bytes payload = 3;
-}
-```
-
-- `timestamp`: The time at which the message was sent, in Unix Epoch seconds,
-- `nick`: The nickname of the user sending the message,
-- `payload`: The text of the messages, UTF-8 encoded.
-
-# Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 22
+title: 22/TOY-CHAT
+name: Waku v2 Toy Chat
+status: draft
+tags: waku/application
+editor: Franck Royer <franck@status.im>
+contributors:
+   - Hanno Cornelius <hanno@status.im>
+---
+
+**Content Topic**: `/toy-chat/2/huilong/proto`.
+
+This specification explains a toy chat example using Waku v2.
+This protocol is mainly used to:
+
+1. Dogfood Waku v2,
+2. Show an example of how to use Waku v2.
+
+Currently, all main Waku v2 implementations support the toy chat protocol:
+[nim-waku](https://github.com/status-im/nim-waku/blob/master/examples/v2/chat2.nim),
+js-waku ([NodeJS](https://github.com/status-im/js-waku/tree/main/examples/cli-chat)
+ and [web](https://github.com/status-im/js-waku/tree/main/examples/web-chat))
+and [go-waku](https://github.com/status-im/go-waku/tree/master/examples/chat2).
+
+Note that this is completely separate from the protocol the Status app
+is using for its chat functionality.
+
+## Design
+
+The chat protocol enables sending and receiving messages in a chat room.
+There is currently only one chat room, which is tied to the content topic.
+The messages SHOULD NOT be encrypted.
+
+The `contentTopic` MUST be set to `/toy-chat/2/huilong/proto`.
+
+## Payloads
+
+```protobuf
+syntax = "proto3";
+
+message Chat2Message {
+   uint64 timestamp = 1;
+   string nick = 2;
+   bytes payload = 3;
+}
+```
+
+- `timestamp`: The time at which the message was sent, in Unix Epoch seconds,
+- `nick`: The nickname of the user sending the message,
+- `payload`: The text of the messages, UTF-8 encoded.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

waku/informational/23/topics.md

@@ -1,170 +1,185 @@
----
-slug: 23
-title: 23/WAKU2-TOPICS
-name: Waku v2 Topic Usage Recommendations
-status: draft
-category: Informational
-editor: Oskar Thoren <oskarth@titanproxy.com>
-contributors:
-  - Hanno Cornelius <hanno@status.im>
-  - Daniel Kaiser <danielkaiser@status.im>
----
-
-This document outlines recommended usage of topic names in Waku v2.
-In [10/WAKU2 spec](../../standards/core/10/waku2.md) there are two types of topics:
-
-- pubsub topics, used for routing
-- Content topics, used for content-based filtering
-
-
-## Pubsub Topics
-
-Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](../../standards/core/11/relay.md)),
-and can be named implicitly by Waku sharding (see [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
-This document comprises recommendations for explicitly naming pubsub topics (e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
-
-### Pubsub Topic Format
-
-Pubsub topics SHOULD follow the following structure:
-
-`/waku/2/{topic-name}`
-
-This namespaced structure makes compatibility, discoverability, and automatic handling of new topics easier.
-
-The first two parts indicate
-
-1) it relates to the Waku protocol domain, and
-2) the version is 2.
-
-If applicable, it is RECOMMENDED to structure `{topic-name}` in a hierarchical way as well.
-
-> *Note*: In previous versions of this document, the structure was `/waku/2/{topic-name}/{encoding}`.
-The now deprecated `/{encoding}` was always set to `/proto`,
-which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition) in pubsub is serialized/encoded as protobuf.
-The inspiration for this format was taken from
-[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
-However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md) must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
-which specifies the wire format as protobuf,`/proto` is the only valid encoding.
-This makes the `/proto` indication obsolete.
-The encoding of the `payload` field of a Waku Message is indicated by the `/{encoding}` part of the content topic name.
-Specifying an encoding is only significant for the actual payload/data field.
-Waku preserves this option by allowing to specify an encoding for the WakuMessage payload field as part of the content topic name.
-
-### Default PubSub Topic
-
-The Waku v2 default pubsub topic is:
-
-`/waku/2/default-waku/proto`
-
-The `{topic name}` part is `default-waku/proto`, which indicates it is default topic for exchanging WakuMessages;
-`/proto` remains for backwards compatibility.
-
-### Application Specific Names
-
-Larger apps can segregate their pubsub meshes using topics named like:
-
-```
-/waku/2/status/
-/waku/2/walletconnect/
-```
-
-This indicates that these networks carry WakuMessages, but for different domains completely.
-
-### Named Topic Sharding Example
-
-The following is an example of named sharding, as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md).
-
-```
-waku/2/waku-9_shard-0/
-...
-waku/2/waku-9_shard-9/
-```
-
-This indicates explicitly that the network traffic has been partitioned into 10 buckets.
-
-## Content Topics
-
-The other type of topic that exists in Waku v2 is a content topic.
-This is used for content based filtering.
-See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md) for where this is specified.
-Note that this doesn't impact routing of messages between relaying nodes,
-but it does impact how request/reply protocols such as 
-[12/WAKU2-FILTER](../../standards/core/14/filter.md) and [13/WAKU2-STORE](../../standards/core/13/store.md) are used.
-
-This is especially useful for nodes that have limited bandwidth,
-and only want to pull down messages that match this given content topic.
-
-Since all messages are relayed using the relay protocol regardless of content topic,
-you MAY use any content topic you wish without impacting how messages are relayed.
-
-### Content Topic Format
-
-The format for content topics is as follows:
-
-`/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}`
-
-The name of a content topic is application-specific.
-As an example, here's the content topic used for an upcoming testnet:
-
-`/toychat/2/huilong/proto`
-
-### Content Topic Naming Recommendations
-
-Application names should be unique to avoid conflicting issues with other protocols.
-Applications should specify their version (if applicable) in the version field.
-The `{content-topic-name}` portion of the content topic is up to the application,
-and depends on the problem domain.
-It can be hierarchical, for instance to separate content, or to indicate different bandwidth and privacy guarantees.
-The encoding field indicates the serialization/encoding scheme for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
-
-## Differences with Waku v1
-
-In [5/WAKU1](../../deprecated/5/waku0.md) there is no actual routing.
-All messages are sent to all other nodes.
-This means that we are implicitly using the same pubsub topic that would be something like:
-
-```
-/waku/1/default-waku/rlp
-```
-
-Topics in Waku v1 correspond to Content Topics in Waku v2.
-
-### Bridging Waku v1 and Waku v2
-
-To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](../../standards/core/15/bridge.md).
-For mapping Waku v1 topics to Waku v2 content topics,
-the following structure for the content topic SHOULD be used:
-
-```
-/waku/1/<4bytes-waku-v1-topic>/rfc26
-```
-
-The `<4bytes-waku-v1-topic>` SHOULD be the lowercase hex representation of the 4-byte Waku v1 topic.
-A `0x` prefix SHOULD be used.
-`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](../../standards/application/26/payload.md).
-See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description of the bridged fields.
-
-This creates a direct mapping between the two protocols.
-For example:
-
-```
-/waku/1/0x007f80ff/rfc26
-```
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-* [10/WAKU2 spec](../../standards/core/10/waku2.md)
-* [11/WAKU2-RELAY](../../standards/core/11/relay.md)
-* [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)
-* [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
-* [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
-* [12/WAKU2-FILTER](../../standards/core/14/filter.md)
-* [13/WAKU2-STORE](../../standards/core/13/store.md)
-* [6/WAKU1](../../deprecated/5/waku0.md)
-* [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
-* [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)
+---
+slug: 23
+title: 23/WAKU2-TOPICS
+name: Waku v2 Topic Usage Recommendations
+status: draft
+category: Informational
+editor: Oskar Thoren <oskarth@titanproxy.com>
+contributors:
+  - Hanno Cornelius <hanno@status.im>
+  - Daniel Kaiser <danielkaiser@status.im>
+---
+
+This document outlines recommended usage of topic names in Waku v2.
+In [10/WAKU2 spec](../../standards/core/10/waku2.md) there are two types of topics:
+
+- pubsub topics, used for routing
+- Content topics, used for content-based filtering
+
+## Pubsub Topics
+
+Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](../../standards/core/11/relay.md)),
+and can be named implicitly by Waku sharding (see [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
+This document comprises recommendations for explicitly naming pubsub topics
+(e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)).
+
+### Pubsub Topic Format
+
+Pubsub topics SHOULD follow the following structure:
+
+`/waku/2/{topic-name}`
+
+This namespaced structure makes compatibility, discoverability,
+and automatic handling of new topics easier.
+
+The first two parts indicate
+
+1) it relates to the Waku protocol domain, and
+2) the version is 2.
+
+If applicable, it is RECOMMENDED to structure `{topic-name}`
+in a hierarchical way as well.
+
+> *Note*: In previous versions of this document, the structure was `/waku/2/{topic-name}/{encoding}`.
+The now deprecated `/{encoding}` was always set to `/proto`,
+which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition)
+in pubsub is serialized/encoded as protobuf.
+The inspiration for this format was taken from
+[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
+However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md)
+must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
+which specifies the wire format as protobuf,`/proto` is the only valid encoding.
+This makes the `/proto` indication obsolete.
+The encoding of the `payload` field of a Waku Message
+is indicated by the `/{encoding}` part of the content topic name.
+Specifying an encoding is only significant for the actual payload/data field.
+Waku preserves this option by allowing to specify an encoding
+for the WakuMessage payload field as part of the content topic name.
+
+### Default PubSub Topic
+
+The Waku v2 default pubsub topic is:
+
+`/waku/2/default-waku/proto`
+
+The `{topic name}` part is `default-waku/proto`,
+which indicates it is default topic for exchanging WakuMessages;
+`/proto` remains for backwards compatibility.
+
+### Application Specific Names
+
+Larger apps can segregate their pubsub meshes using topics named like:
+
+```text
+/waku/2/status/
+/waku/2/walletconnect/
+```
+
+This indicates that these networks carry WakuMessages,
+but for different domains completely.
+
+### Named Topic Sharding Example
+
+The following is an example of named sharding, as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md).
+
+```text
+waku/2/waku-9_shard-0/
+...
+waku/2/waku-9_shard-9/
+```
+
+This indicates explicitly that the network traffic has been partitioned into 10 buckets.
+
+## Content Topics
+
+The other type of topic that exists in Waku v2 is a content topic.
+This is used for content based filtering.
+See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md)
+for where this is specified.
+Note that this doesn't impact routing of messages between relaying nodes,
+but it does impact how request/reply protocols such as
+[12/WAKU2-FILTER](../../standards/core/12/filter.md) and
+[13/WAKU2-STORE](../../standards/core/13/store.md) are used.
+
+This is especially useful for nodes that have limited bandwidth,
+and only want to pull down messages that match this given content topic.
+
+Since all messages are relayed using the relay protocol regardless of content topic,
+you MAY use any content topic you wish without impacting how messages are relayed.
+
+### Content Topic Format
+
+The format for content topics is as follows:
+
+`/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}`
+
+The name of a content topic is application-specific.
+As an example, here's the content topic used for an upcoming testnet:
+
+`/toychat/2/huilong/proto`
+
+### Content Topic Naming Recommendations
+
+Application names should be unique to avoid conflicting issues with other protocols.
+Applications should specify their version (if applicable) in the version field.
+The `{content-topic-name}` portion of the content topic is up to the application,
+and depends on the problem domain.
+It can be hierarchical, for instance to separate content, or
+to indicate different bandwidth and privacy guarantees.
+The encoding field indicates the serialization/encoding scheme
+for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
+
+## Differences with Waku v1
+
+In [5/WAKU1](../../deprecated/5/waku0.md) there is no actual routing.
+All messages are sent to all other nodes.
+This means that we are implicitly using the same pubsub topic
+that would be something like:
+
+```text
+/waku/1/default-waku/rlp
+```
+
+Topics in Waku v1 correspond to Content Topics in Waku v2.
+
+### Bridging Waku v1 and Waku v2
+
+To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](../../standards/core/15/bridge.md).
+For mapping Waku v1 topics to Waku v2 content topics,
+the following structure for the content topic SHOULD be used:
+
+```text
+/waku/1/<4bytes-waku-v1-topic>/rfc26
+```
+
+The `<4bytes-waku-v1-topic>` SHOULD be the lowercase hex representation
+of the 4-byte Waku v1 topic.
+A `0x` prefix SHOULD be used.
+`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](../../standards/application/26/payload.md).
+See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description
+of the bridged fields.
+
+This creates a direct mapping between the two protocols.
+For example:
+
+```text
+/waku/1/0x007f80ff/rfc26
+```
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [10/WAKU2 spec](../../standards/core/10/waku2.md)
+- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
+- [RELAY-SHARDING](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/relay-sharding.md)
+- [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
+- [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
+- [12/WAKU2-FILTER](../../standards/core/12/filter.md)
+- [13/WAKU2-STORE](../../standards/core/13/store.md)
+- [6/WAKU1](../../deprecated/5/waku0.md)
+- [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
+- [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)

waku/informational/27/peers.md

@@ -1,99 +1,119 @@
----
-slug: 27
-title: 27/WAKU2-PEERS
-name: Waku v2 Client Peer Management Recommendations
-status: draft
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
----
-
-`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and peer management features to be implemented by Waku v2 clients.
-
-In this context, peer _storage_ refers to a client's ability to keep track of discovered or statically-configured peers and their metadata.
-It also deals with matters of peer _persistence_,
-or the ability to store peer data on disk to resume state after a client restart.
-
-Peer _management_ is a closely related concept and refers to the set of actions a client MAY choose to perform based on its knowledge of its connected peers,
-e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
-
-## Peer store
-
-The peer store SHOULD be an in-memory data structure where information about discovered or configured peers are stored.
-It SHOULD be considered the main source of truth for peer-related information in a Waku v2 client.
-Clients MAY choose to persist this store on-disk.
-
-### Tracked peer metadata
-
-It is RECOMMENDED that a Waku v2 client tracks at least the following information about each of its peers in a peer store:
-
-| Metadata | Description  |
-| --- | --- |
-| _Public key_  | The public key for this peer. This is related to the libp2p [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/). |
-| _Addresses_ | Known transport layer [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/) for this peer. |
-| _Protocols_ | The libp2p [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids) supported by this peer. This can be used to track the client's connectivity to peers supporting different Waku v2 protocols, e.g. [`11/WAKU2-RELAY`](../../standards/core/11/relay.md) or [`13/WAKU2-STORE`](../../standards/core/13/store.md). |
-| _Connectivity_ | Tracks the peer's current connectedness state. See [**Peer connectivity**](#peer-connectivity) below. |
-| _Disconnect time_ | The timestamp at which this peer last disconnected. This becomes important when managing [peer reconnections](#reconnecting-peers) |
-
-### Peer connectivity
-
-A Waku v2 client SHOULD track _at least_ the following connectivity states for each of its peers:
- - **`NotConnected`**: The peer has been discovered or configured on this client,
- but no attempt has yet been made to connect to this peer.
- This is the default state for a new peer.
- - **`CannotConnect`**: The client attempted to connect to this peer, but failed.
- - **`CanConnect`**: The client was recently connected to this peer and disconnected gracefully.
- - **`Connected`**: The client is actively connected to this peer.
-
-This list does not preclude clients from tracking more advanced connectivity metadata,
-such as a peer's blacklist status (see [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)).
-
-### Persistence
-
-A Waku v2 client MAY choose to persist peers across restarts,
-using any offline storage technology, such as an on-disk database.
-Peer persistence MAY be used to resume peer connections after a client restart.
-
-## Peer management
-
-Waku v2 clients will have different requirements when it comes to managing the peers tracked in the [**peer store**](#peer-store).
-It is RECOMMENDED that clients support:
-- [automatic reconnection](#reconnecting-peers) to peers under certain conditions
-- [connection keep-alive](#connection-keep-alive)
-
-### Reconnecting peers
-
-A Waku v2 client MAY choose to reconnect to previously connected, managed peers under certain conditions.
-Such conditions include, but are not limited to:
-- Reconnecting to all `relay`-capable peers after a client restart. This will require [persistent peer storage](#persistence).
-
-If a client chooses to automatically reconnect to previous peers,
-it MUST respect the [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange) specified for GossipSub v1.1 before attempting to reconnect.
-This requires keeping track of the [last time each peer was disconnected](#tracked-peer-metadata).
-
-### Connection keep-alive
-
-A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers.
-If a client chooses to implement keep-alive on a connection,
-it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping) as per `10/WAKU2` [client recommendations](../../standards/core/10/WAKU2.md/#recommendations-for-clients).
-The recommended period between pings SHOULD be _at most_ 50% of the shortest idle connection timeout for the specific client and transport.
-For example, idle TCP connections often times out after 10 to 15 minutes.
-
-> **Implementation note:** the `nim-waku` client currently implements a keep-alive mechanism every `5 minutes`,
-in response to a TCP connection timeout of `10 minutes`.
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/)
-- [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/)
-- [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids)
-- [`11/WAKU2-RELAY`](../../standards/core/11/relay.md)
-- [`13/WAKU2-STORE`](../../standards/core/13/store.md)
-- [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)
-- [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
-- [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
-- [`10/WAKU2` client recommendations](https://rfc.vac.dev/spec/10/#recommendations-for-clients)
+---
+slug: 27
+title: 27/WAKU2-PEERS
+name: Waku v2 Client Peer Management Recommendations
+status: draft
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+---
+
+`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and
+peer management features to be implemented by Waku v2 clients.
+
+In this context, peer _storage_ refers to a client's ability to keep track of discovered
+or statically-configured peers and their metadata.
+It also deals with matters of peer _persistence_,
+or the ability to store peer data on disk to resume state after a client restart.
+
+Peer _management_ is a closely related concept and
+refers to the set of actions a client MAY choose to perform
+based on its knowledge of its connected peers,
+e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
+
+## Peer store
+
+The peer store SHOULD be an in-memory data structure
+where information about discovered or configured peers are stored.
+It SHOULD be considered the main source of truth
+for peer-related information in a Waku v2 client.
+Clients MAY choose to persist this store on-disk.
+
+### Tracked peer metadata
+
+It is RECOMMENDED that a Waku v2 client tracks at least the following information
+about each of its peers in a peer store:
+
+| Metadata | Description  |
+| --- | --- |
+| _Public key_  | The public key for this peer. This is related to the libp2p [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/). |
+| _Addresses_ | Known transport layer [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/) for this peer. |
+| _Protocols_ | The libp2p [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids) supported by this peer. This can be used to track the client's connectivity to peers supporting different Waku v2 protocols, e.g. [`11/WAKU2-RELAY`](../../standards/core/11/relay.md) or [`13/WAKU2-STORE`](../../standards/core/13/store.md). |
+| _Connectivity_ | Tracks the peer's current connectedness state. See [**Peer connectivity**](#peer-connectivity) below. |
+| _Disconnect time_ | The timestamp at which this peer last disconnected. This becomes important when managing [peer reconnections](#reconnecting-peers) |
+
+### Peer connectivity
+
+A Waku v2 client SHOULD track _at least_ the following connectivity states
+for each of its peers:
+
+- **`NotConnected`**: The peer has been discovered or configured on this client,
+ but no attempt has yet been made to connect to this peer.
+ This is the default state for a new peer.
+- **`CannotConnect`**: The client attempted to connect to this peer, but failed.
+- **`CanConnect`**: The client was recently connected to this peer and
+disconnected gracefully.
+- **`Connected`**: The client is actively connected to this peer.
+
+This list does not preclude clients from tracking more advanced connectivity metadata,
+such as a peer's blacklist status (see [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)).
+
+### Persistence
+
+A Waku v2 client MAY choose to persist peers across restarts,
+using any offline storage technology, such as an on-disk database.
+Peer persistence MAY be used to resume peer connections after a client restart.
+
+## Peer management
+
+Waku v2 clients will have different requirements
+when it comes to managing the peers tracked in the [**peer store**](#peer-store).
+It is RECOMMENDED that clients support:
+
+- [automatic reconnection](#reconnecting-peers) to peers under certain conditions
+- [connection keep-alive](#connection-keep-alive)
+
+### Reconnecting peers
+
+A Waku v2 client MAY choose to reconnect to previously connected,
+managed peers under certain conditions.
+Such conditions include, but are not limited to:
+
+- Reconnecting to all `relay`-capable peers after a client restart.
+This will require [persistent peer storage](#persistence).
+
+If a client chooses to automatically reconnect to previous peers,
+it MUST respect the
+[backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
+specified for GossipSub v1.1 before attempting to reconnect.
+This requires keeping track of the [last time each peer was disconnected](#tracked-peer-metadata).
+
+### Connection keep-alive
+
+A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers.
+If a client chooses to implement keep-alive on a connection,
+it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
+as per `10/WAKU2` [client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients).
+The recommended period between pings SHOULD be _at most_ 50%
+of the shortest idle connection timeout for the specific client and transport.
+For example, idle TCP connections often times out after 10 to 15 minutes.
+
+> **Implementation note:**
+the `nim-waku` client currently implements a keep-alive mechanism every `5 minutes`,
+in response to a TCP connection timeout of `10 minutes`.
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/)
+- [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/)
+- [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids)
+- [`11/WAKU2-RELAY`](../../standards/core/11/relay.md)
+- [`13/WAKU2-STORE`](../../standards/core/13/store.md)
+- [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)
+- [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
+- [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
+- [`10/WAKU2` client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients)

waku/informational/29/config.md

@@ -1,74 +1,79 @@
----
-slug: 29
-title: 29/WAKU2-CONFIG
-name: Waku v2 Client Parameter Configuration Recommendations
-status: draft
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
----
-
-`29/WAKU2-CONFIG` describes the RECOMMENDED values to assign to configurable parameters for Waku v2 clients.
-Since Waku v2 is built on [libp2p](https://github.com/libp2p/specs),
-most of the parameters and reasonable defaults are derived from there.
-
-Waku v2 relay messaging is specified in [`11/WAKU2-RELAY`](../../standards/core/11/relay.md),
-a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
-GossipSub behaviour is controlled by a series of adjustable parameters.
-Waku v2 clients SHOULD configure these parameters to the recommended values below.
-
-## GossipSub v1.0 parameters
-
-GossipSub v1.0 parameters are defined in the [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters).
-We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
-
-| Parameter            | Purpose                                               | RECOMMENDED value |
-|----------------------|-------------------------------------------------------|-------------------|
-| `D`                  | The desired outbound degree of the network            | 6                 |
-| `D_low`              | Lower bound for outbound degree                       | 4                 |
-| `D_high`             | Upper bound for outbound degree                       | 8                |
-| `D_lazy`             | (Optional) the outbound degree for gossip emission    | `D`               |
-| `heartbeat_interval` | Time between heartbeats                               | 1 second          |
-| `fanout_ttl`         | Time-to-live for each topic's fanout state            | 60 seconds        |
-| `mcache_len`         | Number of history windows in message cache            | 5                 |
-| `mcache_gossip`      | Number of history windows to use when emitting gossip | 3                 |
-| `seen_ttl`           | Expiry time for cache of seen message ids             | 2 minutes         |
-
-## GossipSub v1.1 parameters
-
-GossipSub v1.1 extended GossipSub v1.0 and introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
-We repeat the global parameters here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
-
-| Parameter      | Description                                                            | RECOMMENDED value |
-|----------------|------------------------------------------------------------------------|-------------------|
-| `PruneBackoff` | Time after pruning a mesh peer before we consider grafting them again. | 1 minute          |
-| `FloodPublish` | Whether to enable flood publishing                                     | true              |
-| `GossipFactor` | % of peers to send gossip to, if we have more than `D_lazy` available  | 0.25              |
-| `D_score`      | Number of peers to retain by score when pruning from oversubscription  | `D_low`           |
-| `D_out`        | Number of outbound connections to keep in the mesh.                    | `D_low` - 1       |
-
-`11/WAKU2-RELAY` clients SHOULD implement a peer scoring mechanism with the parameter constraints as [specified by libp2p](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
-
-## Other configuration
-
-The following behavioural parameters are not specified by `libp2p`,
-but nevertheless describes constraints that `11/WAKU2-RELAY` clients MAY choose to implement.
-
-| Parameter          | Description                                                               | RECOMMENDED value |
-|--------------------|---------------------------------------------------------------------------|-------------------|
-| `BackoffSlackTime` | Slack time to add to prune backoff before attempting to graft again       | 2 seconds         |
-| `IWantPeerBudget`  | Maximum number of IWANT messages to accept from a peer within a heartbeat | 25                |
-| `IHavePeerBudget`  | Maximum number of IHAVE messages to accept from a peer within a heartbeat | 10                |
-| `IHaveMaxLength`   | Maximum number of messages to include in an IHAVE message                 | 5000              |
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [libp2p](https://github.com/libp2p/specs)
-- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
-- [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
-- [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters)
-- [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters)
+---
+slug: 29
+title: 29/WAKU2-CONFIG
+name: Waku v2 Client Parameter Configuration Recommendations
+status: draft
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+---
+
+`29/WAKU2-CONFIG` describes the RECOMMENDED values
+to assign to configurable parameters for Waku v2 clients.
+Since Waku v2 is built on [libp2p](https://github.com/libp2p/specs),
+most of the parameters and reasonable defaults are derived from there.
+
+Waku v2 relay messaging is specified in [`11/WAKU2-RELAY`](../../standards/core/11/relay.md),
+a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
+GossipSub behaviour is controlled by a series of adjustable parameters.
+Waku v2 clients SHOULD configure these parameters to the recommended values below.
+
+## GossipSub v1.0 parameters
+
+GossipSub v1.0 parameters are defined in the [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters).
+We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
+
+| Parameter            | Purpose                                               | RECOMMENDED value |
+|----------------------|-------------------------------------------------------|-------------------|
+| `D`                  | The desired outbound degree of the network            | 6                 |
+| `D_low`              | Lower bound for outbound degree                       | 4                 |
+| `D_high`             | Upper bound for outbound degree                       | 8                |
+| `D_lazy`             | (Optional) the outbound degree for gossip emission    | `D`               |
+| `heartbeat_interval` | Time between heartbeats                               | 1 second          |
+| `fanout_ttl`         | Time-to-live for each topic's fanout state            | 60 seconds        |
+| `mcache_len`         | Number of history windows in message cache            | 5                 |
+| `mcache_gossip`      | Number of history windows to use when emitting gossip | 3                 |
+| `seen_ttl`           | Expiry time for cache of seen message ids             | 2 minutes         |
+
+## GossipSub v1.1 parameters
+
+GossipSub v1.1 extended GossipSub v1.0 and introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
+We repeat the global parameters here with RECOMMMENDED values
+for `11/WAKU2-RELAY` implementations.
+
+| Parameter      | Description                                                            | RECOMMENDED value |
+|----------------|------------------------------------------------------------------------|-------------------|
+| `PruneBackoff` | Time after pruning a mesh peer before we consider grafting them again. | 1 minute          |
+| `FloodPublish` | Whether to enable flood publishing                                     | true              |
+| `GossipFactor` | % of peers to send gossip to, if we have more than `D_lazy` available  | 0.25              |
+| `D_score`      | Number of peers to retain by score when pruning from oversubscription  | `D_low`           |
+| `D_out`        | Number of outbound connections to keep in the mesh.                    | `D_low` - 1       |
+
+`11/WAKU2-RELAY` clients SHOULD implement a peer scoring mechanism
+with the parameter constraints as
+[specified by libp2p](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
+
+## Other configuration
+
+The following behavioural parameters are not specified by `libp2p`,
+but nevertheless describes constraints that `11/WAKU2-RELAY` clients
+MAY choose to implement.
+
+| Parameter          | Description                                                               | RECOMMENDED value |
+|--------------------|---------------------------------------------------------------------------|-------------------|
+| `BackoffSlackTime` | Slack time to add to prune backoff before attempting to graft again       | 2 seconds         |
+| `IWantPeerBudget`  | Maximum number of IWANT messages to accept from a peer within a heartbeat | 25                |
+| `IHavePeerBudget`  | Maximum number of IHAVE messages to accept from a peer within a heartbeat | 10                |
+| `IHaveMaxLength`   | Maximum number of messages to include in an IHAVE message                 | 5000              |
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [libp2p](https://github.com/libp2p/specs)
+- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
+- [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+- [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters)
+- [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters)

waku/informational/30/adaptive-nodes.md

@@ -1,102 +1,117 @@
----
-slug: 30
-title: 30/ADAPTIVE-NODES
-name: Adaptive nodes
-status: draft
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributors:
----
-
-This is an informational spec that show cases the concept of adaptive nodes.
-
-## Node types - a continuum
-
-We can look at node types as a continuum, from more restricted to less restricted, fewer resources to more resources.
-
-![Node types - a continuum](./images/adaptive_node_continuum2.png)
-
-### Possible limitations
-
-- Connectivity: Not publicly connectable vs static IP and DNS
-- Connectivity: Mostly offline to mostly online to always online
-- Resources: Storage, CPU, Memory, Bandwidth
-
-### Accessibility and motivation
-
-Some examples:
-
-- Opening browser window: costs nothing, but contribute nothing
-- Desktop: download, leave in background, contribute somewhat
-- Cluster: expensive, upkeep, but can contribute a lot
-
-These are also illustrative, so a node in a browser in certain environment might contribute similarly to Desktop.
-
-### Adaptive nodes
-
-We call these nodes *adaptive nodes* to highlights different modes of contributing, such as:
-
-- Only leeching from the network
-- Relaying messages for one or more topics
-- Providing services for lighter nodes such as lightpush and filter
-- Storing historical messages to various degrees
-- Ensuring relay network can't be spammed with RLN
-
-### Planned incentives
-
-Incentives to run a node is currently planned around:
-
-- SWAP for accounting and settlement of services provided
-- RLN RELAY for spam protection
-- Other incentivization schemes are likely to follow and is an area of active research
-
-## Node protocol selection
-
-Each node can choose which protocols to support, depending on its resources and goals.
-
-![Protocol selection](./images/adaptive_node_protocol_selection2.png)
-
-In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md) etc (12, 13, 19, 21) these correspond to Libp2p protocols.
-
-However, other protocols like 16/WAKU2-RPC (local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY, Discovery v5 (DevP2P) or interfacing with distributed storage, are running on different network stacks.
-
-This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE, 26/WAKU2-PAYLOAD, or application specific ones. As well as specs that act more as recommendations, such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
-
-## Waku network visualization
-
-We can better visualize the network with some illustrative examples.
-
-### Topology and topics
-
-The first one shows an example topology with different PubSub topics for the relay protocol.
-
-![Waku Network visualization](./images/adaptive_node_network_topology_protocols2.png)
-
-### Legend
-
-![Waku Network visualization legend](./images/adaptive_node_network_topology_protocols_legend.png)
-
-The dotted box shows what content topics (application-specific) a node is interested in.
-
-A node that is purely providing a service to the network might not care.
-
-In this example, we see support for toy chat, a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
-
-### Auxiliary network
-
-This is a separate component with its own topology.
-
-Behavior and interaction with other protocols specified in Vac RFCs, e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
-
-### Node Cross Section
-
-This one shows a cross-section of nodes in different dimensions and shows how the connections look different for different protocols.
-
-![Node Cross Section](./images/adaptive_node_cross_section2.png)
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
+---
+slug: 30
+title: 30/ADAPTIVE-NODES
+name: Adaptive nodes
+status: draft
+editor: Oskar Thorén <oskarth@titanproxy.com>
+contributors:
+---
+
+This is an informational spec that show cases the concept of adaptive nodes.
+
+## Node types - a continuum
+
+We can look at node types as a continuum,
+from more restricted to less restricted, fewer resources to more resources.
+
+![Node types - a continuum](./images/adaptive_node_continuum2.png)
+
+### Possible limitations
+
+- Connectivity: Not publicly connectable vs static IP and DNS
+- Connectivity: Mostly offline to mostly online to always online
+- Resources: Storage, CPU, Memory, Bandwidth
+
+### Accessibility and motivation
+
+Some examples:
+
+- Opening browser window: costs nothing, but contribute nothing
+- Desktop: download, leave in background, contribute somewhat
+- Cluster: expensive, upkeep, but can contribute a lot
+
+These are also illustrative,
+so a node in a browser in certain environment might contribute similarly to Desktop.
+
+### Adaptive nodes
+
+We call these nodes *adaptive nodes* to highlights different modes of contributing,
+such as:
+
+- Only leeching from the network
+- Relaying messages for one or more topics
+- Providing services for lighter nodes such as lightpush and filter
+- Storing historical messages to various degrees
+- Ensuring relay network can't be spammed with RLN
+
+### Planned incentives
+
+Incentives to run a node is currently planned around:
+
+- SWAP for accounting and settlement of services provided
+- RLN RELAY for spam protection
+- Other incentivization schemes are likely to follow and is an area of active research
+
+## Node protocol selection
+
+Each node can choose which protocols to support, depending on its resources and goals.
+
+![Protocol selection](./images/adaptive_node_protocol_selection2.png)
+
+In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md)
+etc (12, 13, 19, 21) these correspond to Libp2p protocols.
+
+However, other protocols like 16/WAKU2-RPC
+(local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY,
+Discovery v5 (DevP2P) or interfacing with distributed storage,
+are running on different network stacks.
+
+This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE,
+26/WAKU2-PAYLOAD, or application specific ones.
+As well as specs that act more as recommendations,
+such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
+
+## Waku network visualization
+
+We can better visualize the network with some illustrative examples.
+
+### Topology and topics
+
+The first one shows an example topology with different PubSub topics
+for the relay protocol.
+
+![Waku Network visualization](./images/adaptive_node_network_topology_protocols2.png)
+
+### Legend
+
+![Waku Network visualization legend](./images/adaptive_node_network_topology_protocols_legend.png)
+
+The dotted box shows what content topics (application-specific)
+a node is interested in.
+
+A node that is purely providing a service to the network might not care.
+
+In this example, we see support for toy chat,
+a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
+
+### Auxiliary network
+
+This is a separate component with its own topology.
+
+Behavior and interaction with other protocols specified in Vac RFCs,
+e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
+
+### Node Cross Section
+
+This one shows a cross-section of nodes in different dimensions and
+shows how the connections look different for different protocols.
+
+![Node Cross Section](./images/adaptive_node_cross_section2.png)
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [11/WAKU2-RELAY](../../standards/core/11/relay.md)

waku/standards/application/18/swap.md

@@ -1,188 +0,0 @@
----
-slug: 18
-title: 18/WAKU2-SWAP
-name: Waku SWAP Accounting
-status: draft
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributor: Ebube Ud <ebube@status.im>
----
-
-## Abstract
-
-This specification outlines how we do accounting and settlement based on the provision and usage of resources, most immediately bandwidth usage and/or storing and retrieving of Waku message. This enables nodes to cooperate and efficiently share resources, and in the case of unequal nodes to settle the difference through a relaxed payment mechanism in the form of sending cheques.
-
-**Protocol identifier***: `/vac/waku/swap/2.0.0-beta1`
-
-## Motivation
-
-The Waku network makes up a service network, and some nodes provide a useful service to other nodes. We want to account for that, and when imbalances arise, settle this. The core of this approach has some theoretical backing in game theory, and variants of it have practically been proven to work in systems such as Bittorrent. The specific model use was developed by the Swarm project (previously part of Ethereum), and we re-use contracts that were written for this purpose.
-
-By using a delayed payment mechanism in the form of cheques, a barter-like mechanism can arise, and nodes can decide on their own policy as opposed to be strictly tied to a specific payment scheme. Additionally, this delayed settlement eases requirements on the underlying network in terms of transaction speed or costs.
-
-Theoretically, nodes providing and using resources over a long, indefinite, period of time can be seen as an iterated form of [Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma). Specifically, and more intuitively, since we have a cost and benefit profile for each provision/usage (of Waku Message's, e.g.), and the pricing can be set such that mutual cooperation is incentivized, this can be analyzed as a form of donations game.
-
-## Game Theory - Iterated prisoner's dilemma / donation game
-
-What follows is a sketch of what the game looks like between two nodes. We can look at it as a special case of iterated prisoner's dilemma called a [Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game) where each node can cooperate with some benefit `b` at a personal cost `c`, where `b>c`.
-
-From A's point of view:
-
-A/B | Cooperate | Defect
------|----------|-------
-Cooperate | b-c | -c
-Defect | b | 0
-
-What this means is that if A and B cooperates, A gets some benefit `b` minus a cost `c`. If A cooperates and B defects she only gets the cost, and if she defects and B cooperates A only gets the benefit. If both defect they get neither benefit nor cost.
-
-The generalized form of PD is:
-
-A/B | Cooperate | Defect
------|----------|-------
-Cooperate | R | S
-Defect | T | P
-
-With R=reward, S=Sucker's payoff, T=temptation, P=punishment
-
-And the following holds:
-
-- `T>R>P>S`
-- `2R>T+S`
-
-In our case, this means `b>b-c>0>-c` and `2(b-c)> b-c` which is trivially true.
-
-As this is an iterated game with no clear finishing point in most circumstances, a tit-for-tat strategy is simple, elegant and functional. To be more theoretically precise, this also requires reasonable assumptions on error rate and discount parameter. This captures notions such as "does the perceived action reflect the intended action" and "how much do you value future (uncertain) actions compared to previous actions". See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation) for more details. In specific circumstances, nodes can choose slightly different policies if there's a strong need for it. A policy is simply how a node chooses to act given a set of circumstances.
-
-A tit-for-tat strategy basically means:
-- cooperate first (perform service/beneficial action to other node)
-- defect when node stops cooperating (disconnect and similar actions), i.e. when it stops performing according to set parameters re settlement
-- resume cooperation if other node does so
-
-This can be complemented with node selection mechanisms.
-
-## SWAP protocol overview
-
-We use SWAP for accounting and settlement in conjunction with other request/reply protocols in Waku v2,
-where accounting is done in a pairwise manner.
-It is an acronym with several possible meanings (as defined in the Book
-of Swarm), for example:
-
-- service wanted and provided
-- settle with automated payments
-- send waiver as payment
-- start without a penny
-
-This approach is based on communicating payment thresholds and sending cheques as indications of later payments.
-Communicating payment thresholds MAY be done out-of-band or as part of the handshake.
-Sending cheques is done once payment threshold is hit.
-
-See [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth) section 3.2. on Peer-to-peer accounting etc., for more context and details.
-
-### Accounting
-
-Nodes perform their own accounting for each relevant peer based on some "volume"/bandwidth metric. For now we take this to mean the number of `WakuMessage`s exchanged.
-
-Additionally, a price is attached to each unit. Currently, this is simply a "karma counter" and equal to 1 per message.
-
-Each accounting balance SHOULD be w.r.t. to a given protocol it is accounting for.
-
-NOTE: This may later be complemented with other metrics, either as part of SWAP or more likely outside of it. For example, online time can be communicated and attested to as a form of enhanced quality of service to inform peer selection.
-
-### Flow
-
-Assuming we have two store nodes, one operating mostly as a client (A) and another as server (B).
-
-1. Node A performs a handshake with B node. B node responds and both nodes communicate their payment threshold.
-2. Node A and B creates an accounting entry for the other peer, keep track of peer and current balance.
-3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`. Based on the number of WakuMessages in the response, both nodes update their accounting records.
-4. When payment threshold is reached, Node A sends over a cheque to reach a neutral balance. Settlement of this is currently out of scope, but would occur through a SWAP contract (to be specified). (mock and hard phase).
-5. If disconnect threshold is reached, Node B disconnects Node A (mock and hard phase).
-
-Note that not all of these steps are mandatory in initial stages, see below for more details. For example, the payment threshold MAY initially be set out of bounds, and policy is only activated in the mock and hard phase.
-
-### Protobufs
-
-We use protobuf to specify the handshake and signature. This current protobuf is a work in progress. This is needed for mock and hard phase.
-
-A handshake gives initial information about payment thresholds and possibly other information. A cheque is best thought of as a promise to pay at a later date.
-
-```protobuf
-
-message Handshake {
-    bytes payment_threshold = 1;
-}
-
-// TODO Signature?
-// Should probably be over the whole Cheque type
-message Cheque {
-    bytes beneficiary = 1;
-    // TODO epoch time or block time?
-    uint32 date = 2;
-    // TODO ERC20 extension?
-    // For now karma counter
-    uint32 amount = 3;
-}
-
-```
-
-## Incremental integration and roll-out
-
-To incrementally integrate this into Waku v2, we have divided up the roll-out into three phases:
-
-- Soft - accounting only
-- Mock - send mock cheques and take word for it
-- Hard Test - blockchain integration and deployed to public testnet (Goerli, Optimism testnet or similar)
-- Hard Main - deployed to a public mainnet
-
-An implementation MAY support any of these phases.
-
-### Soft phase
-
-In the soft phase only accounting is performed, without consequence for the
-peers. No disconnect or sending of cheques is performed at this tage.
-
-SWAP protocol is performed in conjunction with another request-reply protocol to account for its usage.
-It SHOULD be done for [13/WAKU2-STORE](../../standards/core/13/store.md)
-and it MAY be done for other request/reply protocols.
-
-A client SHOULD log accounting state per peer
-and SHOULD indicate when a peer is out of bounds (either of its thresholds met).
-
-### Mock phase
-
-In the mock phase, we send mock cheques and send cheques/disconnect peers as appropriate.
-
-- If a node reaches a disconnect threshold, which MUST be outside the payment threshold, it SHOULD disconnect the other peer.
-- If a node is within payment balance, the other node SHOULD stay connected to it.
-- If a node receives a valid Cheque it SHOULD update its internal accounting records.
-- If any node behaves badly, the other node is free to disconnect and pick another node.
-  - Peer rating is out of scope of this specification.
-
-### Hard phase
-
-In the hard phase, in addition to sending cheques and activating policy, this is
-done with blockchain integration on a public testnet. More details TBD.
-
-This also includes settlements where cheques can be redeemed.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [Prisoner's Dilemma](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma)
-2. [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
-3. [Book of Swarm](https://web.archive.org/web/20210126130038/https://gateway.ethswarm.org/bzz/latest.bookofswarm.eth)
-4. [13/WAKU2-STORE](../../standards/core/13/store.md)
-
-<!--
-
-General TODOs:
-
-- Find new link for book of swarm
-- Illustrate payment and disconnection thresholds (mscgen not great for this?)
-- Elaborate on how accounting works with amount in the context of e.g. store
-- Illustrate flow
-- Specify chequeboo
-
--->

waku/standards/application/20/toy-eth-pm.md

@@ -1,225 +1,262 @@
----
-slug: 20
-title: 20/TOY-ETH-PM
-name: Toy Ethereum Private Message
-status: draft
-tags: waku/application
-editor: Franck Royer <franck@status.im>
-contributors:
----
-
-**Content Topics**:
-
-- Public Key Broadcast: `/eth-pm/1/public-key/proto`,
-- Private Message: `/eth-pm/1/private-message/proto`.
-
-This specification explains the Toy Ethereum Private Message protocol
-which enables a peer to send an encrypted message to another peer
-using the Waku v2 network, and the peer's Ethereum address.
-
-The main purpose of this specification is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
-using Ethereum accounts for identity.
-This protocol caters for Web3 wallets restrictions, allowing it to be implemented only using standard Web3 API.
-In the current state, the protocol has privacy and features [limitations](#limitations), has not been audited
-and hence is not fit for production usage.
-We hope this can be an inspiration for developers wishing to build on top of Waku v2.
-
-## Goal
-
-Alice wants to send an encrypted message to Bob, where only Bob can decrypt the message.
-Alice only knows Bob's Ethereum Address.
-
-## Variables
-
-Here are the variables used in the protocol and their definition:
-
-- `B` is Bob's Ethereum address (or account),
-- `b` is the private key of `B`, and is only known by Bob.
-- `B'` is Bob's Encryption Public Key, for which `b'` is the private key.
-- `M` is the private message that Alice sends to Bob.
-
-## Design Requirements
-
-The proposed protocol MUST adhere to the following design requirements:
-
-1. Alice knows Bob's Ethereum address, 
-2. Bob is willing to participate to Eth-PM, and publishes `B'`,
-3. Bob's ownership of `B'` MUST be verifiable,
-4. Alice wants to send message `M` to Bob,
-5. Bob SHOULD be able to get `M` using [10/WAKU2 spec](../../standards/core/10/waku2.md),
-6. Participants only have access to their Ethereum Wallet via the Web3 API,
-7. Carole MUST NOT be able to read `M`'s content even if she is storing it or relaying it,
-8. [Waku Message Version 1](../../standards/application/26/payload.md) Asymmetric Encryption is used for encryption purposes.
-
-## Limitations
-
-Alice's details are not included in the message's structure,
-meaning that there is no programmatic way for Bob to reply to Alice
-or verify her identity.
-
-Private messages are sent on the same content topic for all users.
-As the recipient data is encrypted, all participants must decrypt all messages which can lead to scalability issues.
-
-This protocol does not guarantee Perfect Forward Secrecy nor Future Secrecy:
-If Bob's private key is compromised, past and future messages could be decrypted.
-A solution combining regular [X3DH](https://www.signal.org/docs/specifications/x3dh/)
-bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) encryption would remove these limitations;
-See the [Status secure transport spec](https://specs.status.im/spec/5) for an example of a protocol that achieves this in a peer-to-peer setting.
-
-Bob MUST decide to participate in the protocol before Alice can send him a message.
-This is discussed in more in details in [Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
-
-## The protocol
-
-### Generate Encryption KeyPair
-
-First, Bob needs to generate a keypair for Encryption purposes.
-
-Bob SHOULD get 32 bytes from a secure random source as Encryption Private Key, `b'`.
-Then Bob can compute the corresponding SECP-256k1 Public Key, `B'`.
-
-### Broadcast Encryption Public Key
-
-For Alice to encrypt messages for Bob,
-Bob SHOULD broadcast his Encryption Public Key `B'`.
-To prove that the Encryption Public Key `B'` is indeed owned by the owner of Bob's Ethereum Account `B`,
-Bob MUST sign `B'` using `B`.
-
-### Sign Encryption Public Key
-
-To prove ownership of the Encryption Public Key,
-Bob must sign it using [EIP-712](https://eips.ethereum.org/EIPS/eip-712) v3,
-meaning calling `eth_signTypedData_v3` on his Wallet's API.
-
-Note: While v4 also exists,
-it is not available on all wallets and the features brought by v4 is not needed for the current use case.
-
-The `TypedData` to be passed to `eth_signTypedData_v3` MUST be as follows, where:
-
-- `encryptionPublicKey` is Bob's Encryption Public Key, `B'`, in hex format, **without** `0x` prefix.
-- `bobAddress` is Bob's Ethereum address, corresponding to `B`, in hex format, **with** `0x` prefix.
-
-```js
-const typedData = {
-    domain: {
-      chainId: 1,
-      name: 'Ethereum Private Message over Waku',
-      version: '1',
-    },
-    message: {
-      encryptionPublicKey: encryptionPublicKey,
-      ownerAddress: bobAddress,
-    },
-    primaryType: 'PublishEncryptionPublicKey',
-    types: {
-      EIP712Domain: [
-        { name: 'name', type: 'string' },
-        { name: 'version', type: 'string' },
-        { name: 'chainId', type: 'uint256' },
-      ],
-      PublishEncryptionPublicKey: [
-        { name: 'encryptionPublicKey', type: 'string' },
-        { name: 'ownerAddress', type: 'string' },
-      ],
-    },
-  }
-```
-
-### Public Key Message
-
-The resulting signature is then included in a `PublicKeyMessage`, where
-
-- `encryption_public_key` is Bob's Encryption Public Key `B'`, not compressed,
-- `eth_address` is Bob's Ethereum Address `B`,
-- `signature` is the EIP-712 as described above.
-
-```protobuf
-syntax = "proto3";
-
-message PublicKeyMessage {
-   bytes encryption_public_key = 1;
-   bytes eth_address = 2;
-   bytes signature = 3;
-}
-```
-
-This MUST be wrapped in a Waku Message version 0, with the Public Key Broadcast content topic.
-Finally, Bob SHOULD publish the message on Waku v2. 
-
-## Consideration for a non-interactive/uncoordinated protocol
-
-Alice has to get Bob's public Key to send a message to Bob.
-Because an Ethereum Address is part of the hash of the public key's account,
-it is not enough in itself to deduce Bob's Public Key.
-
-This is why the protocol dictates that Bob MUST send his Public Key first,
-and Alice MUST receive it before she can send him a message.
-
-Moreover, nim-waku, the reference implementation of [13/WAKU2-STORE](../../standards/core/13/store.md)),
-stores messages for a maximum period of 30 days.
-This means that Bob would need to broadcast his public key at least every 30 days to be reachable.
-
-Below we are reviewing possible solutions to mitigate this "sign up" step.
-
-### Retrieve the public key from the blockchain
-
-If Bob has signed at least one transaction with his account then his Public Key can be extracted from the transaction's ECDSA signature.
-The challenge with this method is that standard Web3 Wallet API does not allow Alice to specifically retrieve all/any transaction sent by Bob.
-
-Alice would instead need to use the `eth.getBlock` API to retrieve Ethereum blocks one by one.
-For each block, she would need to check the `from` value of each transaction until she finds a transaction sent by Bob.
-
-This process is resource intensive and can be slow when using services such as Infura due to rate limits in place,
-which makes it inappropriate for a browser or mobile phone environment.
-
-An alternative would be to either run a backend that can connect directly to an Ethereum node,
-use a centralized blockchain explorer
-or use a decentralized indexing service such as [The Graph](https://thegraph.com/).
-
-Note that these would resolve a UX issue only if a sender wants to proceed with _air drops_.
-
-Indeed, if Bob does not publish his Public Key in the first place
-then it can be an indication that he simply does not participate in this protocol and hence will not receive messages.
-
-However, these solutions would be helpful if the sender wants to proceed with an _air drop_ of messages:
-Send messages over Waku for users to retrieve later, once they decide to participate in this protocol.
-Bob may not want to participate first but may decide to participate at a later stage
-and would like to access previous messages.
-This could make sense in an NFT offer scenario:
-Users send offers to any NFT owner,
-NFT owner may decide at some point to participate in the protocol and retrieve previous offers.
-
-### Publishing the public in long term storage
-
-Another improvement would be for Bob not having to re-publish his public key every 30 days or less.
-Similarly to above, if Bob stops publishing his public key then it may be an indication that he does not participate in the protocol anymore.
-
-In any case, the protocol could be modified to store the Public Key in a more permanent storage, such as a dedicated smart contract on the blockchain.
-
-## Send Private Message
-
-Alice MAY monitor the Waku v2 to collect Ethereum Address and Encryption Public Key tuples.
-Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives are valid as per EIP-712.
-She SHOULD drop any message without a signature or with an invalid signature.
-
-Using Bob's Encryption Public Key, retrieved via [10/WAKU2](../../standards/core/10/waku2.md), Alice MAY now send an encrypted message to Bob.
-
-If she wishes to do so, Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
-as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](../../standards/application/26/payload.md/#asymmetric).
-
-Alice SHOULD now publish this message on the Private Message content topic.
-
-# Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-- [10/WAKU2 spec](../../standards/core/10/waku2.md)
-- [Waku Message Version 1](../../standards/application/26/payload.md)
-- [X3DH](https://www.signal.org/docs/specifications/x3dh/)
-- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
-- [Status secure transport spec](https://specs.status.im/spec/5)
-- [EIP-712](https://eips.ethereum.org/EIPS/eip-712)
-- [13/WAKU2-STORE](../../standards/core/13/store.md))
-- [The Graph](https://thegraph.com/)
-  
+---
+slug: 20
+title: 20/TOY-ETH-PM
+name: Toy Ethereum Private Message
+status: draft
+tags: waku/application
+editor: Franck Royer <franck@status.im>
+contributors:
+---
+
+**Content Topics**:
+
+- Public Key Broadcast: `/eth-pm/1/public-key/proto`,
+- Private Message: `/eth-pm/1/private-message/proto`.
+
+This specification explains the Toy Ethereum Private Message protocol
+which enables a peer to send an encrypted message to another peer
+using the Waku v2 network, and the peer's Ethereum address.
+
+The main purpose of this specification
+is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
+using Ethereum accounts for identity.
+This protocol caters for Web3 wallets restrictions,
+allowing it to be implemented only using standard Web3 API.
+In the current state,
+the protocol has privacy and features [limitations](#limitations),
+has not been audited and hence is not fit for production usage.
+We hope this can be an inspiration for developers
+wishing to build on top of Waku v2.
+
+## Goal
+
+Alice wants to send an encrypted message to Bob, where only Bob can decrypt the message.
+Alice only knows Bob's Ethereum Address.
+
+## Variables
+
+Here are the variables used in the protocol and their definition:
+
+- `B` is Bob's Ethereum address (or account),
+- `b` is the private key of `B`, and is only known by Bob.
+- `B'` is Bob's Encryption Public Key, for which `b'` is the private key.
+- `M` is the private message that Alice sends to Bob.
+
+## Design Requirements
+
+The proposed protocol MUST adhere to the following design requirements:
+
+1. Alice knows Bob's Ethereum address,
+2. Bob is willing to participate to Eth-PM, and publishes `B'`,
+3. Bob's ownership of `B'` MUST be verifiable,
+4. Alice wants to send message `M` to Bob,
+5. Bob SHOULD be able to get `M` using [10/WAKU2 spec](../../core/10/waku2.md),
+6. Participants only have access to their Ethereum Wallet via the Web3 API,
+7. Carole MUST NOT be able to read `M`'s content
+even if she is storing it or relaying it,
+8. [Waku Message Version 1](../26/payload.md) Asymmetric Encryption
+is used for encryption purposes.
+
+## Limitations
+
+Alice's details are not included in the message's structure,
+meaning that there is no programmatic way for Bob to reply to Alice
+or verify her identity.
+
+Private messages are sent on the same content topic for all users.
+As the recipient data is encrypted,
+all participants must decrypt all messages which can lead to scalability issues.
+
+This protocol does not guarantee Perfect Forward Secrecy nor Future Secrecy:
+If Bob's private key is compromised, past and future messages could be decrypted.
+A solution combining regular [X3DH](https://www.signal.org/docs/specifications/x3dh/)
+bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
+encryption would remove these limitations;
+See the [Status secure transport spec](https://specs.status.im/spec/5)
+for an example of a protocol that achieves this in a peer-to-peer setting.
+
+Bob MUST decide to participate in the protocol before Alice can send him a message.
+This is discussed in more in details in
+[Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
+
+## The protocol
+
+### Generate Encryption KeyPair
+
+First, Bob needs to generate a keypair for Encryption purposes.
+
+Bob SHOULD get 32 bytes from a secure random source as Encryption Private Key, `b'`.
+Then Bob can compute the corresponding SECP-256k1 Public Key, `B'`.
+
+### Broadcast Encryption Public Key
+
+For Alice to encrypt messages for Bob,
+Bob SHOULD broadcast his Encryption Public Key `B'`.
+To prove that the Encryption Public Key `B'`
+is indeed owned by the owner of Bob's Ethereum Account `B`,
+Bob MUST sign `B'` using `B`.
+
+### Sign Encryption Public Key
+
+To prove ownership of the Encryption Public Key,
+Bob must sign it using [EIP-712](https://eips.ethereum.org/EIPS/eip-712) v3,
+meaning calling `eth_signTypedData_v3` on his Wallet's API.
+
+Note: While v4 also exists, it is not available on all wallets and
+the features brought by v4 is not needed for the current use case.
+
+The `TypedData` to be passed to `eth_signTypedData_v3` MUST be as follows, where:
+
+- `encryptionPublicKey` is Bob's Encryption Public Key, `B'`,
+in hex format, **without** `0x` prefix.
+- `bobAddress` is Bob's Ethereum address, corresponding to `B`,
+in hex format, **with** `0x` prefix.
+
+```js
+const typedData = {
+    domain: {
+      chainId: 1,
+      name: 'Ethereum Private Message over Waku',
+      version: '1',
+    },
+    message: {
+      encryptionPublicKey: encryptionPublicKey,
+      ownerAddress: bobAddress,
+    },
+    primaryType: 'PublishEncryptionPublicKey',
+    types: {
+      EIP712Domain: [
+        { name: 'name', type: 'string' },
+        { name: 'version', type: 'string' },
+        { name: 'chainId', type: 'uint256' },
+      ],
+      PublishEncryptionPublicKey: [
+        { name: 'encryptionPublicKey', type: 'string' },
+        { name: 'ownerAddress', type: 'string' },
+      ],
+    },
+  }
+```
+
+### Public Key Message
+
+The resulting signature is then included in a `PublicKeyMessage`, where
+
+- `encryption_public_key` is Bob's Encryption Public Key `B'`, not compressed,
+- `eth_address` is Bob's Ethereum Address `B`,
+- `signature` is the EIP-712 as described above.
+
+```protobuf
+syntax = "proto3";
+
+message PublicKeyMessage {
+   bytes encryption_public_key = 1;
+   bytes eth_address = 2;
+   bytes signature = 3;
+}
+```
+
+This MUST be wrapped in a Waku Message version 0,
+with the Public Key Broadcast content topic.
+Finally, Bob SHOULD publish the message on Waku v2.
+
+## Consideration for a non-interactive/uncoordinated protocol
+
+Alice has to get Bob's public Key to send a message to Bob.
+Because an Ethereum Address is part of the hash of the public key's account,
+it is not enough in itself to deduce Bob's Public Key.
+
+This is why the protocol dictates that Bob MUST send his Public Key first,
+and Alice MUST receive it before she can send him a message.
+
+Moreover, nim-waku, the reference implementation of [13/WAKU2-STORE](../../core/13/store.md),
+stores messages for a maximum period of 30 days.
+This means that Bob would need to broadcast his public key
+at least every 30 days to be reachable.
+
+Below we are reviewing possible solutions to mitigate this "sign up" step.
+
+### Retrieve the public key from the blockchain
+
+If Bob has signed at least one transaction with his account
+then his Public Key can be extracted from the transaction's ECDSA signature.
+The challenge with this method is that standard Web3 Wallet API
+does not allow Alice to specifically retrieve all/any transaction sent by Bob.
+
+Alice would instead need to use the `eth.getBlock` API
+to retrieve Ethereum blocks one by one.
+For each block, she would need to check the `from` value of each transaction
+until she finds a transaction sent by Bob.
+
+This process is resource intensive and
+can be slow when using services such as Infura due to rate limits in place,
+which makes it inappropriate for a browser or mobile phone environment.
+
+An alternative would be to either run a backend
+that can connect directly to an Ethereum node,
+use a centralized blockchain explorer
+or use a decentralized indexing service such as [The Graph](https://thegraph.com/).
+
+Note that these would resolve a UX issue
+only if a sender wants to proceed with _air drops_.
+
+Indeed, if Bob does not publish his Public Key in the first place
+then it can be an indication that he simply does not participate in this protocol
+and hence will not receive messages.
+
+However, these solutions would be helpful
+if the sender wants to proceed with an _air drop_ of messages:
+Send messages over Waku for users to retrieve later,
+once they decide to participate in this protocol.
+Bob may not want to participate first but may decide to participate at a later stage
+and would like to access previous messages.
+This could make sense in an NFT offer scenario:
+Users send offers to any NFT owner,
+NFT owner may decide at some point to participate in the protocol and
+retrieve previous offers.
+
+### Publishing the public in long term storage
+
+Another improvement would be for Bob not having to re-publish his public key
+every 30 days or less.
+Similarly to above,
+if Bob stops publishing his public key
+then it may be an indication that he does not participate in the protocol anymore.
+
+In any case,
+the protocol could be modified to store the Public Key in a more permanent storage,
+such as a dedicated smart contract on the blockchain.
+
+## Send Private Message
+
+Alice MAY monitor the Waku v2 to collect Ethereum Address and
+Encryption Public Key tuples.
+Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives
+are valid as per EIP-712.
+She SHOULD drop any message without a signature or with an invalid signature.
+
+Using Bob's Encryption Public Key,
+retrieved via [10/WAKU2 spec](../../core/10/waku2.md),
+Alice MAY now send an encrypted message to Bob.
+
+If she wishes to do so,
+Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
+as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](../26/payload.md/#asymmetric).
+
+Alice SHOULD now publish this message on the Private Message content topic.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [10/WAKU2 spec](../../core/10/waku2.md)
+- [Waku Message Version 1](../26/payload.md)
+- [X3DH](https://www.signal.org/docs/specifications/x3dh/)
+- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
+- [Status secure transport spec](https://specs.status.im/spec/5)
+- [EIP-712](https://eips.ethereum.org/EIPS/eip-712)
+- [13/WAKU2-STORE](../../core/13/store.md)
+- [The Graph](https://thegraph.com/)
+  

waku/standards/application/21/fault-tolerant-store.md

@@ -1,74 +1,115 @@
----
-slug: 21
-title: 21/WAKU2-FAULT-TOLERANT-STORE
-name: Waku v2 Fault-Tolerant Store
-status: draft
-editor: Sanaz Taheri <sanaz@status.im>
-contributors:
----
-
- The reliability of [13/WAKU2-STORE](../../standards/core/13/store.md) protocol heavily relies on the fact that full nodes i.e., those who persist messages have high availability and uptime and do not miss any messages. 
- If a node goes offline, then it will risk missing all the messages transmitted in the network during that time. 
- In this specification, we provide a method that makes the store protocol resilient in presence of faulty nodes. 
- Relying on this method,  nodes that have been offline for a time window will be able to fix the gap in their message history when getting back online. 
- Moreover, nodes with lower availability and uptime can leverage this method to reliably provide the store protocol services as a full node.
-
-## Method description 
- As the first step towards making the [13/WAKU2-STORE](../../standards/core/13/store.md) protocol fault-tolerant, we introduce a new type of time-based query through which nodes fetch message history from each other based on their desired time window. 
- This method operates based on the assumption that the querying node knows some other nodes in the store protocol which have been online for that targeted time window.  
-
-## Security Consideration
-
-The main security consideration to take into account while using this method is that a querying node has to reveal its offline time to the queried node. 
-This will gradually result in the extraction of the node's activity pattern which can lead to inference attacks. 
-
-## Wire Specification
-We extend the [HistoryQuery](../../standards/core/13/store.md/#payloads) protobuf message with two fields of `start_time` and `end_time` to signify the time range to be queried. 
-
-### Payloads
-
-```diff
-syntax = "proto3";
-
-message HistoryQuery {
-  // the first field is reserved for future use
-  string pubsubtopic = 2;
-  repeated ContentFilter contentFilters = 3;
-  PagingInfo pagingInfo = 4;
-  + sint64 start_time = 5;
-  + sint64 end_time = 6;
-}
-
-```
-  
-### HistoryQuery
-
-RPC call to query historical messages.
-- `start_time`: this field MAY be filled out to signify the starting point of the queried time window. 
-  This field holds the Unix epoch time in nanoseconds.  
-  The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../standards/core/14/message.md/#Payloads) is larger than or equal to the `start_time`.
-- `end_time` this field MAY be filled out to signify the ending point of the queried time window. 
-  This field holds the Unix epoch time in nanoseconds.
-  The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages whose [`timestamp`](../../standards/core/14/message.md/#Payloads) is less than or equal to the `end_time`.
-
-  A time-based query is considered valid if its `end_time` is larger than or equal to the `start_time`. 
-  Queries that do not adhere to this condition will not get through e.g. an open-end time query in which the `start_time` is given but no  `end_time` is supplied is not valid. 
-  If both `start_time` and `end_time` are omitted then no time-window filter takes place. 
-
-
-
-In order to account for nodes asynchrony, and assuming that nodes may be out of sync for at most 20 seconds (i.e., 20000000000 nanoseconds), the querying nodes SHOULD add an offset of 20 seconds to their offline time window. 
-That is if the original window is [`l`,`r`] then the history query SHOULD be made for `[start_time: l - 20s, end_time: r + 20s]`.
-
-Note that `HistoryQuery` preserves `AND` operation among the queried attributes. 
-As such,  The `messages` field of the corresponding [`HistoryResponse`](../../standards/core/13/store.md/#HistoryResponse) MUST contain historical waku messages that satisfy the indicated  `pubsubtopic` AND `contentFilters` AND the time range [`start_time`, `end_time`]. 
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [13/WAKU2-STORE](../../standards/core/13/store.md)
-- [`timestamp`](../../standards/core/14/message.md/#Payloads)
+---
+slug: 21
+title: 21/WAKU2-FAULT-TOLERANT-STORE
+name: Waku v2 Fault-Tolerant Store
+status: draft
+editor: Sanaz Taheri <sanaz@status.im>
+contributors:
+---
+
+ The reliability of [13/WAKU2-STORE](../../core/13/store.md)
+protocol heavily relies on the fact that full nodes i.e.,
+those who persist messages have high availability and
+uptime and do not miss any messages.
+If a node goes offline,
+then it will risk missing all the messages transmitted
+in the network during that time.
+In this specification,
+we provide a method that makes the store protocol resilient
+in presence of faulty nodes.
+Relying on this method,
+nodes that have been offline for a time window will be able to fix the gap
+in their message history when getting back online.
+Moreover, nodes with lower availability and
+uptime can leverage this method to reliably provide the store protocol services
+as a full node.
+
+## Method description
+
+ As the first step
+towards making the [13/WAKU2-STORE](../../core/13/store.md) protocol fault-tolerant,
+we introduce a new type of time-based query through which nodes fetch message history
+from each other based on their desired time window.
+This method operates based on the assumption that the querying node
+knows some other nodes in the store protocol
+which have been online for that targeted time window.  
+
+## Security Consideration
+
+The main security consideration to take into account
+while using this method is that a querying node
+has to reveal its offline time to the queried node.
+This will gradually result in the extraction of the node's activity pattern
+which can lead to inference attacks.
+
+## Wire Specification
+
+We extend the [HistoryQuery](../../core/13/store.md/#payloads) protobuf message
+with two fields of `start_time` and `end_time` to signify the time range to be queried.
+
+### Payloads
+
+```diff
+syntax = "proto3";
+
+message HistoryQuery {
+  // the first field is reserved for future use
+  string pubsubtopic = 2;
+  repeated ContentFilter contentFilters = 3;
+  PagingInfo pagingInfo = 4;
+  + sint64 start_time = 5;
+  + sint64 end_time = 6;
+}
+
+```
+  
+### HistoryQuery
+
+RPC call to query historical messages.
+
+- `start_time`:
+this field MAY be filled out to signify the starting point of the queried time window.
+This field holds the Unix epoch time in nanoseconds.  
+The `messages` field of the corresponding
+[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
+MUST contain historical waku messages whose
+[`timestamp`](../../core/14/message.md/#Payloads)
+is larger than or equal to the `start_time`.
+- `end_time`:
+this field MAY be filled out to signify the ending point of the queried time window.
+This field holds the Unix epoch time in nanoseconds.
+The `messages` field of the corresponding
+[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
+MUST contain historical waku messages whose
+[`timestamp`](../../core/14/message.md/#Payloads) is less than or equal to the `end_time`.
+
+A time-based query is considered valid if
+its `end_time` is larger than or equal to the `start_time`.
+Queries that do not adhere to this condition will not get through e.g.
+an open-end time query in which the `start_time` is given but
+no  `end_time` is supplied is not valid.
+If both `start_time` and
+`end_time` are omitted then no time-window filter takes place.
+
+In order to account for nodes asynchrony, and
+assuming that nodes may be out of sync for at most 20 seconds
+(i.e., 20000000000 nanoseconds),
+the querying nodes SHOULD add an offset of 20 seconds to their offline time window.
+That is if the original window is [`l`,`r`]
+then the history query SHOULD be made for `[start_time: l - 20s, end_time: r + 20s]`.
+
+Note that `HistoryQuery` preserves `AND` operation among the queried attributes.
+As such, the `messages` field of the corresponding
+[`HistoryResponse`](../../core/13/store.md/#HistoryResponse)
+MUST contain historical waku messages that satisfy the indicated  `pubsubtopic` AND
+`contentFilters` AND the time range [`start_time`, `end_time`].
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [13/WAKU2-STORE](../../core/13/store.md)
+- [`timestamp`](../../standards/core/14/message.md/#Payloads)

waku/standards/application/26/payload.md

@@ -1,154 +1,208 @@
----
-slug: 26
-title: 26/WAKU2-PAYLOAD
-name: Waku Message Payload Encryption
-status: draft
-editor: Oskar Thoren <oskarth@titanproxy.com>
-contributors:
----
-
-This specification describes how Waku provides confidentiality, authenticity, and integrity, as well as some form of unlinkability.
-Specifically, it describes how encryption, decryption and signing works in [6/WAKU1](../../standards/core/6/waku1.md) and in [10/WAKU2 spec](../../standards/core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../standards/core/14/message.md/#version1).
-
-This specification effectively replaces [7/WAKU-DATA](../../standards/application/7/DATA.md) as well as [6/WAKU1 Payload encryption](../../standards/core/6/waku1.md/#payload-encryption) but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
-
-Large sections of the specification originate from [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) with some modifications.
-
-## Design requirements
-
-- *Confidentiality*: The adversary should not be able to learn what data is being sent from one Waku node to one or several other Waku nodes.
-- *Authenticity*: The adversary should not be able to cause Waku endpoint to accept data from any third party as though it came from the other endpoint.
-- *Integrity*: The adversary should not be able to cause a Waku endpoint to accept data that has been tampered with.
-
-Notable, *forward secrecy* is not provided for at this layer.
-If this property is desired, a more fully featured secure communication protocol can be used on top, such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
-
-It also provides some form of *unlinkability* since:
-- only participants who are able to decrypt a message can see its signature
-- payload are padded to a fixed length
-
-## Cryptographic primitives
-
-- AES-256-GCM (for symmetric encryption)
-- ECIES
-- ECDSA
-- KECCAK-256
-
-ECIES is using the following cryptosystem:
-- Curve: secp256k1
-- KDF: NIST SP 800-56 Concatenation Key Derivation Function, with SHA-256 option
-- MAC: HMAC with SHA-256
-- AES: AES-128-CTR
-
-## Specification
-
-For 6/WAKU1, the `data` field is used in the `waku envelope`, and the field MAY contain the encrypted payload.
-
-For 10/WAKU2, the `payload` field is used in `WakuMessage` and MAY contain the encrypted payload.
-
-The fields that are concatenated and encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
- - flags
- - payload-length
- - payload
- - padding
- - signature
-
-### ABNF
-
-Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
-
-```abnf
-; 1 byte; first two bits contain the size of payload-length field,
-; third bit indicates whether the signature is present.
-flags           = 1OCTET
-
-; contains the size of payload.
-payload-length  = 4*OCTET
-
-; byte array of arbitrary size (may be zero).
-payload         = *OCTET
-
-; byte array of arbitrary size (may be zero).
-padding         = *OCTET
-
-; 65 bytes, if present.
-signature       = 65OCTET
-
-data            = flags payload-length payload padding [signature]
-
-; This field is called payload in Waku v2
-payload         = data
-```
-
-### Signature
-
-Those unable to decrypt the payload/data are also unable to access the signature.
-The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity.
-The signature is serialized as the concatenation of the `r`, `s` and `v` parameters of the SECP-256k1 ECDSA signature, in that order.
-`r` and `s` MUST be big-endian encoded, fixed-width 256-bit unsigned.
-`v` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
-
-See [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf) for more information on signature generation, parameters and public key recovery.
-
-### Encryption
-
-#### Symmetric
-
-Symmetric encryption uses AES-256-GCM for [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption). 
-The output of encryption is of the form (`ciphertext`, `tag`, `iv`) where `ciphertext` is the encrypted message, `tag` is a 16 byte message authentication tag and `iv` is a 12 byte initialization vector  (nonce). 
-The message authentication `tag` and initialization vector `iv` field MUST be appended to the resulting `ciphertext`, in that order.
-Note that previous specifications and some implementations might refer to `iv` as `nonce` or `salt`.
-
-#### Asymmetric
-
-Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme (ECIES) with SECP-256k1 public key.
-
-#### ECIES
-
-This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption) spec with minor modifications.
-
-The cryptosystem used is:
-
-- The elliptic curve secp256k1 with generator `G`.
-- `KDF(k, len)`: the NIST SP 800-56 Concatenation Key Derivation Function.
-- `MAC(k, m)`: HMAC using the SHA-256 hash function.
-- `AES(k, iv, m)`: the AES-128 encryption function in CTR mode.
-
-Special notation used: `X || Y` denotes concatenation of `X` and `Y`.
-
-Alice wants to send an encrypted message that can be decrypted by Bob's static private key `kB`. Alice knows about Bobs static public key `KB`.
-
-To encrypt the message `m`, Alice generates a random number `r` and corresponding elliptic curve public key `R = r * G` and computes the shared secret `S = Px` where `(Px, Py) = r * KB`.
-She derives key material for encryption and authentication as `kE || kM = KDF(S, 32)` as well as a random initialization vector `iv`.
-Alice sends the encrypted message `R || iv || c || d` where `c = AES(kE, iv , m)` and `d = MAC(sha256(kM), iv || c)` to Bob.
-
-For Bob to decrypt the message `R || iv || c || d`, he derives the shared secret `S = Px` where `(Px, Py) = kB * R` as well as the encryption and authentication keys `kE || kM = KDF(S, 32)`.
-Bob verifies the authenticity of the message by checking whether `d == MAC(sha256(kM), iv || c)` then obtains the plaintext as `m = AES(kE, iv || c)`.
-
-### Padding
-
-The padding field is used to align data size, since data size alone might reveal important metainformation.
-Padding can be arbitrary size.
-However, it is recommended that the size of Data Field (excluding the IV and tag) before encryption (i.e. plain text) SHOULD be a multiple of 256 bytes.
-
-### Decoding a message
-
-In order to decode a message, a node SHOULD try to apply both symmetric and asymmetric decryption operations.
-This is because the type of encryption is not included in the message.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [6/WAKU1](../../standards/core/6/waku1.md)
-2. [10/WAKU2 spec](../../standards/core/10/waku2.md)
-3. [14/WAKU-MESSAGE version 1](../../standards/core/14/message.md/#version1)
-4. [7/WAKU-DATA](../../standards/application/7/DATA.md)
-5. [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627)
-6. [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
-7. [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
-8. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
-9. [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
-10. [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption)
+---
+slug: 26
+title: 26/WAKU2-PAYLOAD
+name: Waku Message Payload Encryption
+status: draft
+editor: Oskar Thoren <oskarth@titanproxy.com>
+contributors:
+---
+
+This specification describes how Waku provides confidentiality, authenticity, and
+integrity, as well as some form of unlinkability.
+Specifically, it describes how encryption, decryption and
+signing works in [6/WAKU1](../../legacy/6/waku1.md) and
+in [10/WAKU2 spec](../../core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1).
+
+This specification effectively replaces [7/WAKU-DATA](../../legacy/7/data.md)
+as well as [6/WAKU1 Payload encryption](../../legacy/6/waku1.md/#payload-encryption)
+but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
+
+Large sections of the specification originate from
+[EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from
+[RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
+with some modifications.
+
+## Design requirements
+
+- *Confidentiality*:
+The adversary should not be able to learn what data is being sent from one Waku node
+to one or several other Waku nodes.
+- *Authenticity*:
+The adversary should not be able to cause Waku endpoint
+to accept data from any third party as though it came from the other endpoint.
+- *Integrity*:
+The adversary should not be able to cause a Waku endpoint to
+accept data that has been tampered with.
+
+Notable, *forward secrecy* is not provided for at this layer.
+If this property is desired,
+a more fully featured secure communication protocol can be used on top,
+such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
+
+It also provides some form of *unlinkability* since:
+
+- only participants who are able to decrypt a message can see its signature
+- payload are padded to a fixed length
+
+## Cryptographic primitives
+
+- AES-256-GCM (for symmetric encryption)
+- ECIES
+- ECDSA
+- KECCAK-256
+
+ECIES is using the following cryptosystem:
+
+- Curve: secp256k1
+- KDF: NIST SP 800-56 Concatenation Key Derivation Function, with SHA-256 option
+- MAC: HMAC with SHA-256
+- AES: AES-128-CTR
+
+## Specification
+
+For [6/WAKU1](../../legacy/6/waku1.md),
+the `data` field is used in the `waku envelope`,
+and the field MAY contain the encrypted payload.
+
+For [10/WAKU2 spec](../../core/10/waku2.md),
+the `payload` field is used in `WakuMessage` and
+MAY contain the encrypted payload.
+
+The fields that are concatenated and
+encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
+
+- flags
+- payload-length
+- payload
+- padding
+- signature
+
+### ABNF
+
+Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
+we have the following format:
+
+```abnf
+; 1 byte; first two bits contain the size of payload-length field,
+; third bit indicates whether the signature is present.
+flags           = 1OCTET
+
+; contains the size of payload.
+payload-length  = 4*OCTET
+
+; byte array of arbitrary size (may be zero).
+payload         = *OCTET
+
+; byte array of arbitrary size (may be zero).
+padding         = *OCTET
+
+; 65 bytes, if present.
+signature       = 65OCTET
+
+data            = flags payload-length payload padding [signature]
+
+; This field is called payload in Waku v2
+payload         = data
+```
+
+### Signature
+
+Those unable to decrypt the payload/data are also unable to access the signature.
+The signature, if provided,
+is the ECDSA signature of the Keccak-256 hash of the unencrypted data
+using the secret key of the originator identity.
+The signature is serialized as the concatenation of the `r`, `s` and `v` parameters
+of the SECP-256k1 ECDSA signature, in that order.
+`r` and `s` MUST be big-endian encoded, fixed-width 256-bit unsigned.
+`v` MUST be an 8-bit big-endian encoded,
+non-normalized and should be either 27 or 28.
+
+See [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
+for more information on signature generation, parameters and public key recovery.
+
+### Encryption
+
+#### Symmetric
+
+Symmetric encryption uses AES-256-GCM for
+[authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption).
+The output of encryption is of the form (`ciphertext`, `tag`, `iv`)
+where `ciphertext` is the encrypted message,
+`tag` is a 16 byte message authentication tag and
+`iv` is a 12 byte initialization vector  (nonce).
+The message authentication `tag` and
+initialization vector `iv` field MUST be appended to the resulting `ciphertext`,
+in that order.
+Note that previous specifications and
+some implementations might refer to `iv` as `nonce` or `salt`.
+
+#### Asymmetric
+
+Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme
+(ECIES) with SECP-256k1 public key.
+
+#### ECIES
+
+This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
+spec with minor modifications.
+
+The cryptosystem used is:
+
+- The elliptic curve secp256k1 with generator `G`.
+- `KDF(k, len)`: the NIST SP 800-56 Concatenation Key Derivation Function.
+- `MAC(k, m)`: HMAC using the SHA-256 hash function.
+- `AES(k, iv, m)`: the AES-128 encryption function in CTR mode.
+
+Special notation used: `X || Y` denotes concatenation of `X` and `Y`.
+
+Alice wants to send an encrypted message that can be decrypted by
+Bob's static private key `kB`.
+Alice knows about Bobs static public key `KB`.
+
+To encrypt the message `m`, Alice generates a random number `r` and
+corresponding elliptic curve public key `R = r * G` and
+computes the shared secret `S = Px` where `(Px, Py) = r * KB`.
+She derives key material for encryption and
+authentication as `kE || kM = KDF(S, 32)`
+as well as a random initialization vector `iv`.
+Alice sends the encrypted message `R || iv || c || d` where `c = AES(kE, iv , m)`
+and `d = MAC(sha256(kM), iv || c)` to Bob.
+
+For Bob to decrypt the message `R || iv || c || d`,
+he derives the shared secret `S = Px` where `(Px, Py) = kB * R`
+as well as the encryption and authentication keys `kE || kM = KDF(S, 32)`.
+Bob verifies the authenticity of the message
+by checking whether `d == MAC(sha256(kM), iv || c)`
+then obtains the plaintext as `m = AES(kE, iv || c)`.
+
+### Padding
+
+The padding field is used to align data size,
+since data size alone might reveal important metainformation.
+Padding can be arbitrary size.
+However, it is recommended that the size of Data Field
+(excluding the IV and tag) before encryption (i.e. plain text)
+SHOULD be a multiple of 256 bytes.
+
+### Decoding a message
+
+In order to decode a message, a node SHOULD try to apply both symmetric and
+asymmetric decryption operations.
+This is because the type of encryption is not included in the message.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [6/WAKU1](../../legacy/6/waku1.md)
+2. [10/WAKU2 spec](../../core/10/waku2.md)
+3. [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1)
+4. [7/WAKU-DATA](../../legacy/7/data.md)
+5. [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627)
+6. [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
+7. [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
+8. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
+9. [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
+10. [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption)

waku/standards/application/53/x3dh.md

@@ -1,258 +1,316 @@
----
-slug: 53
-title: 53/WAKU2-X3DH
-name: X3DH usage for Waku payload encryption
-status: draft
-category: Standards Track
-tags: waku-application
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-- Andrea Piana <andreap@status.im>
-- Pedro Pombeiro <pedro@status.im>
-- Corey Petty <corey@status.im>
-- Oskar Thorén <oskarth@titanproxy.com>
-- Dean Eigenmann <dean@status.im>
----
-
-## Abstract
-
-This document describes a method that can be used to provide a secure channel between two peers, and thus provide confidentiality, integrity, authenticity and forward secrecy. 
-It is transport-agnostic and works over asynchronous networks.
-
-It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications, with some adaptations to operate in a decentralized environment.
-
-## Motivation
-
-Nodes on a network may want to communicate with each other in a secure manner, without other nodes network being able to read their messages.
-
-## Specification
-
-### Definitions
-
-- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols which provide assurances that session keys will not be compromised even if the private keys of the participants are compromised. 
-Specifically, past messages cannot be decrypted by a third-party who manages to get a hold of a private key.
-
-- **Secret channel** describes a communication channel where a Double Ratchet algorithm is in use.
-
-
-### Design Requirements
-
-- **Confidentiality**: The adversary should not be able to learn what data is being exchanged between two Status clients.
-- **Authenticity**: The adversary should not be able to cause either endpoint to accept data from any third party as though it came from the other endpoint.
-- **Forward Secrecy**: The adversary should not be able to learn what data was exchanged between two clients if, at some later time, the adversary compromises one or both of the endpoints.
-- **Integrity**: The adversary should not be able to cause either endpoint to accept data that has been tampered with.
-
-All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
-
-### Conventions
-
-Types used in this specification are defined using the [Protobuf](https://developers.google.com/protocol-buffers/) wire format.
-
-## Specification
-
-### End-to-End Encryption
-
-End-to-end encryption (E2EE) takes place between two clients. 
-The main cryptographic protocol is a Double Ratchet protocol, which is derived from the [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html), using a different ratchet. 
-[The Waku v2 protocol](../../standards/core/10/waku2.md) subsequently encrypts the message payload, using symmetric key encryption. 
-Furthermore, the concept of prekeys (through the use of [X3DH](https://signal.org/docs/specifications/x3dh/)) is used to allow the protocol to operate in an asynchronous environment.
-It is not necessary for two parties to be online at the same time to initiate an encrypted conversation.
-
-### Cryptographic Protocols
-
-This protocol uses the following cryptographic primitives:
-- X3DH
-    - Elliptic curve Diffie-Hellman key exchange (secp256k1)
-    - KECCAK-256
-    - ECDSA
-    - ECIES
-- Double Ratchet
-    - HMAC-SHA-256 as MAC
-    - Elliptic curve Diffie-Hellman key exchange (Curve25519)
-    - AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
-
-    The node achieves key derivation using [HKDF](https://www.rfc-editor.org/rfc/rfc5869).
-
-### Pre-keys
-
-Every client SHOULD initially generate some key material which is stored locally:
-- Identity keypair based on secp256k1 - `IK`
-- A signed prekey based on secp256k1 - `SPK`
-- A prekey signature - `Sig(IK, Encode(SPK))`
-
-More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles).
-
-Prekey bundles MAY be extracted from any peer's messages, or found via searching for their specific topic, `{IK}-contact-code`.
-
-The following methods can be used to retrieve prekey bundles from a peer's messages:
-- contact codes;
-- public and one-to-one chats;
-- QR codes;
-- ENS record;
-- Decentralized permanent storage (e.g. Swarm, IPFS).
-- Waku
-
-Waku SHOULD be used for retrieving prekey bundles.
-
-Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys, the bundle MAY be rotated every 24 hours, and distributed via Waku.
-
-### Flow
-
-The key exchange can be summarized as follows:
-
-1. Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles, and derive a shared secret.
-
-2. Double Ratchet: The two parties use the shared secret to derive a new encryption key for each message they send.
-
-3. Chain key update: The two parties update their chain keys. The chain key is used to derive new encryption keys for future messages.
-
-4. Message key derivation: The two parties derive a new message key from their chain key, and use it to encrypt a message.
-
-#### 1. Initial key exchange flow (X3DH)
-
-[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message) describes the initial key exchange flow, with some additional context:
-- The peers' identity keys `IK_A` and `IK_B` correspond to their public keys;
-- Since it is not possible to guarantee that a prekey will be used only once in a decentralized world, the one-time prekey `OPK_B` is not used in this scenario;
-- Nodes SHOULD not send Bundles to a centralized server, but instead provide them in a decentralized way as described in the [Pre-keys section](#pre-keys).
-
-Alice retrieves Bob's prekey bundle, however it is not specific to Alice. It contains:
-
-([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12))
-
-**Wire format:**
-
-``` protobuf
-// X3DH prekey bundle
-message Bundle {
-  // Identity key 'IK_B'
-  bytes identity = 1;
-  // Signed prekey 'SPK_B' for each device, indexed by 'installation-id'
-  map<string,SignedPreKey> signed_pre_keys = 2;
-  // Prekey signature 'Sig(IK_B, Encode(SPK_B))'
-  bytes signature = 4;
-  // When the bundle was created locally
-  int64 timestamp = 5;
-}
-```
-
-([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L5))
-
-``` protobuf
-message SignedPreKey {
-  bytes signed_pre_key = 1;
-  uint32 version = 2;
-}
-```
-
-The `signature` is generated by sorting `installation-id` in lexicographical order, and concatenating the `signed-pre-key` and `version`:
-
-`installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2`
-
-#### 2. Double Ratchet
-
-Having established the initial shared secret `SK` through X3DH, it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
-
-Refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/) for more details.
-
-The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage` ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65)) containing a map of `DirectMessageProtocol` indexed by `installation-id` ([reference wire format](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
-
-``` protobuf
-message ProtocolMessage {
-  // The installation id of the sender
-  string installation_id = 2;
-  // A sequence of bundles
-  repeated Bundle bundles = 3;
-  // One to one message, encrypted, indexed by installation_id
-  map<string,DirectMessageProtocol> direct_message = 101;
-  // Public message, not encrypted
-  bytes public_message = 102;
-}
-```
-
-``` protobuf
-message EncryptedMessageProtocol {
-  X3DHHeader X3DH_header = 1;
-  DRHeader DR_header = 2; 
-  DHHeader DH_header = 101;
-  // Encrypted payload
-  // if a bundle is available, contains payload encrypted with the Double Ratchet algorithm;
-  // otherwise, payload encrypted with output key of DH exchange (no Perfect Forward Secrecy).
-  bytes payload = 3;
-}
-```
-Where:
-- `X3DH_header`: the `X3DHHeader` field in `DirectMessageProtocol` contains:
-
-    ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
-    ``` protobuf
-    message X3DHHeader {
-      // Alice's ephemeral key `EK_A`
-      bytes key = 1;
-      // Bob's bundle signed prekey
-      bytes id = 4;
-    }
-    ```
-
-- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)). Used when Bob's public bundle is available:
-    ``` protobuf
-    message DRHeader {
-      // Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet))
-      bytes key = 1;
-      // number of the message in the sending chain
-      uint32 n = 2;
-      // length of the previous sending chain
-      uint32 pn = 3;
-      // Bob's bundle ID
-      bytes id = 4;
-    }
-    ```
-
-- `DH_header`: Diffie-Hellman header (used when Bob's bundle is not available):
-    ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
-    ``` protobuf
-    message DHHeader {
-      // Alice's compressed ephemeral public key.
-      bytes key = 1;
-    }
-    ```
-
-#### 3. Chain key update
-
-The chain key MUST be updated according to the `DR_Header` received in the `EncryptedMessageProtocol` message, described in [2.Double Ratchet](#2-double-ratchet).
-
-#### 4. Message key derivation
-
-The message key MUST be derived from a single ratchet step in the symmetric-key ratchet as described in [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
-
-The message key MUST be used to encrypt the next message to be sent.
-
-## Security Considerations
-
-1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
-
-2. Inherits the security considerations of the [Waku v2 protocol](../../standards/core/10/waku2.md).
-
-3. The protocol is designed to be used in a decentralized manner, however, it is possible to use a centralized server to serve prekey bundles. In this case, the server is trusted.
-
-## Privacy Considerations
-
-1. This protocol does not provide message unlinkability. It is possible to link messages signed by the same keypair.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [X3DH](https://signal.org/docs/specifications/x3dh/)
-- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
-- [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
-- [Protobuf](https://developers.google.com/protocol-buffers/)
-- [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html)
-- [The Waku v2 protocol](../../standards/core/10/waku2.md)
-- [HKDF](https://www.rfc-editor.org/rfc/rfc5869)
-- [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles)
-- [reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12)
-- [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
-- 
-- 
+---
+slug: 53
+title: 53/WAKU2-X3DH
+name: X3DH usage for Waku payload encryption
+status: draft
+category: Standards Track
+tags: waku-application
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Andrea Piana <andreap@status.im>
+- Pedro Pombeiro <pedro@status.im>
+- Corey Petty <corey@status.im>
+- Oskar Thorén <oskarth@titanproxy.com>
+- Dean Eigenmann <dean@status.im>
+---
+
+## Abstract
+
+This document describes a method that can be used to provide a secure channel
+between two peers, and thus provide confidentiality, integrity,
+authenticity and forward secrecy.
+It is transport-agnostic and works over asynchronous networks.
+
+It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/)
+and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications,
+with some adaptations to operate in a decentralized environment.
+
+## Motivation
+
+Nodes on a network may want to communicate with each other in a secure manner,
+without other nodes network being able to read their messages.
+
+## Specification
+
+### Definitions
+
+- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols
+which provide assurances that session keys will not be compromised
+even if the private keys of the participants are compromised.
+Specifically, past messages cannot be decrypted by a third-party
+who manages to get a hold of a private key.
+
+- **Secret channel** describes a communication channel
+where a Double Ratchet algorithm is in use.
+
+### Design Requirements
+
+- **Confidentiality**:
+The adversary should not be able to learn what data is being exchanged
+between two Status clients.
+- **Authenticity**:
+The adversary should not be able to cause either endpoint
+to accept data from any third party as though it came from the other endpoint.
+- **Forward Secrecy**:
+The adversary should not be able to learn what data was exchanged
+between two clients if, at some later time,
+the adversary compromises one or both of the endpoints.
+- **Integrity**:
+The adversary should not be able to cause either endpoint
+to accept data that has been tampered with.
+
+All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
+
+### Conventions
+
+Types used in this specification are defined using the
+[Protobuf](https://developers.google.com/protocol-buffers/) wire format.
+
+### End-to-End Encryption
+
+End-to-end encryption (E2EE) takes place between two clients.
+The main cryptographic protocol is a Double Ratchet protocol,
+which is derived from the
+[Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html),
+using a different ratchet.
+[The Waku v2 protocol](../../core/10/waku2.md)
+subsequently encrypts the message payload, using symmetric key encryption.
+Furthermore, the concept of prekeys
+(through the use of [X3DH](https://signal.org/docs/specifications/x3dh/))
+is used to allow the protocol to operate in an asynchronous environment.
+It is not necessary for two parties to be online at the same time
+to initiate an encrypted conversation.
+
+### Cryptographic Protocols
+
+This protocol uses the following cryptographic primitives:
+
+- X3DH
+  - Elliptic curve Diffie-Hellman key exchange (secp256k1)
+  - KECCAK-256
+  - ECDSA
+  - ECIES
+- Double Ratchet
+  - HMAC-SHA-256 as MAC
+  - Elliptic curve Diffie-Hellman key exchange (Curve25519)
+  - AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
+
+    The node achieves key derivation using [HKDF](https://www.rfc-editor.org/rfc/rfc5869).
+
+### Pre-keys
+
+Every client SHOULD initially generate some key material which is stored locally:
+
+- Identity keypair based on secp256k1 - `IK`
+- A signed prekey based on secp256k1 - `SPK`
+- A prekey signature - `Sig(IK, Encode(SPK))`
+
+More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles).
+
+Prekey bundles MAY be extracted from any peer's messages,
+or found via searching for their specific topic, `{IK}-contact-code`.
+
+The following methods can be used to retrieve prekey bundles from a peer's messages:
+
+- contact codes;
+- public and one-to-one chats;
+- QR codes;
+- ENS record;
+- Decentralized permanent storage (e.g. Swarm, IPFS).
+- Waku
+
+Waku SHOULD be used for retrieving prekey bundles.
+
+Since bundles stored in QR codes or
+ENS records cannot be updated to delete already used keys,
+the bundle MAY be rotated every 24 hours, and distributed via Waku.
+
+### Flow
+
+The key exchange can be summarized as follows:
+
+1. Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles,
+and derive a shared secret.
+
+2. Double Ratchet:
+The two parties use the shared secret to derive a new encryption key
+for each message they send.
+
+3. Chain key update: The two parties update their chain keys.
+The chain key is used to derive new encryption keys for future messages.
+
+4. Message key derivation:
+The two parties derive a new message key from their chain key, and
+use it to encrypt a message.
+
+#### 1. Initial key exchange flow (X3DH)
+
+[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message)
+describes the initial key exchange flow, with some additional context:
+
+- The peers' identity keys `IK_A` and `IK_B` correspond to their public keys;
+- Since it is not possible to guarantee that a prekey will be used only once
+in a decentralized world, the one-time prekey `OPK_B` is not used in this scenario;
+- Nodes SHOULD not send Bundles to a centralized server,
+but instead provide them in a decentralized way as described in the [Pre-keys section](#pre-keys).
+
+Alice retrieves Bob's prekey bundle, however it is not specific to Alice.
+It contains:
+
+([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12))
+
+**Wire format:**
+
+``` protobuf
+// X3DH prekey bundle
+message Bundle {
+  // Identity key 'IK_B'
+  bytes identity = 1;
+  // Signed prekey 'SPK_B' for each device, indexed by 'installation-id'
+  map<string,SignedPreKey> signed_pre_keys = 2;
+  // Prekey signature 'Sig(IK_B, Encode(SPK_B))'
+  bytes signature = 4;
+  // When the bundle was created locally
+  int64 timestamp = 5;
+}
+```
+
+([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L5))
+
+``` protobuf
+message SignedPreKey {
+  bytes signed_pre_key = 1;
+  uint32 version = 2;
+}
+```
+
+The `signature` is generated by sorting `installation-id` in lexicographical order,
+and concatenating the `signed-pre-key` and `version`:
+
+`installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2`
+
+#### 2. Double Ratchet
+
+Having established the initial shared secret `SK` through X3DH,
+it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
+
+Refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/)
+for more details.
+
+The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage`
+([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65))
+containing a map of `DirectMessageProtocol` indexed by `installation-id`
+([reference wire format](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
+
+``` protobuf
+message ProtocolMessage {
+  // The installation id of the sender
+  string installation_id = 2;
+  // A sequence of bundles
+  repeated Bundle bundles = 3;
+  // One to one message, encrypted, indexed by installation_id
+  map<string,DirectMessageProtocol> direct_message = 101;
+  // Public message, not encrypted
+  bytes public_message = 102;
+}
+```
+
+``` protobuf
+message EncryptedMessageProtocol {
+  X3DHHeader X3DH_header = 1;
+  DRHeader DR_header = 2; 
+  DHHeader DH_header = 101;
+  // Encrypted payload
+  // if a bundle is available, contains payload encrypted with the Double Ratchet algorithm;
+  // otherwise, payload encrypted with output key of DH exchange (no Perfect Forward Secrecy).
+  bytes payload = 3;
+}
+```
+
+Where:
+
+- `X3DH_header`: the `X3DHHeader` field in `DirectMessageProtocol` contains:
+
+    ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
+
+```protobuf
+    message X3DHHeader {
+      // Alice's ephemeral key `EK_A`
+      bytes key = 1;
+      // Bob's bundle signed prekey
+      bytes id = 4;
+    }
+```
+
+- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)).
+Used when Bob's public bundle is available:
+
+``` protobuf
+    message DRHeader {
+      // Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet))
+      bytes key = 1;
+      // number of the message in the sending chain
+      uint32 n = 2;
+      // length of the previous sending chain
+      uint32 pn = 3;
+      // Bob's bundle ID
+      bytes id = 4;
+    }
+```
+
+- `DH_header`: Diffie-Hellman header (used when Bob's bundle is not available):
+    ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
+
+``` protobuf
+    message DHHeader {
+      // Alice's compressed ephemeral public key.
+      bytes key = 1;
+    }
+```
+
+#### 3. Chain key update
+
+The chain key MUST be updated according to the `DR_Header`
+received in the `EncryptedMessageProtocol` message,
+described in [2.Double Ratchet](#2-double-ratchet).
+
+#### 4. Message key derivation
+
+The message key MUST be derived from a single ratchet step in the symmetric-key ratchet
+as described in [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
+
+The message key MUST be used to encrypt the next message to be sent.
+
+## Security Considerations
+
+1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations)
+and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
+
+2. Inherits the security considerations of the [Waku v2 protocol](../../core/10/waku2.md).
+
+3. The protocol is designed to be used in a decentralized manner, however,
+it is possible to use a centralized server to serve prekey bundles.
+In this case, the server is trusted.
+
+## Privacy Considerations
+
+1. This protocol does not provide message unlinkability.
+It is possible to link messages signed by the same keypair.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [X3DH](https://signal.org/docs/specifications/x3dh/)
+- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
+- [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
+- [Protobuf](https://developers.google.com/protocol-buffers/)
+- [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html)
+- [The Waku v2 protocol](../../core/10/waku2.md)
+- [HKDF](https://www.rfc-editor.org/rfc/rfc5869)
+- [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles)
+- [reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12)
+- [Symmetric key ratchet](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
+-

waku/standards/application/54/x3dh-sessions.md

@@ -1,164 +1,202 @@
----
-slug: 54
-title: 54/WAKU2-X3DH-SESSIONS
-name: Session management for Waku X3DH 
-status: draft
-category: Standards Track
-tags: waku-application
-editor: Aaryamann Challani <aaryamann@status.im>
-contributors:
-- Andrea Piana <andreap@status.im>
-- Pedro Pombeiro <pedro@status.im>
-- Corey Petty <corey@status.im>
-- Oskar Thorén <oskarth@titanproxy.com>
-- Dean Eigenmann <dean@status.im>
----
-
-## Abstract
-
-This document specifies how to manage sessions based on an X3DH key exchange.
-This includes how to establish new sessions, how to re-establish them, how to maintain them, and how to close them.
-
-[53/WAKU2-X3DH](../../standards/application/53/X3DH.md) specifies the Waku `X3DH` protocol for end-to-end encryption. 
-Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
-
-## Session Establishment
-
-A node identifies a peer by their `installation-id` which MAY be interpreted as a device identifier.
-
-### Discovery of pre-key bundles
-
-The node's pre-key bundle MUST be broadcast on a content topic derived from the node's public key, so that the first message may be PFS-encrypted.
-Each peer MUST publish their pre-key bundle periodically to this topic, otherwise they risk not being able to perform key-exchanges with other peers.
-Each peer MAY publish to this topic when their metadata changes, so that the other peer can update their local record.
-
-If peer A wants to send a message to peer B, it MUST derive the topic from peer B's public key, which has been shared out of band.
-Partitioned topics have been used to balance privacy and efficiency of broadcasting pre-key bundles.
-
-The number of partitions that MUST be used is 5000.
-
-The topic MUST be derived as follows:
-
-```
-var partitionsNum *big.Int = big.NewInt(5000)
-var partition *big.Int = big.NewInt(0).Mod(peerBPublicKey, partitionsNum)
-
-partitionTopic := "contact-discovery-" + strconv.FormatInt(partition.Int64(), 10)
-
-var hash []byte = keccak256(partitionTopic)
-var topicLen int = 4
-
-if len(hash) < topicLen {
-    topicLen = len(hash)
-}
-
-var contactCodeTopic [4]byte
-for i = 0; i < topicLen; i++ {
-    contactCodeTopic[i] = hash[i]
-}
-```
-
-### Initialization
-A node initializes a new session once a successful X3DH exchange has taken place. 
-Subsequent messages will use the established session until re-keying is necessary.
-
-### Negotiated topic to be used for the session
-
-After the peers have performed the initial key exchange, they MUST derive a topic from their shared secret to send messages on.
-To obtain this value, take the first four bytes of the keccak256 hash of the shared secret encoded in hexadecimal format.
-
-```
-sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
-      ecies.ImportECDSAPublic(theirPublicKey),
-      16,
-      16,
-)
-
-
-hexEncodedKey := hex.EncodeToString(sharedKey)
-
-var hash []byte = keccak256(hexEncodedKey)
-var topicLen int = 4
-
-if len(hash) < topicLen {
-    topicLen = len(hash)
-}
-
-var topic [4]byte
-for i = 0; i < topicLen; i++ {
-    topic[i] = hash[i]
-}
-```
-
-To summarize, following is the process for peer B to establish a session with peer A:
-1. Listen to peer B's Contact Code Topic to retrieve their bundle information, including a list of active devices
-2. Peer A sends their pre-key bundle on peer B's partitioned topic
-3. Peer A and peer B perform the key-exchange using the shared pre-key bundles
-3. The negotiated topic is derived from the shared secret
-4. Peers A & B exchange messages on the negotiated topic
-
-### Concurrent sessions
-
-If a node creates two sessions concurrently between two peers, the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
-
-### Re-keying
-
-On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
-
-### Multi-device support
-
-Multi-device support is quite challenging as there is not a central place where information on which and how many devices (identified by their respective `installation-id`) a peer has, is stored.
-
-Furthermore, account recovery always needs to be taken into consideration, where a user wipes clean the whole device and the node loses all the information about any previous sessions.
-Taking these considerations into account, the way the network propagates multi-device information using X3DH bundles, which will contain information about paired devices as well as information about the sending device.
-This means that every time a new device is paired, the bundle needs to be updated and propagated with the new information, the user has the responsibility to make sure the pairing is successful.
-
-The method is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/).
-
-### Pairing
-
-A new `installation-id` MUST be generated on a per-device basis. 
-The device should be paired as soon as possible if other devices are present. 
-
-If a bundle is received, which has the same `IK` as the keypair present on the device, the devices MAY be paired.
-Once a user enables a new device, a new bundle MUST be generated which includes pairing information.
-
-The bundle MUST be propagated to contacts through the usual channels.
-
-Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
-
-### Sending messages to a paired group
-
-When sending a message, the peer SHOULD send a message to other `installation-id` that they have seen. 
-The node caps the number of devices to `n`, ordered by last activity. 
-The node sends messages using pairwise encryption, including their own devices.
-
-Where `n` is the maximum number of devices that can be paired.
-
-### Account recovery
-
-Account recovery is the same as adding a new device, and it MUST be handled the same way.
-
-### Partitioned devices
-
-In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own `installation-id`.
-In this case an empty message containing bundle information MUST be sent back, which will notify the receiving end not to include the device in any further communication.
-
-## Security Considerations
-
-1. Inherits all security considerations from [53/WAKU2-X3DH](../../standards/application/53/X3DH.md).
-
-### Recommendations
-
-1. The value of `n` SHOULD be configured by the app-protocol.
-    - The default value SHOULD be 3, since a larger number of devices will result in a larger bundle size, which may not be desirable in a peer-to-peer network.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [53/WAKU2-X3DH](../../standards/application/53/X3DH.md)
-2. [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)
-
+---
+slug: 54
+title: 54/WAKU2-X3DH-SESSIONS
+name: Session management for Waku X3DH 
+status: draft
+category: Standards Track
+tags: waku-application
+editor: Aaryamann Challani <p1ge0nh8er@proton.me>
+contributors:
+- Andrea Piana <andreap@status.im>
+- Pedro Pombeiro <pedro@status.im>
+- Corey Petty <corey@status.im>
+- Oskar Thorén <oskarth@titanproxy.com>
+- Dean Eigenmann <dean@status.im>
+---
+
+## Abstract
+
+This document specifies how to manage sessions based on an X3DH key exchange.
+This includes how to establish new sessions,
+how to re-establish them, how to maintain them, and how to close them.
+
+[53/WAKU2-X3DH](../53/x3dh.md) specifies the Waku `X3DH` protocol
+for end-to-end encryption.
+Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
+
+## Session Establishment
+
+A node identifies a peer by their `installation-id`
+which MAY be interpreted as a device identifier.
+
+### Discovery of pre-key bundles
+
+The node's pre-key bundle MUST be broadcast on a content topic
+derived from the node's public key, so that the first message may be PFS-encrypted.
+Each peer MUST publish their pre-key bundle periodically to this topic,
+otherwise they risk not being able to perform key-exchanges with other peers.
+Each peer MAY publish to this topic when their metadata changes,
+so that the other peer can update their local record.
+
+If peer A wants to send a message to peer B,
+it MUST derive the topic from peer B's public key, which has been shared out of band.
+Partitioned topics have been used to balance privacy and
+efficiency of broadcasting pre-key bundles.
+
+The number of partitions that MUST be used is 5000.
+
+The topic MUST be derived as follows:
+
+```js
+var partitionsNum *big.Int = big.NewInt(5000)
+var partition *big.Int = big.NewInt(0).Mod(peerBPublicKey, partitionsNum)
+
+partitionTopic := "contact-discovery-" + strconv.FormatInt(partition.Int64(), 10)
+
+var hash []byte = keccak256(partitionTopic)
+var topicLen int = 4
+
+if len(hash) < topicLen {
+    topicLen = len(hash)
+}
+
+var contactCodeTopic [4]byte
+for i = 0; i < topicLen; i++ {
+    contactCodeTopic[i] = hash[i]
+}
+```
+
+### Initialization
+
+A node initializes a new session once a successful X3DH exchange has taken place.
+Subsequent messages will use the established session until re-keying is necessary.
+
+### Negotiated topic to be used for the session
+
+After the peers have performed the initial key exchange,
+they MUST derive a topic from their shared secret to send messages on.
+To obtain this value, take the first four bytes of the keccak256 hash
+of the shared secret encoded in hexadecimal format.
+
+```js
+sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
+      ecies.ImportECDSAPublic(theirPublicKey),
+      16,
+      16,
+)
+
+
+hexEncodedKey := hex.EncodeToString(sharedKey)
+
+var hash []byte = keccak256(hexEncodedKey)
+var topicLen int = 4
+
+if len(hash) < topicLen {
+    topicLen = len(hash)
+}
+
+var topic [4]byte
+for i = 0; i < topicLen; i++ {
+    topic[i] = hash[i]
+}
+```
+
+To summarize,
+following is the process for peer B to establish a session with peer A:
+
+1. Listen to peer B's Contact Code Topic to retrieve their bundle information,
+including a list of active devices
+2. Peer A sends their pre-key bundle on peer B's partitioned topic
+3. Peer A and peer B perform the key-exchange using the shared pre-key bundles
+4. The negotiated topic is derived from the shared secret
+5. Peers A & B exchange messages on the negotiated topic
+
+### Concurrent sessions
+
+If a node creates two sessions concurrently between two peers,
+the one with the symmetric key first in byte order SHOULD be used,
+this marks that the other has expired.
+
+### Re-keying
+
+On receiving a bundle from a given peer with a higher version,
+the old bundle SHOULD be marked as expired and
+a new session SHOULD be established on the next message sent.
+
+### Multi-device support
+
+Multi-device support is quite challenging
+as there is not a central place where information on which and how many devices
+(identified by their respective `installation-id`) a peer has, is stored.
+
+Furthermore, account recovery always needs to be taken into consideration,
+where a user wipes clean the whole device and
+the node loses all the information about any previous sessions.
+Taking these considerations into account,
+the way the network propagates multi-device information using X3DH bundles,
+which will contain information about paired devices
+as well as information about the sending device.
+This means that every time a new device is paired,
+the bundle needs to be updated and propagated with the new information,
+the user has the responsibility to make sure the pairing is successful.
+
+The method is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/).
+
+### Pairing
+
+A new `installation-id` MUST be generated on a per-device basis.
+The device should be paired as soon as possible if other devices are present.
+
+If a bundle is received, which has the same `IK` as the keypair present on the device,
+the devices MAY be paired.
+Once a user enables a new device,
+a new bundle MUST be generated which includes pairing information.
+
+The bundle MUST be propagated to contacts through the usual channels.
+
+Removal of paired devices is a manual step that needs to be applied on each device,
+and consist simply in disabling the device,
+at which point pairing information will not be propagated anymore.
+
+### Sending messages to a paired group
+
+When sending a message,
+the peer SHOULD send a message to other `installation-id` that they have seen.
+The node caps the number of devices to `n`, ordered by last activity.
+The node sends messages using pairwise encryption, including their own devices.
+
+Where `n` is the maximum number of devices that can be paired.
+
+### Account recovery
+
+Account recovery is the same as adding a new device,
+and it MUST be handled the same way.
+
+### Partitioned devices
+
+In some cases
+(i.e. account recovery when no other pairing device is available, device not paired),
+it is possible that a device will receive a message
+that is not targeted to its own `installation-id`.
+In this case an empty message containing bundle information MUST be sent back,
+which will notify the receiving end not to include the device in any further communication.
+
+## Security Considerations
+
+1. Inherits all security considerations from [53/WAKU2-X3DH](../53/x3dh.md).
+
+### Recommendations
+
+1. The value of `n` SHOULD be configured by the app-protocol.
+    - The default value SHOULD be 3,
+    since a larger number of devices will result in a larger bundle size,
+    which may not be desirable in a peer-to-peer network.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [53/WAKU2-X3DH](../53/x3dh.md)
+2. [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)

waku/standards/core/10/images/overview.msc

@@ -1,33 +1,33 @@
-# Sequence diagram for Waku v2 (WakuMessage, WakuData, Relay, Store, Filter)
-# PNG generated with https://mscgen.js.org
-msc {
-  hscale="1",
-  wordwraparcs=true;
-
-  a [label="A\nrelay\n(0)"],
-  b [label="B relay(pubtopic1)\n(0)"],
-  c [label="C relay(pubtopic2)\n(0)"],
-  d [label="D relay(pubtopic1), store(pubtopic1), filter\n(0)"],
-  e [label="E\nrelay, store\n(0)"],
-  f [label="F\nrelay, filter\n(0)"];
-
-  a rbox a [label="msg1=WakuMessage(contentTopic1, data) [14/WAKU2-MESSAGE] (1)"];
-  a note a [label="If version=1, encrypt data per [7/WAKU-DATA] (1)"];
-
-  f => d [label="FilterRequest(pubtopic1, contentTopic1) [12/WAKU2-FILTER] (2)"];
-  d rbox d [label="Subscribe F to filter [12/WAKU2-FILTER] (2)"];
-
-  a => b [label="Publish msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
-  b => d [label="relay msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
-
-  d rbox d [label="store: saves msg1 [13/WAKU2-STORE] (4)"];
-
-  d => f [label="MessagePush(msg1)[12/WAKU2-FILTER] (5)"];
-
-  ---;
-
-  e note e [label="E comes online (6)"];
-  e => d [label="HistoryQuery(pubtopic1, contentTopic1) [13/WAKU2-STORE] (6)"];
-  d => e [label="HistoryResponse(msg1, ...) [13/WAKU2-STORE] (6)"];
-
-}
+# Sequence diagram for Waku v2 (WakuMessage, WakuData, Relay, Store, Filter)
+# PNG generated with https://mscgen.js.org
+msc {
+  hscale="1",
+  wordwraparcs=true;
+
+  a [label="A\nrelay\n(0)"],
+  b [label="B relay(pubtopic1)\n(0)"],
+  c [label="C relay(pubtopic2)\n(0)"],
+  d [label="D relay(pubtopic1), store(pubtopic1), filter\n(0)"],
+  e [label="E\nrelay, store\n(0)"],
+  f [label="F\nrelay, filter\n(0)"];
+
+  a rbox a [label="msg1=WakuMessage(contentTopic1, data) [14/WAKU2-MESSAGE] (1)"];
+  a note a [label="If version=1, encrypt data per [7/WAKU-DATA] (1)"];
+
+  f => d [label="FilterRequest(pubtopic1, contentTopic1) [12/WAKU2-FILTER] (2)"];
+  d rbox d [label="Subscribe F to filter [12/WAKU2-FILTER] (2)"];
+
+  a => b [label="Publish msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
+  b => d [label="relay msg1 on pubtopic1 [11/WAKU2-RELAY] (3)"];
+
+  d rbox d [label="store: saves msg1 [13/WAKU2-STORE] (4)"];
+
+  d => f [label="MessagePush(msg1)[12/WAKU2-FILTER] (5)"];
+
+  ---;
+
+  e note e [label="E comes online (6)"];
+  e => d [label="HistoryQuery(pubtopic1, contentTopic1) [13/WAKU2-STORE] (6)"];
+  d => e [label="HistoryResponse(msg1, ...) [13/WAKU2-STORE] (6)"];
+
+}

waku/standards/core/11/relay.md

@@ -1,197 +1,284 @@
----
-slug: 11
-title: 11/WAKU2-RELAY
-name: Waku v2 Relay
-status: stable
-tags: waku-core
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
-  - Oskar Thorén <oskarth@titanproxy.com>
-  - Sanaz Taheri <sanaz@status.im>
----
-
-`11/WAKU2-RELAY` specifies a [Publish/Subscribe approach](https://docs.libp2p.io/concepts/publish-subscribe/) to peer-to-peer messaging with a strong focus on privacy, censorship-resistance, security and scalability.
-Its current implementation is a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) and prescribes gossip-based dissemination.
-As such the scope is limited to defining a separate [`protocol id`](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation) for `11/WAKU2-RELAY`, establishing privacy and security requirements, and defining how the underlying GossipSub is to be interpreted and implemented within the Waku and cryptoeconomic domain.
-`11/WAKU2-RELAY` should not be confused with [libp2p circuit relay](https://github.com/libp2p/specs/tree/master/relay).
-
-**Protocol identifier**: `/vac/waku/relay/2.0.0`
-
-## Security Requirements
-
-The `11/WAKU2-RELAY` protocol is designed to provide the following security properties under a static [Adversarial Model](#adversarial-model).
-Note that data confidentiality, integrity, and authenticity are currently considered out of scope for `11/WAKU2-RELAY` and must be handled by higher layer protocols such as [`14/WAKU2-MESSAGE`](../14/message.md).
-
-<!-- May add the definition of the unsupported feature: 
-Confidentiality indicates that an adversary should not be able to learn the data carried by the `WakuRelay` protocol.
-Integrity indicates that the data transferred by the `WakuRelay` protocol can not be tampered with by an adversarial entity without being detected.
-Authenticity no adversary can forge data on behalf of a targeted publisher and make it accepted by other subscribers as if the origin is the target. -->
-
-- **Publisher-Message Unlinkability**:
-This property indicates that no adversarial entity can link a published `Message` to its publisher.
-This feature also implies the unlinkability of the publisher to its published topic ID as the `Message` embodies the topic IDs.
-
-- **Subscriber-Topic Unlinkability**:
-This feature stands for the inability of any adversarial entity from linking a subscriber to its subscribed topic IDs.
-
-<!-- TODO: more requirements can be added, but that needs further and deeper investigation-->
-
-### Terminology
-
-_Personally identifiable information_ (PII) refers to any piece of data that can be used to uniquely identify a user.
-For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
-
-## Adversarial Model
-
--  Any entity running the `11/WAKU2-RELAY` protocol is considered an adversary.
-This includes publishers, subscribers, and all the peers' direct connections. 
-Furthermore, we consider the adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions.
-For example, under the passive adversarial model, no malicious subscriber hides the messages it receives from other subscribers as it is against the description of `11/WAKU2-RELAY`.
-However, a malicious subscriber may learn which topics are subscribed to by which peers. 
-- The following are **not** considered as part of the adversarial model: 
-  -  An adversary with a global view of all the peers and their connections. 
-  -  An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
-  In other words, the communication channels are assumed to be secure.
-
-## Wire Specification
-
-The [PubSub interface specification](https://github.com/libp2p/specs/blob/master/pubsub/README.md) defines the protobuf RPC messages exchanged between peers participating in a GossipSub network.
-We republish these messages here for ease of reference and define how `11/WAKU2-RELAY` uses and interprets each field.
-
-### Protobuf definitions
-
-The PubSub RPC messages are specified using [protocol buffers v2](https://developers.google.com/protocol-buffers/)
-
-```protobuf
-syntax = "proto2";
-
-message RPC {
-  repeated SubOpts subscriptions = 1;
-  repeated Message publish = 2;
-
-  message SubOpts {
-    optional bool subscribe = 1;
-    optional string topicid = 2;
-  }
-
-  message Message {
-    optional string from = 1;
-    optional bytes data = 2;
-    optional bytes seqno = 3;
-    repeated string topicIDs = 4;
-    optional bytes signature = 5;
-    optional bytes key = 6;
-  }
-}
-```
-
-> **_NOTE:_**
-The various [control messages](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#control-messages) defined for GossipSub are used as specified there.
-
-> **_NOTE:_**
-The [`TopicDescriptor`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) is not currently used by `11/WAKU2-RELAY`.
-
-### Message fields
-
-The `Message` protobuf defines the format in which content is relayed between peers.
-`11/WAKU2-RELAY` specifies the following usage requirements for each field:
-
-- The `from` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
-
-- The `data` field MUST be filled out with a `WakuMessage`.
-See [`14/WAKU2-MESSAGE`](../14/message.md) for more details.
-
-- The `seqno` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
-
-- The `topicIDs` field MUST contain the content-topics that a message is being published on.
-
-- The `signature` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
-
-- The `key` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-solicy).
-
-### SubOpts fields
-
-The `SubOpts` protobuf defines the format in which subscription options are relayed between peers.
-A `11/WAKU2-RELAY` node MAY decide to subscribe or unsubscribe from topics by sending updates using `SubOpts`.
-The following usage requirements apply:
-
-- The `subscribe` field MUST contain a boolean, where `true` indicates subscribe and `false` indicates unsubscribe to a topic.
-
-- The `topicid` field MUST contain the pubsub topic.
-
-> Note: The `topicid` refering to pubsub topic and
-`topicId` refering to content-topic are detailed in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
-
-### Signature Policy
-
-The [`StrictNoSign` option](https://github.com/libp2p/specs/blob/master/pubsub/README.md#signature-policy-options) MUST be used, to ensure that messages are built without the `signature`, `key`, `from` and `seqno` fields.
-Note that this does not merely imply that these fields be empty, but that they MUST be _absent_ from the marshalled message.
-
-## Security Analysis
-
-<!-- TODO: realized that the prime security objective of the `WakuRelay` protocol is to provide peers unlinkability as such this feature is prioritized over other features e.g., unlinkability is preferred over authenticity and integrity. It might be good to motivate unlinkability and its impact on the relay protocol or other protocols invoking relay protocol.-->
-
-- **Publisher-Message Unlinkability**:
-To address publisher-message unlinkability, one should remove any PII from the published message.
-As such, `11/WAKU2-RELAY` follows the `StrictNoSign` policy as described in [libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing).
-As the result of the `StrictNoSign` policy, `Message`s should be built without the  `from`, `signature` and `key` fields since each of these three fields individually counts as PII for the author of the message (one can link the creation of the message with libp2p peerId and thus indirectly with the IP address of the publisher). 
-Note that removing identifiable information from messages cannot lead to perfect unlinkability.
-The direct connections of a publisher might be able to figure out which `Message`s belong to that publisher by analyzing its traffic.
-The possibility of such inference may get higher when the `data` field is also not encrypted by the upper-level protocols. <!-- TODO: more investigation on traffic analysis attacks and their success probability-->
-
-- **Subscriber-Topic Unlinkability:**
-To preserve subscriber-topic unlinkability, it is recommended by [`10/WAKU2`](../10/waku2.md) to use a single PubSub topic in the `11/WAKU2-RELAY` protocol.
-This allows an immediate subscriber-topic unlinkability where subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID.
-This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of participants of Waku relay protocol).
-However, note that `11/WAKU2-RELAY` supports the use of more than one topic.
-In case that more than one topic id is utilized, preserving unlinkability is the responsibility of the upper-level protocols which MAY adopt [partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic) to achieve K-anonymity for the subscribed peers.
-
-## Future work
-
-- **Economic spam resistance**:
-In the spam-protected `11/WAKU2-RELAY` protocol, no adversary can flood the system with spam messages (i.e., publishing a large number of messages in a short amount of time).
-Spam protection is partly provided by GossipSub v1.1 through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures).
-At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
-`11/WAKU2-RELAY` aims at enabling an advanced spam protection mechanism with economic disincentives by utilizing Rate Limiting Nullifiers.
-In a nutshell, peers must conform to a certain message publishing rate per a system-defined epoch, otherwise, they get financially penalized for exceeding the rate.
-More details on this new technique can be found in [`17/WAKU2-RLN-RELAY`](../17/rln-relay.md). 
-  <!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1 are taken in the nim-libp2p as well, may need to audit the code --> 
-
-- Providing **Unlinkability**, **Integrity** and  **Authenticity** simultaneously:
-Integrity and authenticity are typically addressed through digital signatures and Message Authentication Code (MAC) schemes, however, the usage of digital signatures (where each signature is bound to a particular peer) contradicts with the unlinkability requirement (messages signed under a certain signature key are verifiable by a verification key that is bound to a particular publisher).
-As such, integrity and authenticity are missing features in `11/WAKU2-RELAY` in the interest of unlinkability.
-In future work, advanced signature schemes like group signatures can be utilized to enable authenticity, integrity, and unlinkability simultaneously.
-In a group signature scheme, a member of a group can anonymously sign a message on behalf of the group as such the true signer is indistinguishable from other group members. <!-- TODO: shall I add a reference for group signatures?-->
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [`10/WAKU2`](../10/waku2.md)
-
-1. [`14/WAKU2-MESSAGE`](../14/message.md)
-
-1. [`17/WAKU-RLN`](../17/rln-relay.md)
-
-1. [GossipSub v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
-
-1. [GossipSub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
-
-1. [K-anonimity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
-
-1. [`libp2p` concepts: Publish/Subscribe](https://docs.libp2p.io/concepts/publish-subscribe/)
-
-1. [`libp2p` protocol negotiation](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation)
-
-1. [Partitioned topics](https://specs.status.im/spec/10#partitioned-topic)
-
-1. [Protocol Buffers](https://developers.google.com/protocol-buffers/)
-
-1. [PubSub interface for libp2p (r2, 2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
-
-1. [Waku v1 spec](../6/waku1.md)
-
-1. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)
+---
+slug: 11
+title: 11/WAKU2-RELAY
+name: Waku v2 Relay
+status: stable
+tags: waku-core
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+  - Oskar Thorén <oskarth@titanproxy.com>
+  - Sanaz Taheri <sanaz@status.im>
+---
+
+`11/WAKU2-RELAY` specifies a [Publish/Subscribe approach](https://docs.libp2p.io/concepts/publish-subscribe/)
+to peer-to-peer messaging with a strong focus on privacy,
+censorship-resistance, security and scalability.
+Its current implementation is a minor extension of the
+[libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+and prescribes gossip-based dissemination.
+As such the scope is limited to defining a separate
+[`protocol id`](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation)
+for `11/WAKU2-RELAY`, establishing privacy and security requirements,
+and defining how the underlying GossipSub is to be interpreted and
+implemented within the Waku and cryptoeconomic domain.
+`11/WAKU2-RELAY` should not be confused with [libp2p circuit relay](https://github.com/libp2p/specs/tree/master/relay).
+
+**Protocol identifier**: `/vac/waku/relay/2.0.0`
+
+## Security Requirements
+
+The `11/WAKU2-RELAY` protocol is designed to provide the following security properties
+under a static [Adversarial Model](#adversarial-model).
+Note that data confidentiality, integrity, and
+authenticity are currently considered out of scope for `11/WAKU2-RELAY` and
+must be handled by higher layer protocols such as [`14/WAKU2-MESSAGE`](../14/message.md).
+
+<!-- May add the definition of the unsupported feature: 
+Confidentiality indicates that an adversary
+should not be able to learn the data carried by the `WakuRelay` protocol.
+Integrity indicates that the data transferred by the `WakuRelay` protocol
+can not be tampered with by an adversarial entity without being detected.
+Authenticity no adversary can forge data on behalf of a targeted publisher and
+make it accepted by other subscribers as if the origin is the target. -->
+
+- **Publisher-Message Unlinkability**:
+This property indicates that no adversarial entity can link a published `Message`
+to its publisher.
+This feature also implies the unlinkability of the publisher
+to its published topic ID as the `Message` embodies the topic IDs.
+
+- **Subscriber-Topic Unlinkability**:
+This feature stands for the inability of any adversarial entity
+from linking a subscriber to its subscribed topic IDs.
+
+<!-- TODO: more requirements can be added,
+but that needs further and deeper investigation-->
+
+### Terminology
+
+_Personally identifiable information_ (PII)
+refers to any piece of data that can be used to uniquely identify a user.
+For example, the signature verification key,
+and the hash of one's static IP address are unique for each user and
+hence count as PII.
+
+## Adversarial Model
+
+- Any entity running the `11/WAKU2-RELAY` protocol is considered an adversary.
+This includes publishers, subscribers, and all the peers' direct connections.
+Furthermore,
+we consider the adversary as a passive entity that attempts to collect information
+from others to conduct an attack but
+it does so without violating protocol definitions and instructions.
+For example, under the passive adversarial model,
+no malicious subscriber hides the messages it receives from other subscribers
+as it is against the description of `11/WAKU2-RELAY`.
+However,
+a malicious subscriber may learn which topics are subscribed to by which peers.
+- The following are **not** considered as part of the adversarial model:
+  - An adversary with a global view of all the peers and their connections.
+  - An adversary that can eavesdrop on communication links between arbitrary pairs
+  of peers (unless the adversary is one end of the communication).
+  In other words, the communication channels are assumed to be secure.
+
+## Wire Specification
+
+The [PubSub interface specification](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
+defines the protobuf RPC messages
+exchanged between peers participating in a GossipSub network.
+We republish these messages here for ease of reference and
+define how `11/WAKU2-RELAY` uses and interprets each field.
+
+### Protobuf definitions
+
+The PubSub RPC messages are specified using [protocol buffers v2](https://developers.google.com/protocol-buffers/)
+
+```protobuf
+syntax = "proto2";
+
+message RPC {
+  repeated SubOpts subscriptions = 1;
+  repeated Message publish = 2;
+
+  message SubOpts {
+    optional bool subscribe = 1;
+    optional string topicid = 2;
+  }
+
+  message Message {
+    optional string from = 1;
+    optional bytes data = 2;
+    optional bytes seqno = 3;
+    repeated string topicIDs = 4;
+    optional bytes signature = 5;
+    optional bytes key = 6;
+  }
+}
+```
+
+> **_NOTE:_**
+The various [control messages](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#control-messages)
+defined for GossipSub are used as specified there.
+> **_NOTE:_**
+The [`TopicDescriptor`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
+is not currently used by `11/WAKU2-RELAY`.
+
+### Message fields
+
+The `Message` protobuf defines the format in which content is relayed between peers.
+`11/WAKU2-RELAY` specifies the following usage requirements for each field:
+
+- The `from` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
+
+- The `data` field MUST be filled out with a `WakuMessage`.
+See [`14/WAKU2-MESSAGE`](../14/message.md) for more details.
+
+- The `seqno` field MUST NOT be used, following the [`StrictNoSign` signature policy](#signature-policy).
+
+- The `topicIDs` field MUST contain the content-topics
+that a message is being published on.
+
+- The `signature` field MUST NOT be used,
+following the [`StrictNoSign` signature policy](#signature-policy).
+
+- The `key` field MUST NOT be used,
+following the [`StrictNoSign` signature policy](#signature-policy).
+
+### SubOpts fields
+
+The `SubOpts` protobuf defines the format
+in which subscription options are relayed between peers.
+A `11/WAKU2-RELAY` node MAY decide to subscribe or
+unsubscribe from topics by sending updates using `SubOpts`.
+The following usage requirements apply:
+
+- The `subscribe` field MUST contain a boolean,
+where `true` indicates subscribe and `false` indicates unsubscribe to a topic.
+
+- The `topicid` field MUST contain the pubsub topic.
+
+> Note: The `topicid` refering to pubsub topic and
+`topicId` refering to content-topic are detailed in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
+
+### Signature Policy
+
+The [`StrictNoSign` option](https://github.com/libp2p/specs/blob/master/pubsub/README.md#signature-policy-options)
+MUST be used, to ensure that messages are built without the `signature`,
+`key`, `from` and `seqno` fields.
+Note that this does not merely imply that these fields be empty, but
+that they MUST be _absent_ from the marshalled message.
+
+## Security Analysis
+
+<!-- TODO: realized that the prime security objective of the `WakuRelay` 
+protocol is to provide peers unlinkability
+as such this feature is prioritized over other features
+e.g., unlinkability is preferred over authenticity and integrity.
+It might be good to motivate unlinkability and
+its impact on the relay protocol or other protocols invoking relay protocol.-->
+
+- **Publisher-Message Unlinkability**:
+To address publisher-message unlinkability,
+one should remove any PII from the published message.
+As such, `11/WAKU2-RELAY` follows the `StrictNoSign` policy as described in
+[libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing).
+As the result of the `StrictNoSign` policy,
+`Message`s should be built without the  `from`,
+`signature` and `key` fields since each of these three fields individually
+counts as PII for the author of the message
+(one can link the creation of the message with libp2p peerId and
+thus indirectly with the IP address of the publisher).
+Note that removing identifiable information from messages
+cannot lead to perfect unlinkability.
+The direct connections of a publisher
+might be able to figure out which `Message`s belong to that publisher
+by analyzing its traffic.
+The possibility of such inference may get higher
+when the `data` field is also not encrypted by the upper-level protocols.
+<!-- TODO: more investigation on traffic analysis attacks and their success probability-->
+
+- **Subscriber-Topic Unlinkability:**
+To preserve subscriber-topic unlinkability,
+it is recommended by [`10/WAKU2`](../10/waku2.md) to use a single PubSub topic
+in the `11/WAKU2-RELAY` protocol.
+This allows an immediate subscriber-topic unlinkability
+where subscribers are not re-identifiable from their subscribed topic IDs
+as the entire network is linked to the same topic ID.
+This level of unlinkability / anonymity
+is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
+where k is proportional to the system size
+(number of participants of Waku relay protocol).
+However, note that `11/WAKU2-RELAY` supports the use of more than one topic.
+In case that more than one topic id is utilized,
+preserving unlinkability is the responsibility of the upper-level protocols
+which MAY adopt
+[partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic)
+to achieve K-anonymity for the subscribed peers.
+
+## Future work
+
+- **Economic spam resistance**:
+In the spam-protected `11/WAKU2-RELAY` protocol,
+no adversary can flood the system with spam messages
+(i.e., publishing a large number of messages in a short amount of time).
+Spam protection is partly provided by GossipSub v1.1 through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures).
+At a high level,
+peers utilize a scoring function to locally score the behavior of their connections
+and remove peers with a low score.
+`11/WAKU2-RELAY` aims at enabling an advanced spam protection mechanism
+with economic disincentives by utilizing Rate Limiting Nullifiers.
+In a nutshell,
+peers must conform to a certain message publishing rate per a system-defined epoch,
+otherwise, they get financially penalized for exceeding the rate.
+More details on this new technique can be found in [`17/WAKU2-RLN-RELAY`](../17/rln-relay.md).
+<!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1
+are taken in the nim-libp2p as well, may need to audit the code -->
+
+- Providing **Unlinkability**, **Integrity** and  **Authenticity** simultaneously:
+Integrity and authenticity are typically addressed through digital signatures and
+Message Authentication Code (MAC) schemes, however,
+the usage of digital signatures (where each signature is bound to a particular peer)
+contradicts with the unlinkability requirement
+(messages signed under a certain signature key are verifiable by a verification key
+that is bound to a particular publisher).
+As such, integrity and authenticity are missing features in `11/WAKU2-RELAY`
+in the interest of unlinkability.
+In future work, advanced signature schemes like group signatures
+can be utilized to enable authenticity, integrity, and unlinkability simultaneously.
+In a group signature scheme, a member of a group can anonymously sign a message
+on behalf of the group as such the true signer
+is indistinguishable from other group members.
+<!-- TODO: shall I add a reference for group signatures?-->
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [`10/WAKU2`](../10/waku2.md)
+
+1. [`14/WAKU2-MESSAGE`](../14/message.md)
+
+1. [`17/WAKU-RLN`](../17/rln-relay.md)
+
+1. [GossipSub v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
+
+1. [GossipSub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
+
+1. [K-anonimity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
+
+1. [`libp2p` concepts: Publish/Subscribe](https://docs.libp2p.io/concepts/publish-subscribe/)
+
+1. [`libp2p` protocol negotiation](https://github.com/libp2p/specs/blob/master/connections/README.md#protocol-negotiation)
+
+1. [Partitioned topics](https://specs.status.im/spec/10#partitioned-topic)
+
+1. [Protocol Buffers](https://developers.google.com/protocol-buffers/)
+
+1. [PubSub interface for libp2p (r2, 2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
+
+1. [Waku v1 spec](../6/waku1.md)
+
+1. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)

waku/standards/core/12/filter.md

@@ -1,267 +1,356 @@
----
-slug: 12
-title: 12/WAKU2-FILTER
-name: Waku v2 Filter
-status: draft
-tags: waku-core
-version: 01
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskar@status.im>
-  - Sanaz Taheri <sanaz@status.im>
-  - Ebube Ud <ebube@status.im>
----
-
-previous versions: [00](./previous-versions00)
-
----
-
-`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
-
-## Content filtering
-
-**Protocol identifiers**:
-- _filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
-- _filter-push_: `/vac/waku/filter-push/2.0.0-beta1`
-
-Content filtering is a way to do [message-based
-filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
-Currently the only content filter being applied is on `contentTopic`. This
-corresponds to topics in Waku v1.
-
-## Rationale
-
-Unlike the `store` protocol for historical messages, this protocol allows for
-native lower latency scenarios such as instant messaging. It is thus
-complementary to it.
-
-Strictly speaking, it is not just doing basic request response, but performs
-sender push based on receiver intent. While this can be seen as a form of light
-pub/sub, it is only used between two nodes in a direct fashion. Unlike the
-Gossip domain, this is meant for light nodes which put a premium on bandwidth.
-No gossiping takes place.
-
-It is worth noting that a light node could get by with only using the `store`
-protocol to query for a recent time window, provided it is acceptable to do
-frequent polling.
-
-
-## Design Requirements
-
-The effectiveness and reliability of the content filtering service enabled by  `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes). 
-
-## Security Consideration
-
-Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section. 
-
-### Terminology
-The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
-
-## Adversarial Model
-Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol. 
-
-The following are not considered as part of the adversarial model: 
-  - An adversary with a global view of all the nodes and their connections. 
-  - An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
-
-### Protobuf
-
-```protobuf
-syntax = "proto3";
-
-// 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/
-package waku.filter.v2;
-
-// Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1
-message FilterSubscribeRequest {
-  enum FilterSubscribeType {
-    SUBSCRIBER_PING = 0;
-    SUBSCRIBE = 1;
-    UNSUBSCRIBE = 2;
-    UNSUBSCRIBE_ALL = 3;
-  }
-
-  string request_id = 1;
-  FilterSubscribeType filter_subscribe_type = 2;
-
-  // Filter criteria
-  optional string pubsub_topic = 10;
-  repeated string content_topics = 11;
-}
-
-message FilterSubscribeResponse {
-  string request_id = 1;
-  uint32 status_code = 10;
-  optional string status_desc = 11;
-}
-
-// Protocol identifier: /vac/waku/filter-push/2.0.0-beta1
-message MessagePush {
-  WakuMessage waku_message = 1;
-  optional string pubsub_topic = 2;
-}
-```
-
-### Filter-Subscribe
-
-A filter service node MUST support the _filter-subscribe_ protocol
-to allow filter clients to subscribe, modify, refresh and unsubscribe a desired set of filter criteria.
-The combination of different filter criteria for a specific filter client node is termed a "subscription".
-A filter client is interested in receiving messages matching the filter criteria in its registered subscriptions.
-
-Since a filter service node is consuming resources to provide this service,
-it MAY account for usage and adapt its service provision to certain clients.
-An incentive mechanism is currently planned but underspecified.
-
-#### Filter Subscribe Request
-
-A client node MUST send all filter requests in a `FilterSubscribeRequest` message.
-This request MUST contain a `request_id`.
-The `request_id` MUST be a uniquely generated string.
-Each request MUST include a `filter_subscribe_type`, indicating the type of request.
-
-#### Filter Subscribe Response
-
-In return to any `FilterSubscribeRequest`,
-a filter service node SHOULD respond with a `FilterSubscribeResponse` with a `requestId` matching that of the request.
-This response MUST contain a `status_code` indicating if the request was successful or not.
-Successful status codes are in the `2xx` range.
-Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
-In addition, the filter service node MAY choose to provide a more detailed status description in the `status_desc` field.
-
-#### Filter matching
-
-In the description of each request type below,
-the term "filter criteria" refers to the combination of `pubsub_topic` and a set of `content_topics`.
-The request MAY include filter criteria, conditional to the selected `filter_subscribe_type`.
-If the request contains filter criteria,
-it MUST contain a `pubsub_topic`
-and the `content_topics` set MUST NOT be empty.
-A `WakuMessage` matches filter criteria when its `content_topic` is in the `content_topics` set
-and it was published on a matching `pubsub_topic`.
-
-#### Filter Subscribe Types
-
-The following filter subscribe types are defined:
-
-##### SUBSCRIBER_PING
-
-A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBER_PING`
-requests that the service node SHOULD indicate if it has any active subscriptions for this client.
-The filter client SHOULD exclude any filter criteria from the request.
-The filter service node SHOULD respond with a success code if it has any active subscriptions for this client
-or an error code if not.
-The filter service node SHOULD ignore any filter criteria in the request.
-
-##### SUBSCRIBE
-
-A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBE`
-requests that the service node SHOULD push messages matching this filter to the client.
-The filter client MUST include the desired filter criteria in the request.
-A client MAY use this request type to _modify_ an existing subscription
-by providing _additional_ filter criteria in a new request.
-A client MAY use this request type to _refresh_ an existing subscription
-by providing _the same_ filter criteria in a new request.
-The filter service node SHOULD respond with a success code if it successfully honored this request
-or an error code if not.
-The filter service node SHOULD respond with an error code and discard the request
-if the subscribe request does not contain valid filter criteria,
-i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
-
-##### UNSUBSCRIBE
-
-A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE`
-requests that the service node SHOULD _stop_ pushing messages matching this filter to the client.
-The filter client MUST include the filter criteria it desires to unsubscribe from in the request.
-A client MAY use this request type to _modify_ an existing subscription
-by providing _a subset of_ the original filter criteria to unsubscribe from in a new request.
-The filter service node SHOULD respond with a success code if it successfully honored this request
-or an error code if not.
-The filter service node SHOULD respond with an error code and discard the request
-if the unsubscribe request does not contain valid filter criteria,
-i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
-
-##### UNSUBSCRIBE_ALL
-
-A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
-requests that the service node SHOULD _stop_ pushing messages matching _any_ filter to the client.
-The filter client SHOULD exclude any filter criteria from the request.
-The filter service node SHOULD remove any existing subscriptions for this client.
-It SHOULD respond with a success code if it successfully honored this request
-or an error code if not.
-
-### Filter-Push
-
-A filter client node MUST support the _filter-push_ protocol
-to allow filter service nodes to push messages matching registered subscriptions to this client.
-
-A filter service node SHOULD push all messages
-matching the filter criteria in a registered subscription
-to the subscribed filter client.
-These [`WakuMessage`s](../14/message.md) are likely to come from [`11/WAKU2-RELAY`](../11/relay.md),
-but there MAY be other sources or protocols where this comes from.
-This is up to the consumer of the protocol.
-
-If a message push fails,
-the filter service node MAY consider the client node to be unreachable.
-If a specific filter client node is not reachable from the service node for a period of time,
-the filter service node MAY choose to stop pushing messages to the client and remove its subscription.
-This period is up to the service node implementation.
-We consider `1 minute` to be a reasonable default.
-
-#### Message Push
-
-Each message MUST be pushed in a `MessagePush` message.
-Each `MessagePush` MUST contain one (and only one) `waku_message`.
-If this message was received on a specific `pubsub_topic`,
-it SHOULD be included in the `MessagePush`.
-A filter client SHOULD NOT respond to a `MessagePush`.
-Since the filter protocol does not include caching or fault-tolerance,
-this is a best effort push service with no bundling
-or guaranteed retransmission of messages.
-A filter client SHOULD verify that each `MessagePush` it receives
-originated from a service node where the client has an active subscription
-and that it matches filter criteria belonging to that subscription.
-
---- 
-## Future Work
-<!-- Alternative title: Filter-subscriber unlinkability -->
-**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways: 
-- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->. 
-  Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity. 
-- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
-
-## Changelog
-
-### Next
-
-- Added initial threat model and security analysis.
-   
-### 2.0.0-beta2
-
-Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
-- Fix: Ensure contentFilter is a repeated field, on implementation
-- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
-
-### 2.0.0-beta1
-
-Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-- [message-based
-filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
-- [`WakuMessage`s](../14/message.md)
-- [`11/WAKU2-RELAY`](../11/relay.md)
-- [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains)
-- previous versions: [00](./previous-versions00)
-
-1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
-
-2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)
+---
+slug: 12
+title: 12/WAKU2-FILTER
+name: Waku v2 Filter
+status: draft
+tags: waku-core
+version: 01
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskar@status.im>
+  - Sanaz Taheri <sanaz@status.im>
+  - Ebube Ud <ebube@status.im>
+---
+
+previous versions: [00](./previous-versions00)
+
+---
+
+`WakuFilter` is a protocol that enables subscribing to messages that a peer receives.
+This is a more lightweight version of `WakuRelay`
+specifically designed for bandwidth restricted devices.
+This is due to the fact that light nodes subscribe to full-nodes and
+only receive the messages they desire.
+
+## Content filtering
+
+**Protocol identifiers**:
+
+- _filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
+- _filter-push_: `/vac/waku/filter-push/2.0.0-beta1`
+
+Content filtering is a way to do [message-based
+filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
+Currently the only content filter being applied is on `contentTopic`. This
+corresponds to topics in Waku v1.
+
+## Rationale
+
+Unlike the `store` protocol for historical messages, this protocol allows for
+native lower latency scenarios such as instant messaging. It is thus
+complementary to it.
+
+Strictly speaking, it is not just doing basic request response, but performs
+sender push based on receiver intent. While this can be seen as a form of light
+pub/sub, it is only used between two nodes in a direct fashion. Unlike the
+Gossip domain, this is meant for light nodes which put a premium on bandwidth.
+No gossiping takes place.
+
+It is worth noting that a light node could get by with only using the `store`
+protocol to query for a recent time window, provided it is acceptable to do
+frequent polling.
+
+## Design Requirements
+
+The effectiveness and reliability of the content filtering service enabled by  
+`WakuFilter` protocol rely on the _high availability_ of the full nodes
+as the service providers.
+To this end, full nodes must feature _high uptime_
+(to persistently listen and capture the network messages)
+as well as _high Bandwidth_ (to provide timely message delivery to the light nodes).
+
+## Security Consideration
+
+Note that while using `WakuFilter` allows light nodes to save bandwidth,
+it comes with a privacy cost in the sense that they need to
+disclose their liking topics to the full nodes to retrieve the relevant messages.
+Currently, anonymous subscription is not supported by the `WakuFilter`, however,
+potential solutions in this regard are sketched
+below in [Future Work](#future-work) section.
+
+### Terminology
+
+The term Personally identifiable information (PII)
+refers to any piece of data that can be used to uniquely identify a user.
+For example, the signature verification key, and
+the hash of one's static IP address are unique for each user and hence count as PII.
+
+## Adversarial Model
+
+Any node running the `WakuFilter` protocol
+i.e., both the subscriber node and the queried node are considered as an adversary.
+Furthermore, we consider the adversary as a passive entity
+that attempts to collect information from other nodes to conduct an attack but
+it does so without violating protocol definitions and instructions.
+For example, under the passive adversarial model,
+no malicious node intentionally hides the messages
+matching to one's subscribed content filter
+as it is against the description of the `WakuFilter` protocol.
+
+The following are not considered as part of the adversarial model:
+
+- An adversary with a global view of all the nodes and their connections.
+- An adversary that can eavesdrop on communication links
+between arbitrary pairs of nodes (unless the adversary is one end of the communication).
+In specific, the communication channels are assumed to be secure.
+
+### Protobuf
+
+```protobuf
+syntax = "proto3";
+
+// 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/
+package waku.filter.v2;
+
+// Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1
+message FilterSubscribeRequest {
+  enum FilterSubscribeType {
+    SUBSCRIBER_PING = 0;
+    SUBSCRIBE = 1;
+    UNSUBSCRIBE = 2;
+    UNSUBSCRIBE_ALL = 3;
+  }
+
+  string request_id = 1;
+  FilterSubscribeType filter_subscribe_type = 2;
+
+  // Filter criteria
+  optional string pubsub_topic = 10;
+  repeated string content_topics = 11;
+}
+
+message FilterSubscribeResponse {
+  string request_id = 1;
+  uint32 status_code = 10;
+  optional string status_desc = 11;
+}
+
+// Protocol identifier: /vac/waku/filter-push/2.0.0-beta1
+message MessagePush {
+  WakuMessage waku_message = 1;
+  optional string pubsub_topic = 2;
+}
+```
+
+### Filter-Subscribe
+
+A filter service node MUST support the _filter-subscribe_ protocol
+to allow filter clients to subscribe, modify, refresh and
+unsubscribe a desired set of filter criteria.
+The combination of different filter criteria
+for a specific filter client node is termed a "subscription".
+A filter client is interested in receiving messages matching the filter criteria
+in its registered subscriptions.
+
+Since a filter service node is consuming resources to provide this service,
+it MAY account for usage and adapt its service provision to certain clients.
+An incentive mechanism is currently planned but underspecified.
+
+#### Filter Subscribe Request
+
+A client node MUST send all filter requests in a `FilterSubscribeRequest` message.
+This request MUST contain a `request_id`.
+The `request_id` MUST be a uniquely generated string.
+Each request MUST include a `filter_subscribe_type`, indicating the type of request.
+
+#### Filter Subscribe Response
+
+In return to any `FilterSubscribeRequest`,
+a filter service node SHOULD respond with a `FilterSubscribeResponse`
+with a `requestId` matching that of the request.
+This response MUST contain a `status_code` indicating if the request was successful
+or not.
+Successful status codes are in the `2xx` range.
+Client nodes SHOULD consider all other status codes as error codes and
+assume that the requested operation had failed.
+In addition,
+the filter service node MAY choose to provide a more detailed status description
+in the `status_desc` field.
+
+#### Filter matching
+
+In the description of each request type below,
+the term "filter criteria" refers to the combination of `pubsub_topic` and
+a set of `content_topics`.
+The request MAY include filter criteria,
+conditional to the selected `filter_subscribe_type`.
+If the request contains filter criteria,
+it MUST contain a `pubsub_topic`
+and the `content_topics` set MUST NOT be empty.
+A `WakuMessage` matches filter criteria
+when its `content_topic` is in the `content_topics` set
+and it was published on a matching `pubsub_topic`.
+
+#### Filter Subscribe Types
+
+The following filter subscribe types are defined:
+
+##### SUBSCRIBER_PING
+
+A filter client that sends a `FilterSubscribeRequest` with
+`filter_subscribe_type` set to `SUBSCRIBER_PING`
+requests that the service node SHOULD indicate if it has any active subscriptions
+for this client.
+The filter client SHOULD exclude any filter criteria from the request.
+The filter service node SHOULD respond with a success code
+if it has any active subscriptions for this client
+or an error code if not.
+The filter service node SHOULD ignore any filter criteria in the request.
+
+##### SUBSCRIBE
+
+A filter client that sends a `FilterSubscribeRequest` with
+`filter_subscribe_type` set to `SUBSCRIBE`
+requests that the service node SHOULD push messages matching this filter to the client.
+The filter client MUST include the desired filter criteria in the request.
+A client MAY use this request type to _modify_ an existing subscription
+by providing _additional_ filter criteria in a new request.
+A client MAY use this request type to _refresh_ an existing subscription
+by providing _the same_ filter criteria in a new request.
+The filter service node SHOULD respond with a success code
+if it successfully honored this request
+or an error code if not.
+The filter service node SHOULD respond with an error code and discard the request
+if the subscribe request does not contain valid filter criteria,
+i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
+
+##### UNSUBSCRIBE
+
+A filter client that sends a `FilterSubscribeRequest` with
+`filter_subscribe_type` set to `UNSUBSCRIBE`
+requests that the service node SHOULD _stop_ pushing messages
+matching this filter to the client.
+The filter client MUST include the filter criteria
+it desires to unsubscribe from in the request.
+A client MAY use this request type to _modify_ an existing subscription
+by providing _a subset of_ the original filter criteria
+to unsubscribe from in a new request.
+The filter service node SHOULD respond with a success code
+if it successfully honored this request
+or an error code if not.
+The filter service node SHOULD respond with an error code and discard the request
+if the unsubscribe request does not contain valid filter criteria,
+i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
+
+##### UNSUBSCRIBE_ALL
+
+A filter client that sends a `FilterSubscribeRequest` with
+`filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
+requests that the service node SHOULD _stop_ pushing messages
+matching _any_ filter to the client.
+The filter client SHOULD exclude any filter criteria from the request.
+The filter service node SHOULD remove any existing subscriptions for this client.
+It SHOULD respond with a success code if it successfully honored this request
+or an error code if not.
+
+### Filter-Push
+
+A filter client node MUST support the _filter-push_ protocol
+to allow filter service nodes to push messages
+matching registered subscriptions to this client.
+
+A filter service node SHOULD push all messages
+matching the filter criteria in a registered subscription
+to the subscribed filter client.
+These [`WakuMessage`s](../14/message.md) are likely to come from [`11/WAKU2-RELAY`](../11/relay.md),
+but there MAY be other sources or protocols where this comes from.
+This is up to the consumer of the protocol.
+
+If a message push fails,
+the filter service node MAY consider the client node to be unreachable.
+If a specific filter client node is not reachable from the service node
+for a period of time,
+the filter service node MAY choose to stop pushing messages to the client and
+remove its subscription.
+This period is up to the service node implementation.
+We consider `1 minute` to be a reasonable default.
+
+#### Message Push
+
+Each message MUST be pushed in a `MessagePush` message.
+Each `MessagePush` MUST contain one (and only one) `waku_message`.
+If this message was received on a specific `pubsub_topic`,
+it SHOULD be included in the `MessagePush`.
+A filter client SHOULD NOT respond to a `MessagePush`.
+Since the filter protocol does not include caching or fault-tolerance,
+this is a best effort push service with no bundling
+or guaranteed retransmission of messages.
+A filter client SHOULD verify that each `MessagePush` it receives
+originated from a service node where the client has an active subscription
+and that it matches filter criteria belonging to that subscription.
+
+---
+
+## Future Work
+<!-- Alternative title: Filter-subscriber unlinkability -->
+**Anonymous filter subscription**:
+This feature guarantees that nodes can anonymously subscribe for a message filter
+(i.e., without revealing their exact content filter).
+As such, no adversary in the `WakuFilter` protocol
+would be able to link nodes to their subscribed content filers.
+The current version of the `WakuFilter` protocol does not provide anonymity
+as the subscribing node has a direct connection to the full node and
+explicitly submits its content filter to be notified about the matching messages.
+However, one can consider preserving anonymity through one of the following ways:
+
+- By hiding the source of the subscription i.e., anonymous communication.
+That is the subscribing node shall hide all its PII in its filter request
+e.g., its IP address.
+This can happen by the utilization of a proxy server or by using Tor
+<!-- TODO: if nodes have to disclose their PeerIDs
+(e.g., for authentication purposes)
+when connecting to other nodes in the WakuFilter protocol,
+then Tor does not preserve anonymity since it only helps in hiding the IP.
+So, the PeerId usage in switches must be investigated further.
+Depending on how PeerId is used,
+one may be able to link between a subscriber and
+its content filter despite hiding the IP address-->.
+Note that the current structure of filter requests
+i.e., `FilterRPC` does not embody any piece of PII, otherwise,
+such data fields must be treated carefully to achieve anonymity.
+
+- By deploying secure 2-party computations in which
+the subscribing node obtains the messages matching a content filter
+whereas the full node learns nothing about the content filter as well as
+the messages pushed to the subscribing node.
+Examples of such 2PC protocols are
+[Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.)
+and one-way Private Set Intersections (PSI).
+
+## Changelog
+
+### Next
+
+- Added initial threat model and security analysis.
+
+### 2.0.0-beta2
+
+Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
+
+- Fix: Ensure contentFilter is a repeated field, on implementation
+- Change: Add ability to unsubscribe from filters.
+Make `subscribe` an explicit boolean indication.
+Edit protobuf field order to be consistent with libp2p.
+
+### 2.0.0-beta1
+
+Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [message-based
+filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
+- [`WakuMessage`s](../14/message.md)
+- [`11/WAKU2-RELAY`](../11/relay.md)
+- [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains)
+- previous versions: [00](./previous-versions00)
+
+1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
+
+2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)

waku/standards/core/12/previous-versions00/filter.md

@@ -1,171 +1,171 @@
----
-slug: 12
-title: 12/WAKU2-FILTER
-name: Waku v2 Filter
-status: draft
-tags: waku-core
-editor: Hanno Cornelius <hanno@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
-  - Sanaz Taheri <sanaz@status.im>
-  - Ebube Ud <ebube@status.im>
----
-version: 00
-
----
-`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
-
-## Content filtering
-
-**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
-
-Content filtering is a way to do [message-based
-filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
-Currently the only content filter being applied is on `contentTopic`. This
-corresponds to topics in Waku v1.
-
-## Rationale
-
-Unlike the `store` protocol for historical messages, this protocol allows for
-native lower latency scenarios such as instant messaging. It is thus
-complementary to it.
-
-Strictly speaking, it is not just doing basic request response, but performs
-sender push based on receiver intent. While this can be seen as a form of light
-pub/sub, it is only used between two nodes in a direct fashion. Unlike the
-Gossip domain, this is meant for light nodes which put a premium on bandwidth.
-No gossiping takes place.
-
-It is worth noting that a light node could get by with only using the `store`
-protocol to query for a recent time window, provided it is acceptable to do
-frequent polling.
-
-
-## Design Requirements
-
-The effectiveness and reliability of the content filtering service enabled by  `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes). 
-
-## Security Consideration
-
-Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section. 
-
-### Terminology
-The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
-
-## Adversarial Model
-Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol. 
-
-The following are not considered as part of the adversarial model: 
-  - An adversary with a global view of all the nodes and their connections. 
-  - An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
-
-### Protobuf
-
-```protobuf
-message FilterRequest {
-  bool subscribe = 1;
-  string topic = 2;
-  repeated ContentFilter contentFilters = 3;
-
-  message ContentFilter {
-    string contentTopic = 1;
-  }
-}
-
-message MessagePush {
-  repeated WakuMessage messages = 1;
-}
-
-message FilterRPC {
-  string requestId = 1;
-  FilterRequest request = 2;
-  MessagePush push = 3;
-}
-```
-
-#### FilterRPC
-
-A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
-`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
-Filter protocol is not a request response based protocol but instead a push based system.
-
-The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
-the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
-matched the message causing it to be pushed.
-
-#### FilterRequest
-
-A `FilterRequest` contains an optional topic, zero or more content filters and
-a boolean signifying whether to subscribe or unsubscribe to the given filters.
-True signifies 'subscribe' and false signifies 'unsubscribe'.
-
-A node that sends the RPC with a filter request and `subscribe` set to 'true' 
-requests that the filter node SHOULD notify the light requesting node of messages
-matching this filter.
-
-A node that sends the RPC with a filter request and `subscribe` set to 'false'
-requests that the filter node SHOULD stop notifying the light requesting node
-of messages matching this filter if it is currently doing so.
-
-The filter matches when content filter and, optionally, a topic is matched.
-Content filter is matched when a `WakuMessage` `contentTopic` field is the same.
-
-A filter node SHOULD honor this request, though it MAY choose not to do so. If
-it chooses not to do so it MAY tell the light why. The mechanism for doing this
-is currently not specified. For notifying the light node a filter node sends a
-MessagePush message.
-
-Since such a filter node is doing extra work for a light node, it MAY also
-account for usage and be selective in how much service it provides. This
-mechanism is currently planned but underspecified.
-
-#### MessagePush
-
-A filter node that has received a filter request SHOULD push all messages that
-match this filter to a light node. These [`WakuMessage`'s](../14/message.md) are likely to come from the
-`relay` protocol and be kept at the Node, but there MAY be other sources or
-protocols where this comes from. This is up to the consumer of the protocol.
-
-A filter node MUST NOT send a push message for messages that have not been
-requested via a FilterRequest.
-
-If a specific light node isn't connected to a filter node for some specific
-period of time (e.g. a TTL), then the filter node MAY choose to not push these
-messages to the node. This period is up to the consumer of the protocol and node
-implementation, though a reasonable default is one minute.
-
---- 
-# Future Work
-<!-- Alternative title: Filter-subscriber unlinkability -->
-**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways: 
-- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->. 
-  Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity. 
-- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
-
-## Changelog
-
-### Next
-
-- Added initial threat model and security analysis.
-   
-### 2.0.0-beta2
-
-Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
-- Fix: Ensure contentFilter is a repeated field, on implementation
-- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
-
-### 2.0.0-beta1
-
-Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
-
-2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)
+---
+slug: 12
+title: 12/WAKU2-FILTER
+name: Waku v2 Filter
+status: draft
+tags: waku-core
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+  - Sanaz Taheri <sanaz@status.im>
+  - Ebube Ud <ebube@status.im>
+---
+version: 00
+
+---
+`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
+
+## Content filtering
+
+**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
+
+Content filtering is a way to do [message-based
+filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
+Currently the only content filter being applied is on `contentTopic`. This
+corresponds to topics in Waku v1.
+
+## Rationale
+
+Unlike the `store` protocol for historical messages, this protocol allows for
+native lower latency scenarios such as instant messaging. It is thus
+complementary to it.
+
+Strictly speaking, it is not just doing basic request response, but performs
+sender push based on receiver intent. While this can be seen as a form of light
+pub/sub, it is only used between two nodes in a direct fashion. Unlike the
+Gossip domain, this is meant for light nodes which put a premium on bandwidth.
+No gossiping takes place.
+
+It is worth noting that a light node could get by with only using the `store`
+protocol to query for a recent time window, provided it is acceptable to do
+frequent polling.
+
+
+## Design Requirements
+
+The effectiveness and reliability of the content filtering service enabled by  `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes). 
+
+## Security Consideration
+
+Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section. 
+
+### Terminology
+The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
+
+## Adversarial Model
+Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol. 
+
+The following are not considered as part of the adversarial model: 
+  - An adversary with a global view of all the nodes and their connections. 
+  - An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
+
+### Protobuf
+
+```protobuf
+message FilterRequest {
+  bool subscribe = 1;
+  string topic = 2;
+  repeated ContentFilter contentFilters = 3;
+
+  message ContentFilter {
+    string contentTopic = 1;
+  }
+}
+
+message MessagePush {
+  repeated WakuMessage messages = 1;
+}
+
+message FilterRPC {
+  string requestId = 1;
+  FilterRequest request = 2;
+  MessagePush push = 3;
+}
+```
+
+#### FilterRPC
+
+A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
+`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
+Filter protocol is not a request response based protocol but instead a push based system.
+
+The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
+the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
+matched the message causing it to be pushed.
+
+#### FilterRequest
+
+A `FilterRequest` contains an optional topic, zero or more content filters and
+a boolean signifying whether to subscribe or unsubscribe to the given filters.
+True signifies 'subscribe' and false signifies 'unsubscribe'.
+
+A node that sends the RPC with a filter request and `subscribe` set to 'true' 
+requests that the filter node SHOULD notify the light requesting node of messages
+matching this filter.
+
+A node that sends the RPC with a filter request and `subscribe` set to 'false'
+requests that the filter node SHOULD stop notifying the light requesting node
+of messages matching this filter if it is currently doing so.
+
+The filter matches when content filter and, optionally, a topic is matched.
+Content filter is matched when a `WakuMessage` `contentTopic` field is the same.
+
+A filter node SHOULD honor this request, though it MAY choose not to do so. If
+it chooses not to do so it MAY tell the light why. The mechanism for doing this
+is currently not specified. For notifying the light node a filter node sends a
+MessagePush message.
+
+Since such a filter node is doing extra work for a light node, it MAY also
+account for usage and be selective in how much service it provides. This
+mechanism is currently planned but underspecified.
+
+#### MessagePush
+
+A filter node that has received a filter request SHOULD push all messages that
+match this filter to a light node. These [`WakuMessage`'s](../14/message.md) are likely to come from the
+`relay` protocol and be kept at the Node, but there MAY be other sources or
+protocols where this comes from. This is up to the consumer of the protocol.
+
+A filter node MUST NOT send a push message for messages that have not been
+requested via a FilterRequest.
+
+If a specific light node isn't connected to a filter node for some specific
+period of time (e.g. a TTL), then the filter node MAY choose to not push these
+messages to the node. This period is up to the consumer of the protocol and node
+implementation, though a reasonable default is one minute.
+
+--- 
+# Future Work
+<!-- Alternative title: Filter-subscriber unlinkability -->
+**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways: 
+- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->. 
+  Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity. 
+- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
+
+## Changelog
+
+### Next
+
+- Added initial threat model and security analysis.
+   
+### 2.0.0-beta2
+
+Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
+- Fix: Ensure contentFilter is a repeated field, on implementation
+- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
+
+### 2.0.0-beta1
+
+Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
+
+2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)

waku/standards/core/13/store.md

@@ -1,259 +1,358 @@
----
-slug: 13
-title: 13/WAKU2-STORE
-name: Waku v2 Store
-status: draft
-tags: waku-core
-editor: Sanaz Taheri <sanaz@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
-  - Aaryamann Challani <aaryamann@status.im>
----
-
-## Abstract
-This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and 
-stored by other nodes. 
-It also supports pagination for more efficient querying of historical messages. 
-
-**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
-
-## Terminology
-The term PII, Personally Identifiable Information, 
-refers to any piece of data that can be used to uniquely identify a user. 
-For example, the signature verification key, and 
-the hash of one's static IP address are unique for each user and hence count as PII.
-
-## Design Requirements
-The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, 
-“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
-
-Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
-SHOULD provide a complete and full view of message history.
-As such, they are required to be *highly available* and 
-specifically have a *high uptime* to consistently receive and store network messages. 
-The high uptime requirement makes sure that no message is missed out hence a complete and 
-intact view of the message history is delivered to the querying nodes.
-Nevertheless, in case storage provider nodes cannot afford high availability, 
-the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
-
-The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
-Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](../14/message.md).
-Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
-
-## Adversarial Model
-Any peer running the `13/WAKU2-STORE` protocol, i.e. 
-both the querying node and the queried node, are considered as an adversary. 
-Furthermore, 
-we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but 
-it does so without violating protocol definitions and instructions. 
-As we evolve the protocol, 
-further adversarial models will be considered.
-For example, under the passive adversarial model, 
-no malicious node hides or 
-lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol. 
-
-The following are not considered as part of the adversarial model:
-- An adversary with a global view of all the peers and their connections.
-- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). 
-In specific, the communication channels are assumed to be secure.
-
-## Wire Specification
-Peers communicate with each other using a request / response API. 
-The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
-The following are the specifications of the Protobuf messages.
-
-### Payloads
-
-```protobuf
-syntax = "proto3";
-
-message Index {
-  bytes digest = 1;
-  sint64 receiverTime = 2;
-  sint64 senderTime = 3;
-  string pubsubTopic = 4;
-}
-
-message PagingInfo {
-  uint64 pageSize = 1;
-  Index cursor = 2;
-  enum Direction {
-    BACKWARD = 0;
-    FORWARD = 1;
-  }
-  Direction direction = 3;
-}
-
-message ContentFilter {
-  string contentTopic = 1;
-}
-
-message HistoryQuery {
-  // the first field is reserved for future use
-  string pubsubtopic = 2;
-  repeated ContentFilter contentFilters = 3;
-  PagingInfo pagingInfo = 4;
-}
-
-message HistoryResponse {
-  // the first field is reserved for future use
-  repeated WakuMessage messages = 2;
-  PagingInfo pagingInfo = 3;
-  enum Error {
-    NONE = 0;
-    INVALID_CURSOR = 1;
-  }
-  Error error = 4;
-}
-
-message HistoryRPC {
-  string request_id = 1;
-  HistoryQuery query = 2;
-  HistoryResponse response = 3;
-}
-```
-
-#### Index
-
-To perform pagination, 
-each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts. 
-- `digest`:  a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
-  The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
-- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node.
-- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender.
-- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
-
-#### PagingInfo
-
-`PagingInfo` holds the information required for pagination. It consists of the following components.
-- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery`
-(or retrieved  `WakuMessage`s in a `HistoryResponse`).
-- `cursor`: holds the `Index` of a `WakuMessage`.
-- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
-
-#### ContentFilter
-`ContentFilter` carries the information required for filtering historical messages. 
-- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
-  This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
-  
-#### HistoryQuery
-
-RPC call to query historical messages.
-
-- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved. 
-  This field denotes the pubsub topic on which `WakuMessage`s are published.
-  This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
-  Leaving this field empty means no filter on the pubsub topic of message history is requested.
-  This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
-- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
-  Leaving this field empty means no filter on the content topic of message history is required.
-  This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
-- `PagingInfo` holds the information required for pagination.  
-  Its `pageSize` field indicates the number of  `WakuMessage`s to be included in the corresponding `HistoryResponse`. 
-  It is RECOMMENDED that the queried node defines a maximum page size internally.
-  If the querying node leaves the `pageSize` unspecified,
-  or if the `pageSize` exceeds the maximum page size,
-  the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
-  This allows mitigation of long response time for `HistoryQuery`.
-  In the forward pagination request,
-  the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
-  the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
-  (and vise versa for the backward pagination). 
-  Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
-  depending on whether the  `direction` is `BACKWARD` or `FORWARD`  the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
-
-#### Sorting Messages
-The queried node MUST sort the `WakuMessage` based on their `Index`, 
-where the `senderTime` constitutes the most significant part and the `digest` comes next, and 
-then perform pagination on the sorted result. 
-As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one.
-Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process. 
-However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and 
-consistent across all the nodes.
-This has the benefit of `cursor` reusability i.e., 
-a `cursor` obtained from one node can be consistently used to query from another node.
-However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and 
-nodes' clock asynchrony.
-
-#### HistoryResponse
-
-RPC call to respond to a HistoryQuery call.
-- The `messages` field MUST contain the messages found,
-these are [14/WAKU2-MESSAGE](../14/message.md) types.
-- `PagingInfo`  holds the paging information based on which the querying node can resume its further history queries. 
-  The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`). 
-  The `direction` is the same direction as in the corresponding `HistoryQuery`. 
-  In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging). 
-  Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent.
-  The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).  
-  The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different.
-- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`.
-  `NONE` stands for no error.
-  This is also the default value.
-  `INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
-
-## Security Consideration
-
-The main security consideration to take into account while using this protocol is that a querying node have to reveal their content filters of interest to the queried node, hence potentially compromising their privacy.
-
-## Future Work
-
-- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
-without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md) they are interested in.  
-As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
-content topics of [14/WAKU2-MESSAGE](../14/message.md). 
-The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries,
-as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and
-explicitly disclose the content filters of its interest to retrieve the corresponding messages. 
-However, one can consider preserving anonymity through one of the following ways: 
-  - By hiding the source of the request i.e., anonymous communication.
-  That is the querying node shall hide all its PII in its history request e.g., its IP address.
-  This can happen by the utilization of a proxy server or by using Tor. 
-  Note that the current structure of historical requests does not embody any piece of PII, otherwise,
-  such data fields must be treated carefully to achieve query anonymity. 
-  <!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address--> 
-  - By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic,
-  the queried node learns nothing about the query. 
-  Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI). 
-  <!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included --> 
-<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).-->
-
-- **Robust and verifiable timestamps**: Messages timestamp is a way to show that the message existed prior to some point in time.
-However, the lack of timestamp verifiability can create room for a range of attacks,
-including injecting messages with invalid timestamps pointing to the far future.   
-To better understand the attack,
-consider a store node whose current clock shows `2021-01-01 00:00:30` (and assume all the other nodes have a synchronized clocks +-20seconds).
-The store node already has a list of messages,
- `(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
-that are sorted based on their timestamp.  
-An attacker sends a message with an arbitrary large timestamp e.g.,
-10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`. 
-The store node places `m'` at the end of the list,
-`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30)`. 
-Now another message arrives with a valid timestamp e.g.,
-`(m11, 2021-01-01 00:00:45)`.
-However, since its timestamp precedes the malicious message `m'`,
-it gets placed before `m'` in the list i.e.,
-`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
-In fact, for the next 10 hours,
-`m'` will always be considered as the most recent message and
-served as the last message to the querying nodes irrespective of how many other messages arrive afterward. 
-
-A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp. 
-One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
-block height in Blockchain-based timestamps. 
-That is, messages contain the most recent block height perceived by their senders at the time of message generation. 
-This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or 
-seconds (e.g., in Ethereum 2.0) from the time of origination. 
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-1. [14/WAKU2-MESSAGE](../14/message.md)
-2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
-3. [11/WAKU2-RELAY](../11/relay.md)
-4. [Open timestamps](https://opentimestamps.org/) 
+---
+slug: 13
+title: 13/WAKU2-STORE
+name: Waku v2 Store
+status: draft
+tags: waku-core
+editor: Simon-Pierre Vivier <simvivier@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+  - Aaryamann Challani <p1ge0nh8er@proton.me>
+  - Sanaz Taheri <sanaz@status.im>
+  - Hanno Cornelius <hanno@status.im>
+---
+
+## Abstract
+
+This specification explains the `13/WAKU2-STORE` protocol
+which enables querying of messages received through the relay protocol and
+stored by other nodes.
+It also supports pagination for more efficient querying of historical messages.
+
+**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
+
+## Terminology
+
+The term PII, Personally Identifiable Information,
+refers to any piece of data that can be used to uniquely identify a user.
+For example, the signature verification key, and
+the hash of one's static IP address are unique for each user and hence count as PII.
+
+## Design Requirements
+
+The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
+“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
+“OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
+
+Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
+SHOULD provide a complete and full view of message history.
+As such, they are required to be *highly available* and
+specifically have a *high uptime* to consistently receive and store network messages.
+The high uptime requirement makes sure that no message is missed out
+hence a complete and intact view of the message history
+is delivered to the querying nodes.
+Nevertheless, in case storage provider nodes cannot afford high availability,
+the querying nodes may retrieve the historical messages from multiple sources
+to achieve a full and intact view of the past.
+
+The concept of `ephemeral` messages introduced in
+[`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
+Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in
+[14/WAKU2-MESSAGE](../14/message.md).
+Nodes running `13/WAKU2-STORE` SHOULD NOT store messages
+with the `ephemeral` flag set to `true`.
+
+## Adversarial Model
+
+Any peer running the `13/WAKU2-STORE` protocol, i.e.
+both the querying node and the queried node, are considered as an adversary.
+Furthermore,
+we currently consider the adversary as a passive entity
+that attempts to collect information from other peers to conduct an attack but
+it does so without violating protocol definitions and instructions.
+As we evolve the protocol,
+further adversarial models will be considered.
+For example, under the passive adversarial model,
+no malicious node hides or
+lies about the history of messages
+as it is against the description of the `13/WAKU2-STORE` protocol.
+
+The following are not considered as part of the adversarial model:
+
+- An adversary with a global view of all the peers and their connections.
+- An adversary that can eavesdrop on communication links
+between arbitrary pairs of peers (unless the adversary is one end of the communication).
+In specific, the communication channels are assumed to be secure.
+
+## Wire Specification
+
+Peers communicate with each other using a request / response API.
+The messages sent are Protobuf RPC messages which are implemented using
+[protocol buffers v3](https://developers.google.com/protocol-buffers/).
+The following are the specifications of the Protobuf messages.
+
+### Payloads
+
+```protobuf
+syntax = "proto3";
+
+message Index {
+  bytes digest = 1;
+  sint64 receiverTime = 2;
+  sint64 senderTime = 3;
+  string pubsubTopic = 4;
+}
+
+message PagingInfo {
+  uint64 pageSize = 1;
+  Index cursor = 2;
+  enum Direction {
+    BACKWARD = 0;
+    FORWARD = 1;
+  }
+  Direction direction = 3;
+}
+
+message ContentFilter {
+  string contentTopic = 1;
+}
+
+message HistoryQuery {
+  // the first field is reserved for future use
+  string pubsubtopic = 2;
+  repeated ContentFilter contentFilters = 3;
+  PagingInfo pagingInfo = 4;
+}
+
+message HistoryResponse {
+  // the first field is reserved for future use
+  repeated WakuMessage messages = 2;
+  PagingInfo pagingInfo = 3;
+  enum Error {
+    NONE = 0;
+    INVALID_CURSOR = 1;
+  }
+  Error error = 4;
+}
+
+message HistoryRPC {
+  string request_id = 1;
+  HistoryQuery query = 2;
+  HistoryResponse response = 3;
+}
+```
+
+#### Index
+
+To perform pagination,
+each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol
+is associated with a unique `Index` that encapsulates the following parts.
+
+- `digest`:  a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
+The hash is computed over the concatenation of `contentTopic`
+and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
+- `receiverTime`: the UNIX time in nanoseconds
+at which the `WakuMessage` is received by the receiving node.
+- `senderTime`: the UNIX time in nanoseconds
+at which the `WakuMessage` is generated by its sender.
+- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
+
+#### PagingInfo
+
+`PagingInfo` holds the information required for pagination.
+It consists of the following components.
+
+- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s
+in a `HistoryQuery`
+(or retrieved  `WakuMessage`s in a `HistoryResponse`).
+- `cursor`: holds the `Index` of a `WakuMessage`.
+- `direction`: indicates the direction of paging
+which can be either `FORWARD` or `BACKWARD`.
+
+#### ContentFilter
+
+`ContentFilter` carries the information required for filtering historical messages.
+
+- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
+  This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
+  
+#### HistoryQuery
+
+RPC call to query historical messages.
+
+- The `pubsubTopic` field MUST indicate the pubsub topic
+of the historical messages to be retrieved.
+This field denotes the pubsub topic on which `WakuMessage`s are published.
+This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
+Leaving this field empty means no filter on the pubsub topic
+of message history is requested.
+This field SHOULD be left empty in order to retrieve the historical `WakuMessage`
+regardless of the pubsub topics on which they are published.
+- The `contentFilters` field MUST indicate the list of content filters
+based on which the historical messages are to be retrieved.
+Leaving this field empty means no filter on the content topic
+of message history is required.
+This field SHOULD be left empty in order
+to retrieve historical `WakuMessage` regardless of their content topics.
+- `PagingInfo` holds the information required for pagination.  
+Its `pageSize` field indicates the number of  `WakuMessage`s
+to be included in the corresponding `HistoryResponse`.
+It is RECOMMENDED that the queried node defines a maximum page size internally.
+If the querying node leaves the `pageSize` unspecified,
+or if the `pageSize` exceeds the maximum page size,
+the queried node SHOULD auto-paginate the `HistoryResponse`
+to no more than the configured maximum page size.
+This allows mitigation of long response time for `HistoryQuery`.
+In the forward pagination request,
+the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
+the `pageSize` amount of `WakuMessage` whose `Index`
+values are larger than the given `cursor`
+(and vise versa for the backward pagination).
+Note that the `cursor` of a `HistoryQuery` MAY be empty
+(e.g., for the initial query), as such, and
+depending on whether the  `direction` is `BACKWARD` or
+`FORWARD`  the last or the first `pageSize` `WakuMessage` SHALL be returned,
+respectively.
+
+#### Sorting Messages
+
+The queried node MUST sort the `WakuMessage` based on their `Index`,
+where the `senderTime` constitutes the most significant part and
+the `digest` comes next, and
+then perform pagination on the sorted result.
+As such, the retrieved page contains an ordered list of `WakuMessage`
+from the oldest messages to the most recent one.
+Alternatively, the `receiverTime` (instead of `senderTime`)
+MAY be used to sort messages during the paging process.
+However, it is RECOMMENDED the use of the `senderTime`
+for sorting as it is invariant and
+consistent across all the nodes.
+This has the benefit of `cursor` reusability i.e.,
+a `cursor` obtained from one node can be consistently used
+to query from another node.
+However, this `cursor` reusability does not hold when the `receiverTime` is utilized
+as the receiver time is affected by the network delay and
+nodes' clock asynchrony.
+
+#### HistoryResponse
+
+RPC call to respond to a HistoryQuery call.
+
+- The `messages` field MUST contain the messages found,
+these are [14/WAKU2-MESSAGE](../14/message.md) types.
+- `PagingInfo`  holds the paging information based
+on which the querying node can resume its further history queries.
+The `pageSize` indicates the number of returned Waku messages
+(i.e., the number of messages included in the `messages` field of `HistoryResponse`).
+The `direction` is the same direction as in the corresponding `HistoryQuery`.
+In the forward pagination, the `cursor` holds the `Index` of the last message
+in the `HistoryResponse` `messages` (and the first message in the backward paging).
+Regardless of the paging direction,
+the retrieved `messages` are always sorted in ascending order
+based on their timestamp as explained in the [sorting messages](#sorting-messages)section,
+that is, from the oldest to the most recent.
+The requester SHALL embed the returned `cursor` inside its next `HistoryQuery`
+to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).  
+The `cursor` obtained from one node SHOULD NOT be used in a request to another node
+because the result may be different.
+- The `error` field contains information about any error that has occurred
+while processing the corresponding `HistoryQuery`.
+`NONE` stands for no error.
+This is also the default value.
+`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery`
+does not match with the `Index` of any of the `WakuMessage`
+persisted by the queried node.
+
+## Security Consideration
+
+The main security consideration to take into account
+while using this protocol is that a querying node
+have to reveal their content filters of interest to the queried node,
+hence potentially compromising their privacy.
+
+## Future Work
+
+- **Anonymous query**: This feature guarantees that nodes
+can anonymously query historical messages from other nodes i.e.,
+without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md)
+they are interested in.  
+As such, no adversary in the `13/WAKU2-STORE` protocol
+would be able to learn which peer is interested in which content filters i.e.,
+content topics of [14/WAKU2-MESSAGE](../14/message.md).
+The current version of the `13/WAKU2-STORE` protocol does not provide anonymity
+for historical queries,
+as the querying node needs to directly connect to another node
+in the `13/WAKU2-STORE` protocol and
+explicitly disclose the content filters of its interest
+to retrieve the corresponding messages.
+However, one can consider preserving anonymity through one of the following ways:
+  - By hiding the source of the request i.e., anonymous communication.
+  That is the querying node shall hide all its PII in its history request
+  e.g., its IP address.
+  This can happen by the utilization of a proxy server or by using Tor.
+  Note that the current structure of historical requests
+  does not embody any piece of PII, otherwise,
+  such data fields must be treated carefully to achieve query anonymity.
+  <!-- TODO: if nodes have to disclose their PeerIDs
+  (e.g., for authentication purposes) when connecting to other nodes
+  in the store protocol,
+  then Tor does not preserve anonymity since it only helps in hiding the IP.
+  So, the PeerId usage in switches must be investigated further.
+  Depending on how PeerId is used, one may be able to link between a querying node
+  and its queried topics despite hiding the IP address-->
+  - By deploying secure 2-party computations in which the querying node
+  obtains the historical messages of a certain topic,
+  the queried node learns nothing about the query.
+  Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
+  <!-- TODO: add a reference for PSIs? -->
+  <!-- TODO: more techniques to be included -->
+<!-- TODO: Censorship resistant:
+this is about a node that hides the historical messages from other nodes.
+This attack is not included in the specs
+since it does not fit the passive adversarial model
+(the attacker needs to deviate from the store protocol).-->
+
+- **Robust and verifiable timestamps**:
+Messages timestamp is a way to show that the message existed
+prior to some point in time.
+However, the lack of timestamp verifiability can create room for a range of attacks,
+including injecting messages with invalid timestamps pointing to the far future.
+To better understand the attack,
+consider a store node whose current clock shows `2021-01-01 00:00:30`
+(and assume all the other nodes have a synchronized clocks +-20seconds).
+The store node already has a list of messages,
+ `(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
+that are sorted based on their timestamp.  
+An attacker sends a message with an arbitrary large timestamp e.g.,
+10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
+The store node places `m'` at the end of the list,
+
+```text
+(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),(m',2021-01-01 10:00:30).
+```
+
+Now another message arrives with a valid timestamp e.g.,
+`(m11, 2021-01-01 00:00:45)`.
+However, since its timestamp precedes the malicious message `m'`,
+it gets placed before `m'` in the list i.e.,
+
+```text
+(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30).
+```
+
+In fact, for the next 10 hours,
+`m'` will always be considered as the most recent message and
+served as the last message to the querying nodes irrespective
+of how many other messages arrive afterward.
+
+A robust and verifiable timestamp allows the receiver of a message
+to verify that a message has been generated prior to the claimed timestamp.
+One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
+block height in Blockchain-based timestamps.
+That is, messages contain the most recent block height
+perceived by their senders at the time of message generation.
+This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
+seconds (e.g., in Ethereum 2.0) from the time of origination.
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [14/WAKU2-MESSAGE](../14/message.md)
+2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
+3. [11/WAKU2-RELAY](../11/relay.md)
+4. [Open timestamps](https://opentimestamps.org/)

waku/standards/core/14/message.md

@@ -1,217 +1,258 @@
----
-slug: 14
-title: 14/WAKU2-MESSAGE
-name: Waku v2 Message
-status: draft
-category: Standards Track
-tags: waku/core-protocol
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributors:
-  - Sanaz Taheri <sanaz@status.im>
-  - Aaryamann Challani <aaryamann@status.im>
-  - Lorenzo Delgado <lorenzo@status.im>
-  - Abhimanyu Rawat <abhi@status.im>
----
-
-## Abstract
-
-Waku v2 is a family of modular peer-to-peer protocols for secure communication.
-These protocols are designed to be secure, privacy-preserving, and censorship-resistant and can run in resource-restricted environments.
-At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and adds capabilities.
-
-The present document specifies the Waku v2 message format, a way to encapsulate the messages sent with specific information security goals, and Whisper/Waku v1 backward compatibility.
-
-
-## Motivation
-
-When sending messages over Waku, there are multiple requirements:
-
-- One may have a separate encryption layer as part of the application.
-- One may want to provide efficient routing for resource-restricted devices.
-- One may want to provide compatibility with [Waku v1 envelopes](../6/waku1.md).
-- One may want encrypted payloads by default.
-- One may want to provide unlinkability to get metadata protection.
-
-This specification attempts to provide for these various requirements.
-
-
-## Semantics
-
-### Waku Message
-
-A Waku message is constituted by the combination of data payload and attributes that, for example, a *publisher* sends to a *topic* and is eventually delivered to *subscribers*.
-
-Waku message attributes are key-value pairs of metadata associated with a message. 
-And the message data payload is the part of the transmitted Waku message that is the actual message information.
-The data payload is also treated as a Waku message attribute for convenience.
-
-### Message Attributes
-
-* The `payload` attribute MUST contain the message data payload to be sent.
-
-* The `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering,
-as described in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
-
-* The `meta` attribute, if present, contains an arbitrary application-specific variable-length byte array with a maximum length limit of 64 bytes.
-This attribute can be utilized to convey supplementary details to various Waku protocols, thereby enabling customized processing based on its contents.
-
-* The `version` attribute, if present, contains a version number to discriminate different types of payload encryption.
-If omitted, the value SHOULD be interpreted as version 0.
-
-* The `timestamp` attribute, if present, signifies the time at which the message was generated by its sender.
-This attribute MAY contain the Unix epoch time in nanoseconds.
-If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
-
-* The `ephemeral` attribute, if present, signifies the transient nature of the message.
-For example, an ephemeral message SHOULD not be persisted by the Waku network.
-If this attribute is set to `true`, the message SHOULD be interpreted as ephemeral.
-If, instead, the attribute is omitted or set to `false`, the message SHOULD be interpreted as non-ephemeral.
-
-## Wire Format
-
-The Waku message wire format is specified using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
-
-```protobuf
-syntax = "proto3";
-
-message WakuMessage {
-  bytes payload = 1;
-  string content_topic = 2;
-  optional uint32 version = 3;
-  optional sint64 timestamp = 10;
-  optional bytes meta = 11;
-  optional bool ephemeral = 31;
-}
-```
-
-An example proto file following this specification can be found [here (vacp2p/waku)](https://github.com/vacp2p/waku/blob/main/waku/message/v1/message.proto).
-
-
-## Payload encryption
-
-The Waku message payload MAY be encrypted.
-The message `version` attribute indicates the schema used to encrypt the payload data.
-
-- **Version 0:**
-  The payload SHOULD be interpreted as unencrypted; additionally, it CAN indicate that the message payload has been encrypted at the application layer.
-
-- **Version 1:**
-The payload SHOULD be encrypted using Waku v1 payload encryption specified in [26/WAKU-PAYLOAD](../../application/26/payload.md).
-This provides asymmetric and symmetric encryption.
-The key agreement is performed out of band.
-And provides an encrypted signature and padding for some form of unlinkability.
-
-- **Version 2:**
-The payload SHOULD be encoded according to [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)).
-Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
-
-Any `version` value not included in this list is reserved for future specification.
-And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
-
-
-## Whisper/Waku v1 envelope compatibility
-
-Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
-
-* Whisper/Waku v1 `topic` field SHOULD be mapped to Waku v2 message's `content_topic` attribute.
-* Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
-
-Waku v2 implements a pub/sub messaging pattern over libp2p.
-This makes redundant some Whisper/Waku v1 envelope fields (e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
-
-## Deterministic message hashing
-
-In Protocol Buffers v3, the deterministic serialization is not canonical across the different implementations and languages.
-It is also unstable across different builds with schema changes due to unknown fields. 
-
-To overcome this interoperability limitation, a Waku v2 message's hash MUST be computed following this schema:
-
-```
-message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp))
-```
-
-If an optional attribute, such as `meta`, is absent, the concatenation of attributes SHOULD exclude it. This recommendation is made to ensure that the concatenation process proceeds smoothly when certain attributes are missing and to maintain backward compatibility.
-
-This hashing schema is deemed appropriate for use cases where a cross-implementation deterministic hash is needed, such as message deduplication and integrity validation. The collision probability offered by this hashing schema can be considered negligible. This is due to the deterministic concatenation order of the message attributes, coupled with using a SHA-2 (256-bit) hashing algorithm.
-
-### Test vectors
-
-Waku message hash computation (`meta` size of 12 bytes):
-```
-pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
-message.payload = 0x010203045445535405060708
-message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
-message.meta = 0x73757065722d736563726574
-message.timestamp = 0x175789bfa23f8400
-
-message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05
-```
-
-Waku message hash computation (`meta` size of 64 bytes):
-
-```
-pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
-message.payload = 0x010203045445535405060708
-message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
-message.meta = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f
-message.timestamp = 0x175789bfa23f8400
-
-message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27
-```
-
-Waku message hash computation (`meta` attribute not present):
-```
-pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
-message.payload = 0x010203045445535405060708
-message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
-message.meta = <not-present>
-message.timestamp = 0x175789bfa23f8400
-
-message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8
-```
-
-Waku message hash computation (`payload` length 0):
-```
-pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
-message.payload = []
-message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
-message.meta = 0x73757065722d736563726574
-message.timestamp = 0x175789bfa23f8400
-
-message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4
-```
-
-## Security Considerations
-
-### Confidentiality, integrity, and authenticity
-
-The level of confidentiality, integrity, and authenticity of the Waku message payload is discretionary.
-Accordingly, the application layer shall utilize the encryption and signature schemes supported by Waku v2 to meet the application-specific privacy needs.
-
-### Reliability of the `timestamp` attribute
-
-The Waku message `timestamp` attribute is set by the sender.
-Therefore, because message timestamps aren’t independently verified, this attribute is prone to exploitation and misuse.
-It should not solely be relied upon for operations such as message ordering.
-For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message to a high value so that it always shows up as the most recent message in a chat application.
-Applications using Waku messages’ `timestamp` attribute are recommended to use additional methods for more robust message ordering.
-An example of how to deal with message ordering against adversarial message timestamps can be found in the Status protocol, see [6/PAYLOADS](../6/waku1.md/#clock-vs-timestamp-and-message-ordering).
-
-### Reliability of the `ephemeral` attribute
-
-The Waku message `ephemeral` attribute is set by the sender.
-Since there is currently no incentive mechanism for network participants to behave correctly, this attribute is inherently insecure.
-A malicious actor can tamper with the value of a Waku message’s `ephemeral` attribute, and the receiver would not be able to verify the integrity of the message.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-
-## References
-
-- [6/WAKU1](/spec/6/)
-- [Google Protocol buffers v3](https://developers.google.com/protocol-buffers/)
-- [26/WAKU-PAYLOAD](../../application/26/payload.md)
-- [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md))
-- [6/PAYLOADS](https://specs.status.im/spec/6#clock-vs-timestamp-and-message-ordering)
+---
+slug: 14
+title: 14/WAKU2-MESSAGE
+name: Waku v2 Message
+status: draft
+category: Standards Track
+tags: waku/core-protocol
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+  - Sanaz Taheri <sanaz@status.im>
+  - Aaryamann Challani <p1ge0nh8er@proton.me>
+  - Lorenzo Delgado <lorenzo@status.im>
+  - Abhimanyu Rawat <abhi@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+## Abstract
+
+Waku v2 is a family of modular peer-to-peer protocols for secure communication.
+These protocols are designed to be secure, privacy-preserving,
+and censorship-resistant and can run in resource-restricted environments.
+At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and
+adds capabilities.
+
+The present document specifies the Waku v2 message format,
+a way to encapsulate the messages sent with specific information security goals,
+and Whisper/Waku v1 backward compatibility.
+
+## Motivation
+
+When sending messages over Waku, there are multiple requirements:
+
+- One may have a separate encryption layer as part of the application.
+- One may want to provide efficient routing for resource-restricted devices.
+- One may want to provide compatibility with [Waku v1 envelopes](../../legacy/6/waku1.md).
+- One may want encrypted payloads by default.
+- One may want to provide unlinkability to get metadata protection.
+
+This specification attempts to provide for these various requirements.
+
+## Semantics
+
+### Waku Message
+
+A Waku message is constituted by the combination of data payload and
+attributes that, for example, a *publisher* sends to a *topic* and
+is eventually delivered to *subscribers*.
+
+Waku message attributes are key-value pairs of metadata associated with a message.
+And the message data payload is the part of the transmitted Waku message
+that is the actual message information.
+The data payload is also treated as a Waku message attribute for convenience.
+
+### Message Attributes
+
+- The `payload` attribute MUST contain the message data payload to be sent.
+
+- The `content_topic` attribute MUST specify a string identifier
+that can be used for content-based filtering,
+as described in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
+
+- The `meta` attribute, if present,
+contains an arbitrary application-specific variable-length byte array
+with a maximum length limit of 64 bytes.
+This attribute can be utilized to convey supplementary details
+to various Waku protocols,
+thereby enabling customized processing based on its contents.
+
+- The `version` attribute, if present,
+contains a version number to discriminate different types of payload encryption.
+If omitted, the value SHOULD be interpreted as version 0.
+
+- The `timestamp` attribute, if present,
+signifies the time at which the message was generated by its sender.
+This attribute MAY contain the Unix epoch time in nanoseconds.
+If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
+
+- The `ephemeral` attribute, if present, signifies the transient nature of the message.
+For example, an ephemeral message SHOULD not be persisted by the Waku network.
+If this attribute is set to `true`, the message SHOULD be interpreted as ephemeral.
+If, instead, the attribute is omitted or set to `false`,
+the message SHOULD be interpreted as non-ephemeral.
+
+## Wire Format
+
+The Waku message wire format is specified using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
+
+```protobuf
+syntax = "proto3";
+
+message WakuMessage {
+  bytes payload = 1;
+  string content_topic = 2;
+  optional uint32 version = 3;
+  optional sint64 timestamp = 10;
+  optional bytes meta = 11;
+  optional bool ephemeral = 31;
+}
+```
+
+An example proto file following this specification can be found [here (vacp2p/waku)](https://github.com/vacp2p/waku/blob/main/waku/message/v1/message.proto).
+
+## Payload encryption
+
+The Waku message payload MAY be encrypted.
+The message `version` attribute indicates
+the schema used to encrypt the payload data.
+
+- **Version 0:**
+  The payload SHOULD be interpreted as unencrypted; additionally,
+  it CAN indicate that the message payload has been encrypted
+  at the application layer.
+
+- **Version 1:**
+The payload SHOULD be encrypted using Waku v1 payload encryption specified in [26/WAKU-PAYLOAD](../../application/26/payload.md).
+This provides asymmetric and symmetric encryption.
+The key agreement is performed out of band.
+And provides an encrypted signature and padding for some form of unlinkability.
+
+- **Version 2:**
+The payload SHOULD be encoded according to [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md)).
+Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
+
+Any `version` value not included in this list is reserved for future specification.
+And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
+
+## Whisper/Waku v1 envelope compatibility
+
+Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
+
+- Whisper/Waku v1 `topic` field
+SHOULD be mapped to Waku v2 message's `content_topic` attribute.
+- Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
+
+Waku v2 implements a pub/sub messaging pattern over libp2p.
+This makes redundant some Whisper/Waku v1 envelope fields
+(e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
+
+## Deterministic message hashing
+
+In Protocol Buffers v3,
+the deterministic serialization is not canonical across the different implementations
+and languages.
+It is also unstable across different builds with schema changes due to unknown fields.
+
+To overcome this interoperability limitation,
+a Waku v2 message's hash MUST be computed following this schema:
+
+```js
+message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp))
+```
+
+If an optional attribute, such as `meta`, is absent,
+the concatenation of attributes SHOULD exclude it.
+This recommendation is made to ensure that the concatenation process proceeds smoothly
+when certain attributes are missing and to maintain backward compatibility.
+
+This hashing schema is deemed appropriate for use cases
+where a cross-implementation deterministic hash is needed,
+such as message deduplication and integrity validation.
+The collision probability offered by this hashing schema can be considered negligible.
+This is due to the deterministic concatenation order of the message attributes,
+coupled with using a SHA-2 (256-bit) hashing algorithm.
+
+### Test vectors
+
+Waku message hash computation (`meta` size of 12 bytes):
+
+```js
+pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
+message.payload = 0x010203045445535405060708
+message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
+message.meta = 0x73757065722d736563726574
+message.timestamp = 0x175789bfa23f8400
+
+message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05
+```
+
+Waku message hash computation (`meta` size of 64 bytes):
+
+```js
+pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
+message.payload = 0x010203045445535405060708
+message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
+message.meta = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f
+message.timestamp = 0x175789bfa23f8400
+
+message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27
+```
+
+Waku message hash computation (`meta` attribute not present):
+
+```js
+pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
+message.payload = 0x010203045445535405060708
+message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
+message.meta = <not-present>
+message.timestamp = 0x175789bfa23f8400
+
+message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8
+```
+
+Waku message hash computation (`payload` length 0):
+
+```js
+pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
+message.payload = []
+message.content_topic = "/waku/2/default-content/proto" (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f)
+message.meta = 0x73757065722d736563726574
+message.timestamp = 0x175789bfa23f8400
+
+message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4
+```
+
+## Security Considerations
+
+### Confidentiality, integrity, and authenticity
+
+The level of confidentiality, integrity, and
+authenticity of the Waku message payload is discretionary.
+Accordingly, the application layer shall utilize the encryption and
+signature schemes supported by Waku v2 to meet the application-specific privacy needs.
+
+### Reliability of the `timestamp` attribute
+
+The Waku message `timestamp` attribute is set by the sender.
+Therefore, because message timestamps aren’t independently verified,
+this attribute is prone to exploitation and misuse.
+It should not solely be relied upon for operations such as message ordering.
+For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message
+to a high value so that it always shows up as the most recent message
+in a chat application.
+Applications using Waku messages’ `timestamp` attribute
+are recommended to use additional methods for more robust message ordering.
+An example of how to deal with message ordering against adversarial message timestamps
+can be found in the Status protocol,
+see [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering).
+
+### Reliability of the `ephemeral` attribute
+
+The Waku message `ephemeral` attribute is set by the sender.
+Since there is currently no incentive mechanism
+for network participants to behave correctly,
+this attribute is inherently insecure.
+A malicious actor can tamper with the value of a Waku message’s `ephemeral` attribute,
+and the receiver would not be able to verify the integrity of the message.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+- [6/WAKU1](../../legacy/6/waku1.md)
+- [Google Protocol buffers v3](https://developers.google.com/protocol-buffers/)
+- [26/WAKU-PAYLOAD](../../application/26/payload.md)
+- [35/WAKU2-NOISE]([/spec/35](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/noise.md))
+- [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering)

waku/standards/core/15/bridge.md

@@ -1,67 +1,68 @@
----
-slug: 15
-title: 15/WAKU-BRIDGE
-name: Waku Bridge
-status: draft
-tags: waku-core
-editor: Hanno Cornelius <hanno@status.im>
----
-
-A bridge between Waku v1 and Waku v2.
-
-## Bridge
-
-A bridge requires supporting both Waku versions:
-
-* Waku v1 - using devp2p RLPx protocol
-* Waku v2 - using libp2p protocols
-
-Packets received on the Waku v1 network SHOULD be published just once on the
-Waku v2 network. More specifically, the bridge SHOULD publish
-this through the Waku Relay (PubSub domain).
-
-Publishing such packet will require the creation of a new `Message` with a
-new `WakuMessage` as data field. The `data` and `topic` field from the Waku v1
-`Envelope` MUST be copied to the `payload` and `contentTopic` fields of the
-`WakuMessage`. Other fields such as nonce, expiry and ttl will be dropped as
-they become obsolete in Waku v2.
-
-Before this is done, the usual envelope verification still applies:
-
-* Expiry & future time verification
-* PoW verification
-* Size verification
-
-Bridging SHOULD occur through the `WakuRelay`, but it MAY also be done on other Waku
-v2 protocols (e.g. `WakuFilter`). The latter is however not advised as it will
-increase the complexity of the bridge and because of the
-[Security Considerations](#security-considerations) explained further below.
-
-Packets received on the Waku v2 network SHOULD be posted just once on the Waku
-v1 network. The Waku v2 `WakuMessage` contains only the `payload` and
-`contentTopic` fields. The bridge MUST create a new Waku v1 `Envelope` and
-copy over the `payload` and `contentFilter` fields to the `data` and `topic`
-fields. Next, before posting on the network, the bridge MUST set a new expiry
-and ttl and do the PoW nonce calculation.
-
-### Security Considerations
-As mentioned above, a bridge will be posting new Waku v1 envelopes, which
-requires doing the PoW nonce calculation.
-
-This could be a DoS attack vector, as the PoW calculation will make it more
-expensive to post the message compared to the original publishing on the Waku v2
-network. Low PoW setting will lower this problem, but it is likely that it is
-still more expensive.
-
-For this reason, bridges SHOULD probably be run independently of other nodes, so
-that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
-traffic.
-
-Bridging functionality SHOULD also be carefully implemented so that messages do
-not bounce back and forth between the two networks. The bridge SHOULD properly
-track messages with a seen filter so that no amplification can be achieved here.
-
-## Copyright
-
-Copyright and related rights waived via
-[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 15
+title: 15/WAKU-BRIDGE
+name: Waku Bridge
+status: draft
+tags: waku-core
+editor: Hanno Cornelius <hanno@status.im>
+---
+
+A bridge between Waku v1 and Waku v2.
+
+## Bridge
+
+A bridge requires supporting both Waku versions:
+
+* Waku v1 - using devp2p RLPx protocol
+* Waku v2 - using libp2p protocols
+
+Packets received on the Waku v1 network SHOULD be published just once on the
+Waku v2 network. More specifically, the bridge SHOULD publish
+this through the Waku Relay (PubSub domain).
+
+Publishing such packet will require the creation of a new `Message` with a
+new `WakuMessage` as data field. The `data` and `topic` field from the Waku v1
+`Envelope` MUST be copied to the `payload` and `contentTopic` fields of the
+`WakuMessage`. Other fields such as nonce, expiry and ttl will be dropped as
+they become obsolete in Waku v2.
+
+Before this is done, the usual envelope verification still applies:
+
+* Expiry & future time verification
+* PoW verification
+* Size verification
+
+Bridging SHOULD occur through the `WakuRelay`, but it MAY also be done on other Waku
+v2 protocols (e.g. `WakuFilter`). The latter is however not advised as it will
+increase the complexity of the bridge and because of the
+[Security Considerations](#security-considerations) explained further below.
+
+Packets received on the Waku v2 network SHOULD be posted just once on the Waku
+v1 network. The Waku v2 `WakuMessage` contains only the `payload` and
+`contentTopic` fields. The bridge MUST create a new Waku v1 `Envelope` and
+copy over the `payload` and `contentFilter` fields to the `data` and `topic`
+fields. Next, before posting on the network, the bridge MUST set a new expiry
+and ttl and do the PoW nonce calculation.
+
+### Security Considerations
+
+As mentioned above, a bridge will be posting new Waku v1 envelopes, which
+requires doing the PoW nonce calculation.
+
+This could be a DoS attack vector, as the PoW calculation will make it more
+expensive to post the message compared to the original publishing on the Waku v2
+network. Low PoW setting will lower this problem, but it is likely that it is
+still more expensive.
+
+For this reason, bridges SHOULD probably be run independently of other nodes, so
+that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
+traffic.
+
+Bridging functionality SHOULD also be carefully implemented so that messages do
+not bounce back and forth between the two networks. The bridge SHOULD properly
+track messages with a seen filter so that no amplification can be achieved here.
+
+## Copyright
+
+Copyright and related rights waived via
+[CC0](https://creativecommons.org/publicdomain/zero/1.0/).

waku/standards/core/17/images/rln-message-verification.msc

@@ -1,43 +1,43 @@
-# Sequence diagram for RLN Relay protocol (publishing,routing, and slashing)
-msc {
-  hscale="1",
-  wordwraparcs=true;
-
-  a [label=" "],
-  b [label=" "],
-  c [label=" "],
-  d [label=" "],
-  e [label=" "];
-
-  a rbox a [label="Relay Node: Publisher"],
-  b rbox b [label="Relay Node: Router"],
-  c rbox c [label="Relay Node"],
-  d rbox d [label="Relay Node"],
-  e note e [label="Membership Contract"];
-  |||;
-  b box b [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n  Initialize an empty map of the received nullifiers \n "],
-  c box c [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "],
-  d box d [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "];
-  |||;
-  ...,
-  a -> a [label="Keep track of epoch"],
-  b -> b [label="Keep track of epoch"],
-  c -> c [label="Keep track of epoch"],
-  d -> d [label="Keep track of epoch"];
-  a box a [label=" \n Message: the intended message \n \n epoch: the current epoch \n "];
-  a box a [label=" \n  A(x) = sk + H(sk, epoch)x \n \n shareX = H(message), shareY = A(shareX) \n \n nullifier = H(H(sk,epoch)) \n "];
-  a box a [label="  \n  zkProof: generate the proof using zkSNARK \n "];
-  |||;
-  a => b [label="Message, epoch, proofBundle:(shareX, shareY, nullifier, zkProof) \n "];
-  b box b [label="1. If the received epoch is far from the current epoch"];
-  b -x c [label="Do not relay"];
-  b box b [label=" \n 2. If verification of zkProof failed \n "];
-  b -x c [label="Do not relay"];
-  b box b [label=" \n 3. If identical nullifier exists in the nullifierMap, \n \n extract the publisher sk \n "];
-  b -x c [label="Do not relay"];
-  b => e [label="Slash the publisher: Unlbock the deposit associated with sk"];
-  e => b [label="x ETH"];
-  b box b [label=" \n 4. If none of  1-3 happens, update the nullifierMap \n "];
-  b => c [label="Relay"];
-  b => d [label="Relay"];
-}
+# Sequence diagram for RLN Relay protocol (publishing,routing, and slashing)
+msc {
+  hscale="1",
+  wordwraparcs=true;
+
+  a [label=" "],
+  b [label=" "],
+  c [label=" "],
+  d [label=" "],
+  e [label=" "];
+
+  a rbox a [label="Relay Node: Publisher"],
+  b rbox b [label="Relay Node: Router"],
+  c rbox c [label="Relay Node"],
+  d rbox d [label="Relay Node"],
+  e note e [label="Membership Contract"];
+  |||;
+  b box b [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n  Initialize an empty map of the received nullifiers \n "],
+  c box c [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "],
+  d box d [label=" \n nullifierMap= [(nullifier, shareX, shareY)...] \n \n Initialize an empty map of the received nullifiers \n "];
+  |||;
+  ...,
+  a -> a [label="Keep track of epoch"],
+  b -> b [label="Keep track of epoch"],
+  c -> c [label="Keep track of epoch"],
+  d -> d [label="Keep track of epoch"];
+  a box a [label=" \n Message: the intended message \n \n epoch: the current epoch \n "];
+  a box a [label=" \n  A(x) = sk + H(sk, epoch)x \n \n shareX = H(message), shareY = A(shareX) \n \n nullifier = H(H(sk,epoch)) \n "];
+  a box a [label="  \n  zkProof: generate the proof using zkSNARK \n "];
+  |||;
+  a => b [label="Message, epoch, proofBundle:(shareX, shareY, nullifier, zkProof) \n "];
+  b box b [label="1. If the received epoch is far from the current epoch"];
+  b -x c [label="Do not relay"];
+  b box b [label=" \n 2. If verification of zkProof failed \n "];
+  b -x c [label="Do not relay"];
+  b box b [label=" \n 3. If identical nullifier exists in the nullifierMap, \n \n extract the publisher sk \n "];
+  b -x c [label="Do not relay"];
+  b => e [label="Slash the publisher: Unlbock the deposit associated with sk"];
+  e => b [label="x ETH"];
+  b box b [label=" \n 4. If none of  1-3 happens, update the nullifierMap \n "];
+  b => c [label="Relay"];
+  b => d [label="Relay"];
+}

waku/standards/core/17/images/rln-relay-reg-with-tree-on-chain.msc

@@ -1,33 +1,33 @@
-# Sequence diagram for RLN Relay protocol (registration)
-msc {
-  hscale = "2";
-
-  d [label = " "], a [label = " "],b [label = " "];
-  a rbox a [label="Relay Node A"], 
-  b note b [label="Membership Contract"],
-  d rbox d [label = "Relay Node B"];
- 
- b abox b [ label=" \n Listening to the membership contract \n "] ;
-
-  a box a [ label=" \n Generate sk,pk \n "] ;
-  a=>b [ label = " \n Register(pk, x ETH) \n " ] ;
- a box a [ label=" \n Listening to the membership contract \n "] ;
-
-  b box b [label=" \n Insert pk as a leaf to the tree \n \n index: The index of the inserted leaf \n \n root: The updated tree root \n \n authPath: The authentication path \n  "];
-  |||;
-  b=>a [ label = "index, root, authPath"];
-  |||;
-  ...,  ---;
-  ...   [ label = "Other relay nodes register and the membership tree gets updatetd" ];
-  ...,  ---;
-  a=>b [ label = "getRoot()" ] ;
-  b box b [label="  \n  root: Get the current root\n  "];
-  b=>a [ label = "root"];
-
-  ...,  ---  [ label = "    " ];
-
-  a=>b [ label = "getAuthPath(index)" ] ;
-  b box b [label="  \n  authPath: Calculate the authentication path of the leaf with the given index and based on the current tree\n  "];
-
-  b=>a [ label = "authPath"];
-}
+# Sequence diagram for RLN Relay protocol (registration)
+msc {
+  hscale = "2";
+
+  d [label = " "], a [label = " "],b [label = " "];
+  a rbox a [label="Relay Node A"], 
+  b note b [label="Membership Contract"],
+  d rbox d [label = "Relay Node B"];
+ 
+ b abox b [ label=" \n Listening to the membership contract \n "] ;
+
+  a box a [ label=" \n Generate sk,pk \n "] ;
+  a=>b [ label = " \n Register(pk, x ETH) \n " ] ;
+ a box a [ label=" \n Listening to the membership contract \n "] ;
+
+  b box b [label=" \n Insert pk as a leaf to the tree \n \n index: The index of the inserted leaf \n \n root: The updated tree root \n \n authPath: The authentication path \n  "];
+  |||;
+  b=>a [ label = "index, root, authPath"];
+  |||;
+  ...,  ---;
+  ...   [ label = "Other relay nodes register and the membership tree gets updatetd" ];
+  ...,  ---;
+  a=>b [ label = "getRoot()" ] ;
+  b box b [label="  \n  root: Get the current root\n  "];
+  b=>a [ label = "root"];
+
+  ...,  ---  [ label = "    " ];
+
+  a=>b [ label = "getAuthPath(index)" ] ;
+  b box b [label="  \n  authPath: Calculate the authentication path of the leaf with the given index and based on the current tree\n  "];
+
+  b=>a [ label = "authPath"];
+}

waku/standards/core/17/images/rln-relay.msc

@@ -1,25 +1,25 @@
-# Sequence diagram for RLN Relay protocol (registration)
-msc {
-  hscale = "1.3";
-
-  d [label = " "], a [label = " "],b [label = " "];
-  a rbox a [label="Relay Node A"], 
-  b note b [label="Membership Contract"],
-  d rbox d [label = "Relay Node B"];
- 
-  |||;
-  d abox d [ label=" \n Listening to the membership contract \n "] ;
-
-  a box a [ label=" \n Generate sk,pk \n "] ;
-  a=>b [ label = "Register(pk, x ETH)" ] ;
-  a abox a [ label=" \n Listening to the membership contract \n "] ;
-
-  b box b [label=" \n Insert pk  to the list. \n Emit an event announcing the insertion of pk and its index in the list. \n  "];
-  |||;
-  ---;
-  b abox b [ label=" \n Block containing the insertion transaction is mined \n "] ;
-  b=>a [ label = "Insert(pk, index)"];
-  b=>d [ label = "Insert(pk, index)"];
-
-
-}
+# Sequence diagram for RLN Relay protocol (registration)
+msc {
+  hscale = "1.3";
+
+  d [label = " "], a [label = " "],b [label = " "];
+  a rbox a [label="Relay Node A"], 
+  b note b [label="Membership Contract"],
+  d rbox d [label = "Relay Node B"];
+ 
+  |||;
+  d abox d [ label=" \n Listening to the membership contract \n "] ;
+
+  a box a [ label=" \n Generate sk,pk \n "] ;
+  a=>b [ label = "Register(pk, x ETH)" ] ;
+  a abox a [ label=" \n Listening to the membership contract \n "] ;
+
+  b box b [label=" \n Insert pk  to the list. \n Emit an event announcing the insertion of pk and its index in the list. \n  "];
+  |||;
+  ---;
+  b abox b [ label=" \n Block containing the insertion transaction is mined \n "] ;
+  b=>a [ label = "Insert(pk, index)"];
+  b=>d [ label = "Insert(pk, index)"];
+
+
+}

waku/standards/core/17/rln-relay.md

@@ -1,247 +1,382 @@
----
-slug: 17
-title: 17/WAKU2-RLN-RELAY
-name: Waku v2 RLN Relay
-status: draft
-tags: waku-core
-editor: Sanaz Taheri <sanaz@status.im>
-contributors:
-  - Oskar Thorén <oskarth@titanproxy.com>
-  - Aaryamann Challani <aaryamann@status.im>
----
-
-The `17/WAKU2-RLN-RELAY` protocol is an extension of `11/WAKU2-RELAY` which additionally provides spam protection using [Rate Limiting Nullifiers (RLN)](../../../../vac/32/rln-v1.md). 
-
-The security objective is to contain spam activity in a GossipSub network by enforcing a global messaging rate to all the peers.
-Peers that violate the messaging rate are considered spammers and their message is considered spam.
-Spammers are also financially punished and removed from the system. 
-
-
-<!-- **Protocol identifier***: `/vac/waku/waku-rln-relay/2.0.0-alpha1` -->
-
-# Motivation
-
-In open and anonymous p2p messaging networks, one big problem is spam resistance. 
-Existing solutions, such as Whisper’s proof of work are computationally expensive hence not suitable for resource-limited nodes. 
-Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.
-
-We augment the [`11/WAKU2-RELAY`](/spec/11) protocol with a novel construct of [RLN](/spec/32) to enable an efficient economic spam prevention mechanism that can be run in resource-constrained environments.
-
-
-# Flow
-
-
-The messaging rate is defined by the `period` which indicates how many messages can be sent in a given period.
-We define an `epoch` as $\lceil$ `unix_time` / `period` $\rceil$. For example, if `unix_time` is `1644810116` and we set `period` to `30`, then `epoch` is  $\lceil$`(unix_time/period)`$\rceil$ `= 54827003`.
-Note that `epoch` refers to epoch in RLN and not Unix epoch. This means a message can only be sent every period, where period is up to the application.
-See see section [Recommended System Parameters](#recommended-system-parameters) for some recommended ways to set a sensible `period` value depending on the application.
-Peers subscribed to a spam-protected `pubsubTopic` are only allowed to send one message per `epoch`.
-The higher-level layers adopting `17/WAKU2-RLN-RELAY` MAY choose to enforce the messaging rate for  `WakuMessages` with a specific `contentTopic` published on a `pubsubTopic`.
-
-
-
-## Setup and Registration
-Peers subscribed to a specific `pubsubTopic` form a [RLN group](/spec/32).
-<!-- link to the RLN group definition in the RLN RFC -->
-Peers MUST be registered to the RLN group to be able to publish messages.
-Registration is moderated through a smart contract deployed on the Ethereum blockchain. 
-Each peer has an [RLN key pair](/spec/32) denoted by `sk`  and `pk`.
-The  secret key `sk` is secret data and MUST be persisted securely by the peer.
-The state of the membership contract contains the list of registered members' public identity keys i.e., `pk`s. 
-For the registration, a peer creates a transaction that invokes the registration function of the contract via which registers its `pk` in the group. 
-The transaction also transfers some amount of ether to the contract to be staked. 
-This amount is denoted by `staked_fund` and is a system parameter.
-The peer who has the secret key `sk` associated with a registered `pk` would be able to withdraw a portion `reward_portion` of the staked fund by providing valid proof. <!-- a secure way to prove the possession of a pk is yet under discussion, maybe via commit and reveal -->
-`reward_portion` is also a system parameter.
-
-Note that  `sk` is initially only known to its owning peer however, it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per `epoch`.
-An overview of registration is illustrated in Figure 1.
-
-![Figure 1: Registration.](./images/rln-relay.png)
-
-
-## Publishing
-
-To publish at a given `epoch`, the publishing peer proceeds based on the regular [`11/WAKU2-RELAY`](/spec/11) protocol.  
-However, to protect against spamming, each `WakuMessage` (which is wrapped inside the `data` field of a PubSub message) MUST carry a [`RateLimitProof`](##RateLimitProof) with the following fields. 
-Section [Payload](#payloads) covers the details about the type and encoding of these fields.
-
-The `merkle_root` contains the root of the Merkle tree.
-
-The `epoch` represents the current epoch.
-
-The `nullifier` is an internal nullifier acting as a fingerprint that allows specifying whether two messages are published by the same peer during the same `epoch`.
-The `nullifier` is a deterministic value derived from `sk` and `epoch` therefore any two messages issued by the same peer (i.e., using the same `sk`) for the same `epoch` are guaranteed to have identical `nullifier`s.
-
-The `share_x` and `share_y` can be seen as partial disclosure of peer's `sk` for the intended `epoch`.
-They are derived deterministically from peer's `sk` and current `epoch` using [Shamir secret sharing scheme](/spec/32). 
-If a peer discloses more than one such pair (`share_x`, `share_y`) for the same `epoch`, it would allow full disclosure of its  `sk` and hence get access to its staked fund in the membership contract.
-
-
-The `proof` field is a zero-knowledge proof signifying that: 
-1. The message owner is the current member of the group i.e., her/his identity commitment key `pk` is part of the membership group Merkle tree with the root `merkle_root`.
-2. `share_x` and `share_y`  are correctly computed.
-3. The `nullifier` is constructed correctly.
-For more details about the proof generation check [RLN](/spec/32)
-The proof generation relies on the knowledge of two pieces of private information i.e., `sk` and `authPath`.
-The `authPath` is a subset of Merkle tree nodes by which a peer can prove the inclusion of its `pk` in the group. <!-- TODO refer to RLN RFC for authPath def -->
-The proof generation also requires a set of public inputs which are: the Merkle tree root `merkle_root`, the current `epoch`, and the message for which the proof is going to be generated. 
-In `17/WAKU2-RLN-RELAY`, the message is the concatenation of `WakuMessage`'s  `payload` filed and its `contentTopic` i.e., `payload||contentTopic`. 
-
-## Group Synchronization
-
-Proof generation relies on the knowledge of Merkle tree root `merkle_root` and `authPath` which both require access to the membership Merkle tree. 
-Getting access to the Merkle tree can be done in various ways. 
-One way is that all the peers construct the tree locally.
-This can be done by listening to the registration and deletion events emitted by the membership contract.
-Peers MUST update the local Merkle tree on a per-block basis.
-This is discussed further in the [Merkle Root Validation](#merkle-root-validation) section.
-
-Another approach for synchronizing the state of slashed `pk`s is to disseminate such information through a p2p GossipSub network to which all peers are subscribed. 
-This is in addition to sending the deletion transaction to the membership contract.
-The benefit of an off-chain slashing is that it allows real-time removal of spammers as opposed to on-chain slashing in which peers get informed with a delay,
-where the delay is due to mining the slashing transaction.
-For the group synchronization, one important security consideration is that peers MUST make sure they always use the most recent Merkle tree root in their proof generation.
-The reason is that using an old root can allow inference about the index of the user's `pk` in the membership tree hence compromising user privacy and breaking message unlinkability.
-
-## Routing
-
-Upon the receipt of a PubSub message via [`11/WAKU2-RELAY`](/spec/11) protocol, the routing peer parses the `data` field as a `WakuMessage` and gets access to the `RateLimitProof` field.  
-The peer then validates the `RateLimitProof`  as explained next.
-
-### Epoch Validation
-If the `epoch` attached to the message is more than `max_epoch_gap` apart from the routing peer's current `epoch` then the message is discarded and considered invalid.
-This is to prevent a newly registered peer from spamming the system by messaging for all the past epochs. 
-`max_epoch_gap` is a system parameter for which we provide some recommendations in section [Recommended System Parameters](#recommended-system-parameters).
-
-### Merkle Root Validation
-The routing peers MUST check whether the provided Merkle root in the `RateLimitProof` is valid.
-It can do so by maintaining a local set of valid Merkle roots, which consist of `acceptable_root_window_size` past roots.
-These roots refer to the final state of the Merkle tree after a whole block consisting of group changes is processed.
-The Merkle roots are updated on a per-block basis instead of a per-event basis.
-This is done because if Merkle roots are updated on a per-event basis, some peers could send messages with a root that refers to a Merkle tree state that might get invalidated while the message is still propagating in the network, due to many registrations happening during this time frame.
-By updating roots on a per-block basis instead, we will have only one root update per-block processed, regardless on how many registrations happened in a block, and peers will be able to successfully propagate messages in a time frame corresponding to roughly the size of the roots window times the block mining time. 
-
-Atomic processing of the blocks are necessary so that even if the peer is unable to process one event, the previous roots remain valid, and can be used to generate valid RateLimitProof's.
-
-This also allows peers which are not well connected to the network to be able to send messages, accounting for network delay.
-This network delay is related to the nature of asynchronous network conditions, which means that peers see membership changes asynchronously, and therefore may have differing local Merkle trees.
-See [Recommended System Parameters](#recommended-system-parameters) on choosing an appropriate `acceptable_root_window_size`. 
-
-### Proof Verification
-The routing peers MUST check whether the zero-knowledge proof `proof` is valid.
-It does so by running the zk verification algorithm as explained in [RLN](/spec/32). 
-If `proof` is invalid then the message is discarded. 
-
-### Spam detection
-To enable local spam detection and slashing, routing peers MUST record the `nullifier`, `share_x`, and `share_y` of incoming messages which are not discarded i.e., not found spam or with invalid proof or epoch.
-To spot spam messages, the peer checks whether a message with an identical `nullifier` has already been relayed. 
-1. If such a message exists and its `share_x` and `share_y` components are different from the incoming message, then slashing takes place.
-That is, the peer uses the  `share_x` and `share_y`  of the new message and the  `share'_x` and `share'_y` of the old record to reconstruct the `sk` of the message owner.
-The `sk` then can be used to delete the spammer from the group and withdraw a portion `reward_portion` of its staked fund.
-2. If the `share_x` and `share_y` fields of the previously relayed message are identical to the incoming message, then the message is a duplicate and shall be discarded.
-3. If none is found, then the message gets relayed.
-
-An overview of the routing procedure and slashing is provided in Figure 2.
-
-
-<!-- TODO: may consider [validator functions](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation) or [extended validators](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for the spam detection -->
-
-![Figure 2: Publishing, Routing and Slashing workflow.](./images/rln-message-verification.png)
-
--------
-
-# Payloads
-
-Payloads are protobuf messages implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
-Nodes MAY extend the  [14/WAKU2-MESSAGE](/spec/14) with a `rate_limit_proof` field to indicate that their message is not spam.
-
-```diff 
-
-syntax = "proto3";
-
-message RateLimitProof {
-  bytes proof = 1;
-  bytes merkle_root = 2;
-  bytes epoch = 3;
-  bytes share_x = 4;
-  bytes share_y = 5;
-  bytes nullifier = 6;
-}
-
-message WakuMessage {
-  bytes payload = 1;
-  string content_topic = 2;
-  optional uint32 version = 3;
-  optional sint64 timestamp = 10;
-  optional bool ephemeral = 31;
-+  optional bytes rate_limit_proof = 21;
-}
-
-```
-## WakuMessage
-
-`rate_limit_proof` holds the information required to prove that the message owner has not exceeded the message rate limit.
- 
-## RateLimitProof
-Below is the description of the fields of `RateLimitProof` and their types.
-
-| Parameter | Type | Description | 
-| ----: | ----------- | ----------- |
-| `proof` | array of 256 bytes | the zkSNARK proof as explained in the [Publishing process](##Publishing) |
-| `merkle_root` | array of 32 bytes in little-endian order | the root of membership group Merkle tree at the time of publishing the message |
-| `share_x` and `share_y`| array of 32 bytes each | Shamir secret shares of the user's secret identity key `sk` . `share_x` is the Poseidon hash of the `WakuMessage`'s `payload` concatenated with its `contentTopic` . `share_y` is calculated using [Shamir secret sharing scheme](/spec/32) | <!-- todo specify the poseidon hash setting -->
-| `nullifier`  | array of 32 bytes | internal nullifier derived from `epoch` and peer's `sk` as explained in [RLN construct](/spec/32)|
-
-
-# Recommended System Parameters
-The system parameters are summarized in the following table, and the recommended values for a subset of them are presented next.
-
-| Parameter | Description |
-| ----: |----------- |
-|  `period`  | the length of `epoch` in seconds |
-| `staked_fund` | the amount of wei to be staked by peers at the registration |
-| `reward_portion` | the percentage of `staked_fund` to be rewarded to the slashers |
-| `max_epoch_gap` | the maximum allowed gap between the `epoch` of a routing peer and the incoming message |
-| `acceptable_root_window_size` | The maximum number of past Merkle roots to store |
-
-## Epoch Length
-A sensible value for the `period` depends on the application for which the spam protection is going to be used.
-For example, while the `period` of `1` second i.e., messaging rate of `1` per second, might be acceptable for a chat application, might be too low for communication among Ethereum network validators.
-One should look at the desired throughput of the application to decide on a proper `period` value.
-In the proof of concept implementation of `17/WAKU2-RLN-RELAY` protocol which is available in [nim-waku](https://github.com/status-im/nim-waku), the `period` is set to `1` second.
-Nevertheless, this value is also subject to change depending on user experience.
-
-## Maximum Epoch Gap
-We discussed in the [Routing](#routing) section that the gap between the epoch observed by the routing peer and the one attached to the incoming message should not exceed a threshold denoted by  `max_epoch_gap` .
-The value of  `max_epoch_gap`  can be measured based on the following factors.
-- Network transmission delay `Network_Delay`: the maximum time that it takes for a message to be fully disseminated in the GossipSub network.
-- Clock asynchrony `Clock_Asynchrony`: The maximum difference between the Unix epoch clocks perceived by network peers which can be due to clock drifts.
-  
-With a reasonable approximation of the preceding values, one can set  `max_epoch_gap`   as 
-`max_epoch_gap` $= \lceil \frac{\text{Network Delay} + \text{Clock Asynchrony}}{\text{Epoch Length}}\rceil$   where  `period`  is the length of the `epoch` in seconds.
-`Network_Delay` and `Clock_Asynchrony` MUST have the same resolution as  `period` .
-By this formulation,  `max_epoch_gap`  indeed measures the maximum number of `epoch`s that can elapse since a message gets routed from its origin to all the other peers in the network.
-
-`acceptable_root_window_size` depends upon the underlying chain's average blocktime, `block_time`
-
-The lower bound for the `acceptable_root_window_size` SHOULD be set as $acceptable_root_window_size=(Network_Delay)/block_time$
-
-`Network_Delay` MUST have the same resolution as `block_time`.
-
-By this formulation, `acceptable_root_window_size` will provide a lower bound of how many roots can be acceptable by a routing peer.
-
-The `acceptable_root_window_size` should indicate how many blocks may have been mined during the time it takes for a peer to receive a message. 
-This formula represents a lower bound of the number of acceptable roots. 
-
-
-# Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-# References
-
-1. [RLN documentation](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view)
-2. [Public inputs to the RLN circuit](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Public-Inputs)
-3. [Shamir secret sharing scheme used in RLN](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Linear-Equation-amp-SSS)
-4. [RLN internal nullifier](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Nullifiers)
+---
+slug: 17
+title: 17/WAKU2-RLN-RELAY
+name: Waku v2 RLN Relay
+status: draft
+tags: waku-core
+editor: Alvaro Revuelta <alvaro@status.im>
+contributors:
+  - Oskar Thorén <oskarth@titanproxy.com>
+  - Aaryamann Challani <p1ge0nh8er@proton.me>
+  - Sanaz Taheri <sanaz@status.im>
+  - Hanno Cornelius <hanno@status.im>
+---
+
+## Abstract
+
+This specification describes the `17/WAKU2-RLN-RELAY` protocol,
+which is an extension of [`11/WAKU2-RELAY`](../11/relay.md)
+to provide spam protection using [Rate Limiting Nullifiers (RLN)](../../../../vac/32/rln-v1.md).
+
+The security objective is to contain spam activity in the (64/WAKU-NETWORK)[]
+by enforcing a global messaging rate to all the peers.
+Peers that violate the messaging rate are considered spammers and
+their message is considered spam.
+Spammers are also financially punished and removed from the system.
+
+## Motivation
+
+In open and anonymous p2p messaging networks,
+one big problem is spam resistance.
+Existing solutions, such as Whisper’s proof of work,
+are computationally expensive hence not suitable for resource-limited nodes.
+Other reputation-based approaches might not be desirable,
+due to issues around arbitrary exclusion and privacy.
+
+We augment the [`11/WAKU2-RELAY`](../11/relay.md) protocol
+with a novel construct of [RLN](../../../../vac/32/rln-v1.md)
+to enable an efficient economic spam prevention mechanism
+that can be run in resource-constrained environments.
+
+## Specification
+
+The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”,
+“SHOULD NOT”, “RECOMMENDED”, “MAY”, and
+“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
+
+### Flow
+
+The messaging rate is defined by the `period`
+which indicates how many messages can be sent in a given period.
+We define an `epoch` as $\lceil$ `unix_time` / `period` $\rceil$.
+For example, if `unix_time` is `1644810116` and we set `period` to `30`,
+then `epoch` is  $\lceil$ `(unix_time/period)` $\rceil$ `= 54827003`.
+
+> **NOTE:** The `epoch` refers to the epoch in RLN and not Unix epoch.
+This means a message can only be sent every period,
+where the `period` is up to the application.
+
+See section [Recommended System Parameters](#recommended-system-parameters)
+for the RECOMMENDED method to set a sensible `period` value depending on the application.
+Peers subscribed to a spam-protected `pubsubTopic`
+are only allowed to send one message per `epoch`.
+The higher-level layers adopting `17/WAKU2-RLN-RELAY`
+MAY choose to enforce the messaging rate for `WakuMessages`
+with a specific `contentTopic` published on a `pubsubTopic`.
+
+#### Setup and Registration
+
+A `pubsubTopic` that is spam-protected requires subscribed peers to form a [RLN group](../../../../vac/32/rln-v1.md).
+
+- Peers MUST be registered to the RLN group to be able to publish messages.
+- Registration MAY be moderated through a smart contract
+deployed on the Ethereum blockchain.
+
+Each peer has an [RLN key pair](../../../../vac/32/rln-v1.md) denoted by `sk`
+and `pk`.
+
+- The secret key `sk` is secret data and MUST be persisted securely by the peer.
+- The state of the membership contract
+SHOULD contain a list of all registered members' public identity keys i.e., `pk`s.
+
+For registration,
+a peer MUST create a transaction to invoke the registration function on the contract,
+which registers its `pk` in the RLN group.
+
+- The transaction MUST transfer additional tokens to the contract to be staked.
+This amount is denoted by `staked_fund` and is a system parameter.
+The peer who has the secret key `sk` associated with a registered `pk`
+would be able to withdraw a portion `reward_portion`
+of the staked fund by providing valid proof.
+
+`reward_portion` is also a system parameter.
+
+> **NOTE:** Initially `sk` is only known to its owning peer however,
+it may get exposed to other peers in case the owner attempts spamming the system
+i.e., sending more than one message per `epoch`.
+
+An overview of registration is illustrated in Figure 1.
+
+![Figure 1: Registration.](./images/rln-relay.png)
+
+#### Publishing
+
+To publish at a given `epoch`, the publishing peer proceeds
+based on the regular [`11/WAKU2-RELAY`](../11/relay.md) protocol.
+However, to protect against spamming, each `WakuMessage`
+(which is wrapped inside the `data` field of a PubSub message)
+MUST carry a [`RateLimitProof`](#ratelimitproof) with the following fields.
+Section [Payload](#payloads) covers the details about the type and
+encoding of these fields.
+
+- The `merkle_root` contains the root of the Merkle tree.
+- The `epoch` represents the current epoch.
+- The `nullifier` is an internal nullifier acting as a fingerprint
+that allows specifying whether two messages are published by the same peer
+during the same `epoch`.
+- The `nullifier` is a deterministic value derived from `sk` and
+`epoch` therefore any two messages issued by the same peer
+(i.e., using the same `sk`)
+for the same `epoch` are guaranteed to have identical `nullifier`s.
+- The `share_x` and
+`share_y` can be seen as partial disclosure of peer's `sk` for the intended `epoch`.
+They are derived deterministically from peer's `sk` and
+current `epoch` using [Shamir secret sharing scheme](../../../../vac/32/rln-v1.md).
+
+If a peer discloses more than one such pair (`share_x`, `share_y`) for the same `epoch`,
+it would allow full disclosure of its  `sk` and
+hence get access to its staked fund in the membership contract.
+
+- The `proof` field is a zero-knowledge proof signifying that:
+
+1. The message owner is the current member of the group i.e.,
+the peer's identity commitment key, `pk`,
+is part of the membership group Merkle tree with the root `merkle_root`.
+2. `share_x` and `share_y` are correctly computed.
+3. The `nullifier` is constructed correctly.
+For more details about the proof generation check [RLN](../../../../vac/32/rln-v1.md)
+The proof generation relies on the knowledge of two pieces of private information
+i.e., `sk` and `authPath`.
+The `authPath` is a subset of Merkle tree nodes
+by which a peer can prove the inclusion of its `pk` in the group.
+<!-- TODO refer to RLN RFC for authPath def -->
+The proof generation also requires a set of public inputs which are:
+the Merkle tree root `merkle_root`, the current `epoch`, and
+the message for which the proof is going to be generated.
+In `17/WAKU2-RLN-RELAY`,
+the message is the concatenation of `WakuMessage`'s  `payload` filed and
+its `contentTopic` i.e., `payload||contentTopic`.
+
+#### Group Synchronization
+
+Proof generation relies on the knowledge of Merkle tree root `merkle_root` and
+`authPath` which both require access to the membership Merkle tree.
+Getting access to the Merkle tree can be done in various ways:
+
+1. Peers construct the tree locally.
+This can be done by listening to the registration and
+deletion events emitted by the membership contract.
+Peers MUST update the local Merkle tree on a per-block basis.
+This is discussed further
+in the [Merkle Root Validation](#merkle-root-validation) section.
+
+2. For synchronizing the state of slashed `pk`s,
+disseminate such information through a `pubsubTopic` to which all peers are subscribed.
+A deletion transaction SHOULD occur on the membership contract.
+The benefit of an off-chain slashing
+is that it allows real-time removal of spammers as opposed to on-chain slashing
+in which peers get informed with a delay,
+where the delay is due to mining the slashing transaction.
+
+For the group synchronization,
+one important security consideration is that peers MUST make sure they always use
+the most recent Merkle tree root in their proof generation.
+The reason is that using an old root can allow inference
+about the index of the user's `pk` in the membership tree
+hence compromising user privacy and breaking message unlinkability.
+
+#### Routing
+
+Upon the receipt of a PubSub message via [`11/WAKU2-RELAY`](../11/relay.md) protocol,
+the routing peer parses the `data` field as a `WakuMessage` and
+gets access to the `RateLimitProof` field.  
+The peer then validates the `RateLimitProof`  as explained next.
+
+##### Epoch Validation
+
+If the `epoch` attached to the `WakuMessage` is more than `max_epoch_gap`,
+apart from the routing peer's current `epoch`,
+then the `WakuMessage` MUST be discarded and considered invalid.
+This is to prevent a newly registered peer from spamming the system
+by messaging for all the past epochs.
+`max_epoch_gap` is a system parameter
+for which we provide some recommendations in section [Recommended System Parameters](#recommended-system-parameters).
+
+##### Merkle Root Validation
+
+The routing peers MUST check whether the provided Merkle root
+in the `RateLimitProof` is valid.
+It can do so by maintaining a local set of valid Merkle roots,
+which consist of `acceptable_root_window_size` past roots.
+These roots refer to the final state of the Merkle tree
+after a whole block consisting of group changes is processed.
+The Merkle roots are updated on a per-block basis instead of a per-event basis.
+This is done because if Merkle roots are updated on a per-event basis,
+some peers could send messages with a root that refers to a Merkle tree state
+that might get invalidated while the message is still propagating in the network,
+due to many registrations happening during this time frame.
+By updating roots on a per-block basis instead,
+we will have only one root update per-block processed,
+regardless on how many registrations happened in a block, and
+peers will be able to successfully propagate messages in a time frame
+corresponding to roughly the size of the roots window times the block mining time.
+
+Atomic processing of the blocks are necessary
+so that even if the peer is unable to process one event,
+the previous roots remain valid, and can be used to generate valid RateLimitProof's.
+
+This also allows peers which are not well connected to the network
+to be able to send messages, accounting for network delay.
+This network delay is related to the nature of asynchronous network conditions,
+which means that peers see membership changes asynchronously, and
+therefore may have differing local Merkle trees.
+See [Recommended System Parameters](#recommended-system-parameters)
+on choosing an appropriate `acceptable_root_window_size`.
+
+##### Proof Verification
+
+The routing peers MUST check whether the zero-knowledge proof `proof` is valid.
+It does so by running the zk verification algorithm as explained in [RLN](../../../../vac/32/rln-v1.md).
+If `proof` is invalid then the message MUST be discarded.
+
+##### Spam detection
+
+To enable local spam detection and slashing,
+routing peers MUST record the `nullifier`, `share_x`, and `share_y`
+of incoming messages which are not discarded i.e.,
+not found spam or with invalid proof or epoch.
+To spot spam messages, the peer checks whether a message
+with an identical `nullifier` has already been relayed.
+
+1. If such a message exists and its `share_x` and `share_y`
+components are different from the incoming message, then slashing takes place.
+That is, the peer uses the  `share_x` and `share_y`
+of the new message and the `share'_x` and `share'_y`
+of the old record to reconstruct the `sk` of the message owner.
+The `sk` then MAY be used to delete the spammer from the group and
+withdraw a portion `reward_portion` of its staked funds.
+2. If the `share_x` and
+`share_y` fields of the previously relayed message are identical
+to the incoming message,
+then the message is a duplicate and MUST be discarded.
+3. If none is found, then the message gets relayed.
+
+An overview of the routing procedure and slashing is provided in Figure 2.
+
+![Figure 2: Publishing, Routing and Slashing workflow.](./images/rln-message-verification.png)
+
+-------
+
+### Payloads
+
+Payloads are protobuf messages implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
+Nodes MAY extend the [14/WAKU2-MESSAGE](../14/message.md)
+with a `rate_limit_proof` field to indicate that their message is not spam.
+
+```diff
+
+syntax = "proto3";
+
+message RateLimitProof {
+  bytes proof = 1;
+  bytes merkle_root = 2;
+  bytes epoch = 3;
+  bytes share_x = 4;
+  bytes share_y = 5;
+  bytes nullifier = 6;
+}
+
+message WakuMessage {
+  bytes payload = 1;
+  string content_topic = 2;
+  optional uint32 version = 3;
+  optional sint64 timestamp = 10;
+  optional bool ephemeral = 31;
+  RateLimitProof rate_limit_proof = 21;
+}
+
+```
+
+#### WakuMessage
+
+`rate_limit_proof` holds the information required to prove that the message owner
+has not exceeded the message rate limit.
+
+#### RateLimitProof
+
+Below is the description of the fields of `RateLimitProof` and their types.
+
+| Parameter | Type | Description |
+| ----: | ----------- | ----------- |
+| `proof` | array of 256 bytes uncompressed or 128 bytes compressed | the zkSNARK proof as explained in the [Publishing process](#publishing) |
+| `merkle_root` | array of 32 bytes in little-endian order | the root of membership group Merkle tree at the time of publishing the message |
+| `share_x` and `share_y`| array of 32 bytes each | Shamir secret shares of the user's secret identity key `sk` . `share_x` is the Poseidon hash of the `WakuMessage`'s `payload` concatenated with its `contentTopic` . `share_y` is calculated using [Shamir secret sharing scheme](../../../../vac/32/rln-v1.md) |
+| `nullifier`  | array of 32 bytes | internal nullifier derived from `epoch` and peer's `sk` as explained in [RLN construct](../../../../vac/32/rln-v1.md)|
+
+### Recommended System Parameters
+
+The system parameters are summarized in the following table,
+and the RECOMMENDED values for a subset of them are presented next.
+
+| Parameter | Description |
+| ----: |----------- |
+|  `period`  | the length of `epoch` in seconds |
+| `staked_fund` | the amount of funds to be staked by peers at the registration |
+| `reward_portion` | the percentage of `staked_fund` to be rewarded to the slashers |
+| `max_epoch_gap` | the maximum allowed gap between the `epoch` of a routing peer and the incoming message |
+| `acceptable_root_window_size` | The maximum number of past Merkle roots to store |
+
+#### Epoch Length
+
+A sensible value for the `period` depends on the application
+for which the spam protection is going to be used.
+For example, while the `period` of `1` second i.e.,
+messaging rate of `1` per second, might be acceptable for a chat application,
+might be too low for communication among Ethereum network validators.
+One should look at the desired throughput of the application
+to decide on a proper `period` value.
+
+#### Maximum Epoch Gap
+
+We discussed in the [Routing](#routing) section that the gap between the epoch
+observed by the routing peer and
+the one attached to the incoming message
+should not exceed a threshold denoted by `max_epoch_gap`.
+The value of `max_epoch_gap` can be measured based on the following factors.
+
+- Network transmission delay `Network_Delay`:
+the maximum time that it takes for a message to be fully disseminated
+in the GossipSub network.
+- Clock asynchrony `Clock_Asynchrony`:
+The maximum difference between the Unix epoch clocks perceived
+by network peers which can be due to clock drifts.
+  
+With a reasonable approximation of the preceding values,
+one can set `max_epoch_gap` as
+
+`max_epoch_gap`
+$= \lceil \frac{\text{Network Delay} + \text{Clock Asynchrony}}{\text{Epoch Length}}\rceil$
+where  `period`  is the length of the `epoch` in seconds.
+`Network_Delay` and `Clock_Asynchrony` MUST have the same resolution as  `period`.
+By this formulation, `max_epoch_gap` indeed measures the maximum number of `epoch`s
+that can elapse since a message gets routed from its origin
+to all the other peers in the network.
+
+`acceptable_root_window_size` depends upon the underlying chain's average blocktime,
+`block_time`
+
+The lower bound for the `acceptable_root_window_size` SHOULD be set as $acceptable_root_window_size=(Network_Delay)/block_time$
+
+`Network_Delay` MUST have the same resolution as `block_time`.
+
+By this formulation,
+`acceptable_root_window_size` will provide a lower bound
+of how many roots can be acceptable by a routing peer.
+
+The `acceptable_root_window_size` should indicate how many blocks may have been mined
+during the time it takes for a peer to receive a message.
+This formula represents a lower bound of the number of acceptable roots.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+1. [`11/WAKU2-RELAY`](../11/relay.md)
+2. [RLN](../../../../vac/32/rln-v1.md)
+3. [14/WAKU2-MESSAGE](../14/message.md)
+4. [RLN documentation](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view)
+5. [Public inputs to the RLN circuit](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Public-Inputs)
+6. [Shamir secret sharing scheme used in RLN](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Linear-Equation-amp-SSS)
+7. [RLN internal nullifier](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Nullifiers)

waku/standards/core/19/lightpush.md

@@ -1,66 +1,75 @@
----
-slug: 19
-title: 19/WAKU2-LIGHTPUSH
-name: Waku v2 Light Push
-status: draft
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributors: 
-  - Daniel Kaiser <danielkaiser@status.im>
----
-
-**Protocol identifier**: `/vac/waku/lightpush/2.0.0-beta1`
-
-## Motivation and Goals
-
-Light nodes with short connection windows and limited bandwidth wish to publish messages into the Waku network.
-Additionally, there is sometimes a need for confirmation that a message has been received "by the network"
-(here, at least one node).
-
-`19/WAKU2-LIGHTPUSH` is a request/response protocol for this.
-
-## Payloads
-
-```protobuf
-syntax = "proto3";
-
-message PushRequest {
-    string pubsub_topic = 1;
-    WakuMessage message = 2;
-}
-
-message PushResponse {
-    bool is_success = 1;
-    // Error messages, etc
-    string info = 2;
-}
-
-message PushRPC {
-    string request_id = 1;
-    PushRequest request = 2;
-    PushResponse response = 3;
-}
-```
-
-### Message Relaying
-
-Nodes that respond to `PushRequests` MUST either
-relay the encapsulated message via [11/WAKU2-RELAY](../11/relay.md) protocol on the specified `pubsub_topic`,
-or forward the `PushRequest` via 19/LIGHTPUSH on a [44/WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md) stem.
-If they are unable to do so for some reason, they SHOULD return an error code in `PushResponse`.
-
-## Security Considerations
-
-Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions.
-This can be done by rate limiting via [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/).
-
-Note that the above is currently not fully implemented.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
-
-## References
-
-* [11/WAKU2-RELAY](../11/relay.md)
-* [44/WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md)
-* [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
+---
+slug: 19
+title: 19/WAKU2-LIGHTPUSH
+name: Waku v2 Light Push
+status: draft
+editor: Hanno Cornelius <hanno@status.im> 
+contributors: 
+  - Daniel Kaiser <danielkaiser@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+**Protocol identifier**: `/vac/waku/lightpush/2.0.0-beta1`
+
+## Motivation and Goals
+
+Light nodes with short connection windows and
+limited bandwidth wish to publish messages into the Waku network.
+Additionally,
+there is sometimes a need for confirmation
+that a message has been received "by the network"
+(here, at least one node).
+
+`19/WAKU2-LIGHTPUSH` is a request/response protocol for this.
+
+## Payloads
+
+```protobuf
+syntax = "proto3";
+
+message PushRequest {
+    string pubsub_topic = 1;
+    WakuMessage message = 2;
+}
+
+message PushResponse {
+    bool is_success = 1;
+    // Error messages, etc
+    string info = 2;
+}
+
+message PushRPC {
+    string request_id = 1;
+    PushRequest request = 2;
+    PushResponse response = 3;
+}
+```
+
+### Message Relaying
+
+Nodes that respond to `PushRequests` MUST either
+relay the encapsulated message via [11/WAKU2-RELAY](../11/relay.md) protocol
+on the specified `pubsub_topic`,
+or forward the `PushRequest` via 19/LIGHTPUSH on a [WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md)
+stem.
+If they are unable to do so for some reason,
+they SHOULD return an error code in `PushResponse`.
+
+## Security Considerations
+
+Since this can introduce an amplification factor,
+it is RECOMMENDED for the node relaying to the rest of the network
+to take extra precautions.
+This can be done by rate limiting via [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
+
+Note that the above is currently not fully implemented.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+* [11/WAKU2-RELAY](../11/relay.md)
+* [WAKU2-DANDELION](https://github.com/waku-org/specs/blob/waku-RFC/standards/application/dandelion.md)
+* [17/WAKU2-RLN-RELAY](../17/rln-relay.md)

waku/standards/core/33/discv5.md

@@ -1,176 +1,225 @@
----
-slug: 33
-title: 33/WAKU2-DISCV5
-name: Waku v2 Discv5 Ambient Peer Discovery
-status: draft
-editor: Daniel Kaiser <danielkaiser@status.im>
-contributors:
----
-
-## Abstract
-
-`33/WAKU2-DISCV5` specifies a modified version of [Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) as a means for ambient node discovery.
-[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network for establishing a decentralized network of interconnected Waku2 nodes.
-In its current version, the `33/WAKU2-DISCV5` discovery network is isolated from the Ethereum Discovery v5 network.
-Isolation improves discovery efficiency, which is especially significant with a low number of Waku nodes compared to the total number of Ethereum nodes.
-
-## Disclaimer
-
-This version of `33/WAKU2-DISCV5` has a focus on timely deployment of an efficient discovery method for [10/WAKU2](../10/waku2.md).
-Establishing a separate discovery network is in line with this focus.
-However, we are aware of potential resilience problems (see section on security considerations) and are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
-and researching hybrid approaches.
-
-
-## Background and Rationale
-
-[11/WAKU2-RELAY](../11/relay.md) assumes the existence of a network of Waku2 nodes.
-For establishing and growing this network, new nodes trying to join the Waku2 network need a means of discovering nodes within the network.
-[10/WAKU2](../10/waku2.md) supports the following discovery methods in order of increasing decentralization
-
-* hard coded bootstrap nodes
-* [`DNS discovery`](https://rfc.vac.dev/spec/10/#discovery-domain) (based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459))
-* `peer-exchange` (work in progress)
-* `33/WAKU2-DISCV5` (specified in this document)
-
-The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md) is discovering Waku2 nodes in a decentralized way.
-The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network, which allows avoiding hotspots and allows merging the network after a split.
-While the other methods provide either a fixed or local set of nodes, `33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
-Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.
-
-### Separate Discovery Network
-
-#### w.r.t. Waku2 Relay Network
-
-`33/WAKU2-DISCV5` spans an overlay network separate from the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) network [11/WAKU2-RELAY](../11/relay.md) builds on.
-Because it is a P2P network on its own, it also depends on bootstrap nodes.
-Having a separate discovery network reduces load on the bootstrap nodes, because the actual work is done by randomly discovered nodes.
-This also increases decentralization.
-
-
-#### w.r.t. Ethereum Discovery v5
-
-`33/WAKU2-DISCV5` spans a discovery network isolated from the Ethereum Discovery v5 network.
-
-Another simple solution would be taking part in the Ethereum Discovery network, and filtering Waku nodes based on whether they support [31/WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md).
-This solution is more resilient towards eclipse attacks.
-However, this discovery method is very inefficient for small percentages of Waku nodes (see [estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)).
-It boils down to random walk discovery and does not offer a O(log(n)) hop bound.
-The rarer the requested property (in this case Waku), the longer a random walk will take until finding an appropriate node, which leads to a needle-in-the-haystack problem.
-Using a dedicated Waku2 discovery network, nodes can query this discovery network for a random set of nodes
-and all (well-behaving) returned nodes can serve as bootstrap nodes for other Waku2 protocols.
-
-A more sophisticated solution would be using [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement).
-However, in its current state it also has efficiency problems for small percentages of Waku nodes and is still in the design phase ([see here](https://github.com/ethereum/devp2p/issues/199)).
-
-Currently, the Ethereum discv5 network is very efficient in finding other discv5 nodes,
-but it is not so efficient for finding discv5 nodes that have a specific property or offer specific services, e.g. Waku.
-
-As part of our [discv5 roadmap](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121), we consider two ideas for future versions of `33/WAKU2-DISCV5`
-
-* [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement) with adjustments (ideally upstream)
-* a hybrid solution that uses both a separate discv5 network and a Waku-ENR-filtered Ethereum discv5 network
-
-## Semantics
-
-`33/WAKU2-DISCV5` fully inherits the [discv5 semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
-
-Before announcing their address via Waku2 discv5, nodes SHOULD check if this address is publicly reachable.
-Nodes MAY use the [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md) to perform that check.
-Nodes SHOULD only announce publicly reachable addresses via Waku2 discv5,
-to avoid cluttering peer lists with nodes that are not reachable.
-
-## Wire Format Specification
-
-`33/WAKU2-DISCV5` inherits the [discv5 wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md) except for the following differences
-
-## WAKU2-Specific `protocol-id`
-
-Ethereum discv5:
-
-<pre>
-<code>
-header        = static-header || authdata
-static-header = protocol-id || version || flag || nonce || authdata-size
-protocol-id   = <b>"discv5"</b>
-version       = 0x0001
-authdata-size = uint16    -- byte length of authdata
-flag          = uint8     -- packet type identifier
-nonce         = uint96    -- nonce of message
-</code>
-</pre>
-
-`33/WAKU2-DISCV5`:
-
-<pre>
-kcode>
-header        = static-header || authdata
-static-header = protocol-id || version || flag || nonce || authdata-size
-protocol-id   = <b>"d5waku"</b>
-version       = 0x0001
-authdata-size = uint16    -- byte length of authdata
-flag          = uint8     -- packet type identifier
-nonce         = uint96    -- nonce of message
-</code>
-</pre>
-
-
-## Suggestions for Implementations
-
-Existing discv5 implementations
-
-* can be augmented to make the `protocol-id` selectable using a compile-time flag as in [this feature branch](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim#L34) of nim-eth/discv5.
-* can be forked followed by changing the `protocol-id` string as in [go-waku](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go#L135-L137).
-
-
-## Security Considerations
-
-### Sybil attack
-
-Implementations should limit the number of bucket entries that have the same network parameters (IP address / port) to mitigate Sybil attacks.
-
-### Eclipse attack
-
-Eclipse attacks aim to eclipse certain regions in a DHT.
-Malicious nodes provide false routing information for certain target regions.
-The larger the desired eclipsed region, the more resources (i.e. controlled nodes) the attacker needs.
-This introduces an efficiency versus resilience tradeoff.
-Discovery is more efficient if information about target objects (e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address.
-If nodes providing specific information are closer to each other, they cover a smaller range in the DHT and are easier to eclipse.
-
-Sybil attacks greatly increase the power of eclipse attacks, because they significantly reduce resources necessary to mount a successful eclipse attack.
-
-## Security Implications of a Separate Discovery Network
-
-A dedicated Waku discovery network is more likely to be subject to successful eclipse attacks (and to DoS attacks in general).
-This is because eclipsing in a smaller network requires less resources for the attacker.
-DoS attacks render the whole network unusable if the percentage of attacker nodes is sufficient.
-
-Using random walk discovery would mitigate eclipse attacks targeted at specific capabilities, e.g. Waku.
-However, this is because eclipse attacks aim at the DHT overlay structure, which is not used by random walks.
-So, this mitigation would come at the cost of giving up overlay routing efficiency.
-The efficiency loss is especially severe with a relatively small number of Waku nodes.
-
-Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.
-
-## References
-
-1. [10/WAKU2](../10/waku2.md)
-1. [11/WAKU2-RELAY](../11/relay.md)
-1. [`31/WAKU2-ENR`](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
-1. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) 
-1. [`discv5` semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
-1. [`discv5` wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md) 
-1. [`discv5` topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
-1. [Waku DNS discovery](https://rfc.vac.dev/spec/10/#discovery-domain)
-1. [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)
-1. [`EIP-1459`](https://eips.ethereum.org/EIPS/eip-1459)
-1. [`GossipSub`](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
-1. [Waku discv5 roadmap discussion](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121)
-1. [discovery efficiency estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
-1. [implementation: Nim](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim)
-1. [implementation: Go](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go)
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 33
+title: 33/WAKU2-DISCV5
+name: Waku v2 Discv5 Ambient Peer Discovery
+status: draft
+editor: Daniel Kaiser <danielkaiser@status.im>
+contributors:
+---
+
+## Abstract
+
+`33/WAKU2-DISCV5` specifies a modified version of
+[Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
+as a means for ambient node discovery.
+[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network
+for establishing a decentralized network of interconnected Waku2 nodes.
+In its current version,
+the `33/WAKU2-DISCV5` discovery network
+is isolated from the Ethereum Discovery v5 network.
+Isolation improves discovery efficiency,
+which is especially significant with a low number of Waku nodes
+compared to the total number of Ethereum nodes.
+
+## Disclaimer
+
+This version of `33/WAKU2-DISCV5` has a focus on timely deployment
+of an efficient discovery method for [10/WAKU2](../10/waku2.md).
+Establishing a separate discovery network is in line with this focus.
+However, we are aware of potential resilience problems
+(see section on security considerations) and
+are [discussing](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
+and researching hybrid approaches.
+
+## Background and Rationale
+
+[11/WAKU2-RELAY](../11/relay.md) assumes the existence of a network of Waku2 nodes.
+For establishing and growing this network,
+new nodes trying to join the Waku2 network need a means of discovering nodes
+within the network.
+[10/WAKU2](../10/waku2.md) supports the following discovery methods
+in order of increasing decentralization
+
+* hard coded bootstrap nodes
+* [`DNS discovery`](https://rfc.vac.dev/spec/10/#discovery-domain) (based on [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459))
+* `peer-exchange` (work in progress)
+* `33/WAKU2-DISCV5` (specified in this document)
+
+The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md)
+is discovering Waku2 nodes in a decentralized way.
+The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network,
+which allows avoiding hotspots and allows merging the network after a split.
+While the other methods provide either a fixed or local set of nodes,
+`33/WAKU2-DISCV5` can provide a random sample of Waku2 nodes.
+Future iterations of this document will add the possibility
+of efficiently discovering Waku2 nodes that have certain capabilities,
+e.g. holding messages of a certain time frame
+during which the querying node was offline.
+
+### Separate Discovery Network
+
+#### w.r.t. Waku2 Relay Network
+
+`33/WAKU2-DISCV5` spans an overlay network separate from the
+[GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+network [11/WAKU2-RELAY](../11/relay.md) builds on.
+Because it is a P2P network on its own, it also depends on bootstrap nodes.
+Having a separate discovery network reduces load on the bootstrap nodes,
+because the actual work is done by randomly discovered nodes.
+This also increases decentralization.
+
+#### w.r.t. Ethereum Discovery v5
+
+`33/WAKU2-DISCV5` spans a discovery network
+isolated from the Ethereum Discovery v5 network.
+
+Another simple solution would be taking part in the Ethereum Discovery network,
+and filtering Waku nodes based on whether they support [WAKU2-ENR](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md).
+This solution is more resilient towards eclipse attacks.
+However, this discovery method is very inefficient
+for small percentages of Waku nodes
+(see [estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)).
+It boils down to random walk discovery and does not offer a O(log(n)) hop bound.
+The rarer the requested property (in this case Waku),
+the longer a random walk will take until finding an appropriate node,
+which leads to a needle-in-the-haystack problem.
+Using a dedicated Waku2 discovery network,
+nodes can query this discovery network for a random set of nodes
+and all (well-behaving)
+returned nodes can serve as bootstrap nodes for other Waku2 protocols.
+
+A more sophisticated solution would be using [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement).
+However, in its current state it also has efficiency problems for small percentages
+of Waku nodes and is still in the design phase
+([see here](https://github.com/ethereum/devp2p/issues/199)).
+
+Currently,
+the Ethereum discv5 network is very efficient in finding other discv5 nodes,
+but it is not so efficient for finding discv5 nodes
+that have a specific property or
+offer specific services, e.g. Waku.
+
+As part of our [discv5 roadmap](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121),
+we consider two ideas for future versions of `33/WAKU2-DISCV5`
+
+* [Discv5 topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
+with adjustments (ideally upstream)
+* a hybrid solution that uses both a separate discv5 network and
+a Waku-ENR-filtered Ethereum discv5 network
+
+## Semantics
+
+`33/WAKU2-DISCV5` fully inherits the [discv5 semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
+
+Before announcing their address via Waku2 discv5,
+nodes SHOULD check if this address is publicly reachable.
+Nodes MAY use the [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)
+to perform that check.
+Nodes SHOULD only announce publicly reachable addresses via Waku2 discv5,
+to avoid cluttering peer lists with nodes that are not reachable.
+
+## Wire Format Specification
+
+`33/WAKU2-DISCV5` inherits the [discv5 wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
+except for the following differences
+
+## WAKU2-Specific `protocol-id`
+
+Ethereum discv5:
+
+```text
+header        = static-header || authdata
+static-header = protocol-id || version || flag || nonce || authdata-size
+protocol-id   = <b>"discv5"</b>
+version       = 0x0001
+authdata-size = uint16    -- byte length of authdata
+flag          = uint8     -- packet type identifier
+nonce         = uint96    -- nonce of message
+
+```
+
+`33/WAKU2-DISCV5`:
+
+```text
+kcode>
+header        = static-header || authdata
+static-header = protocol-id || version || flag || nonce || authdata-size
+protocol-id   = <b>"d5waku"</b>
+version       = 0x0001
+authdata-size = uint16    -- byte length of authdata
+flag          = uint8     -- packet type identifier
+nonce         = uint96    -- nonce of message
+
+```
+
+## Suggestions for Implementations
+
+Existing discv5 implementations
+
+* can be augmented to make the `protocol-id` selectable using a compile-time flag
+ as in [this feature branch](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim#L34)
+of nim-eth/discv5.
+* can be forked followed by changing the `protocol-id` string as in [go-waku](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go#L135-L137).
+
+## Security Considerations
+
+### Sybil attack
+
+Implementations should limit the number of bucket entries
+that have the same network parameters (IP address / port) to mitigate Sybil attacks.
+
+### Eclipse attack
+
+Eclipse attacks aim to eclipse certain regions in a DHT.
+Malicious nodes provide false routing information for certain target regions.
+The larger the desired eclipsed region,
+the more resources (i.e. controlled nodes) the attacker needs.
+This introduces an efficiency versus resilience tradeoff.
+Discovery is more efficient if information about target objects
+(e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address.
+If nodes providing specific information are closer to each other,
+they cover a smaller range in the DHT and are easier to eclipse.
+
+Sybil attacks greatly increase the power of eclipse attacks,
+because they significantly reduce resources necessary
+to mount a successful eclipse attack.
+
+## Security Implications of a Separate Discovery Network
+
+A dedicated Waku discovery network is more likely to be subject
+to successful eclipse attacks (and to DoS attacks in general).
+This is because eclipsing in a smaller network requires less resources for the attacker.
+DoS attacks render the whole network unusable
+if the percentage of attacker nodes is sufficient.
+
+Using random walk discovery would mitigate eclipse attacks
+targeted at specific capabilities, e.g. Waku.
+However, this is because eclipse attacks aim at the DHT overlay structure,
+which is not used by random walks.
+So, this mitigation would come at the cost of giving up overlay routing efficiency.
+The efficiency loss is especially severe with a relatively small number of Waku nodes.
+
+Properly protecting against eclipse attacks is challenging and
+raises research questions that we will address in future stages of our discv5 roadmap.
+
+## References
+
+1. [10/WAKU2](../10/waku2.md)
+1. [11/WAKU2-RELAY](../11/relay.md)
+1. [`WAKU2-ENR`](https://github.com/waku-org/specs/blob/waku-RFC/standards/core/enr.md)
+1. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
+1. [`discv5` semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
+1. [`discv5` wire protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)
+1. [`discv5` topic discovery](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement)
+1. [Waku DNS discovery](https://rfc.vac.dev/spec/10/#discovery-domain)
+1. [libp2p AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)
+1. [`EIP-1459`](https://eips.ethereum.org/EIPS/eip-1459)
+1. [`GossipSub`](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
+1. [Waku discv5 roadmap discussion](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121)
+1. [discovery efficiency estimation](https://forum.vac.dev/t/waku-v2-discv5-roadmap-discussion/121/8)
+1. [implementation: Nim](https://github.com/kaiserd/nim-eth/blob/add-selectable-protocol-id-static/eth/p2p/discoveryv5/encoding.nim)
+1. [implementation: Go](https://github.com/status-im/go-waku/blob/master/waku/v2/discv5/discover.go)
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

waku/standards/core/64/network.md

@@ -0,0 +1,385 @@
+---
+slug: 64
+title: 64/WAKU2-NETWORK
+name: Waku v2 Network
+status: draft
+category: Best Current Practice
+tags: waku/application
+editor: Hanno Cornelius <hanno@status.im>
+contributors:
+---
+
+## Abstract
+
+This specification describes an opinionated deployment of [10/WAKU2](../10/waku2.md)
+protocols to form a coherent and
+shared decentralized messaging network that is open-access,
+useful for generalized messaging, privacy-preserving, scalable and
+accessible even to resource-restricted devices.
+We'll refer to this opinionated deployment simply as
+_the public Waku Network_, _the Waku Network_ or, if the context is clear, _the network_
+in the rest of this document.
+All The Waku Network configuration parameters are listed [here](https://github.com/waku-org/nwaku/blob/8bfad3ab453f96ac545c7cb0af06d0c0f34d1356/waku/factory/networks_config.nim#L31).
+
+## Theory / Semantics
+
+### Routing protocol
+
+The Waku Network is built on the
+[17/WAKU2-RLN-RELAY](../17/rln-relay.md) routing protocol,
+which in turn is an extension of
+[11/WAKU2-RELAY](../11/relay.md) with spam protection measures.
+
+### Network shards
+
+Traffic in the Waku Network is sharded into eight
+[17/WAKU2-RLN-RELAY](../17/rln-relay.md) pubsub topics.
+Each pubsub topic is named according to the static shard naming format
+defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)
+with:
+
+* `<cluster_id>` set to `1`
+* `<shard_number>` occupying the range `0` to `7`.
+In other words, the Waku Network is a [17/WAKU2-RLN-RELAY](../17/rln-relay.md) network
+routed on the combination of the eight pubsub topics:
+
+```text
+/waku/2/rs/1/0
+/waku/2/rs/1/1
+...
+/waku/2/rs/1/7
+```
+
+A node MUST use [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
+protocol to identify the `<cluster_id>` that every
+inbound/outbound peer that attempts to connect supports.
+In any of the following cases, the node MUST trigger a disconnection:
+
+* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
+dial fails.
+* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
+reports an empty `<cluster_id>`.
+* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
+reports a `<cluster_id>` different than `1`.
+
+## Roles
+
+There are two distinct roles evident in the network, those of:
+
+1) nodes, and
+2) applications.
+
+### Nodes
+
+Nodes are the individual software units
+using [10/WAKU2](../10/waku2.md) protocols to form a p2p messaging network.
+Nodes, in turn, can participate in a shard as full relayers, i.e. _relay nodes_,
+or by running a combination of protocols suitable for resource-restricted environments,
+i.e. _non-relay nodes_.
+Nodes can also provide various services to the network,
+such as storing historical messages or protecting the network against spam.
+See the section on [default services](#default-services) for more.
+
+#### Relay nodes
+
+Relay nodes MUST follow [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
+to route messages to other nodes in the network
+for any of the pubsub topics [defined as the Waku Network shards](#network-shards).
+Relay nodes MAY choose to subscribe to any of these shards,
+but MUST be subscribed to at least one defined shard.
+Each relay node SHOULD be subscribed to as many shards as it has resources to support.
+If a relay node supports an encapsulating application,
+it SHOULD be subscribed to all the shards servicing that application.
+If resource restrictions prevent a relay node from servicing all shards
+used by the encapsulating application,
+it MAY choose to support some shards as a non-relay node.
+
+#### Bootstrapping and discovery
+
+Nodes MAY use any method to bootstrap connection to the network,
+but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect
+to using [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459).
+Relay nodes SHOULD use [33/WAKU2-DISCV5](../33/discv5.md) to continually discover
+other peers in the network.
+Each relay node MUST encode its supported shards into its discoverable ENR,
+as described in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#discovery).
+The ENR MUST be updated if the set of supported shards change.
+A node MAY choose to ignore discovered peers that do not support any of the shards
+in its own subscribed set.
+
+#### Transports
+
+Relay nodes MUST follow [10/WAKU2](../10/waku2.md) specifications with regards
+to supporting different transports.
+If TCP transport is available,
+each relay node MUST support it as transport for both dialing and listening.
+In addition,
+a relay node SHOULD support secure websockets for bidirectional communication streams,
+for example to allow connections from and to web browser-based clients.
+A relay node MAY support unsecure websockets if required by the application or
+running environment.
+
+#### Default services
+
+For each supported shard,
+each relay node SHOULD enable and support the following protocols as a service node:
+
+1. [12/WAKU2-FILTER](../12/filter.md) to allow resource-restricted peers to subscribe
+to messages matching a specific content filter.
+2. [13/WAKU2-STORE](../13/store.md) to allow other peers to request historical messages
+from this node.
+3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md) to allow resource-restricted peers to
+request publishing a message to the network on their behalf.
+4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md)
+to allow resource-restricted peers to discover more peers
+in a resource efficient way.
+
+#### Store service nodes
+
+Each relay node SHOULD support [13/WAKU2-STORE](../13/store.md)
+as a store service node, for each supported shard.
+The store SHOULD be configured to retain at least `12` hours of messages
+per supported shard.
+Store service nodes SHOULD only store messages
+with a valid [`rate_limit_proof`](#message-attributes) attribute.
+
+#### Non-relay nodes
+
+Nodes MAY opt out of relay functionality on any network shard
+and instead request services from relay nodes as clients
+using any of the defined service protocols:
+
+1. [12/WAKU2-FILTER](../12/filter.md)
+to subscribe to messages matching a specific content filter.
+2. [13/WAKU2-STORE](../13/store.md)
+to request historical messages matching a specific content filter.
+3. [19/WAKU2-LIGHTPUSH](../19/lightpush.md)
+to request publishing a message to the network.
+4. [WAKU2-PEER-EXCHANGE](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md)
+to discover more peers in a resource efficient way.
+
+#### Store client nodes
+
+Nodes MAY request historical messages from [13/WAKU2-STORE](../13/store.md)
+service nodes as store clients.
+A store client SHOULD discard any messages retrieved from a store service node
+that do not contain a valid [`rate_limit_proof`](#message-attributes) attribute.
+The client MAY consider service nodes returning messages
+without a valid [`rate_limit_proof`](#message-attributes) attribute as untrustworthy.
+The mechanism by which this may happen is currently underdefined.
+
+### Applications
+
+Applications are the higher-layer projects or
+platforms that make use of the generalized messaging capability of the network.
+In other words,
+an application defines a payload used in the various [10/WAKU2](../10/waku2.md) protocols.
+Any participant in an application SHOULD make use of an underlying node
+in order to communicate on the network.
+Applications SHOULD make use of an [autosharding](#autosharding) API
+to allow the underlying node to automatically select the target shard
+on the Waku Network.
+See the section on [autosharding](#autosharding) for more.
+
+## RLN rate-limiting
+
+The [17/WAKU2-RLN-RELAY](../17/rln-relay.md) protocol uses [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md)
+proofs to ensure that a pre-agreed rate limit
+of `x` messages every `y` seconds is not exceeded by any publisher.
+While the network is under capacity,
+individual relayers MAY choose to freely route messages without RLN proofs
+up to a discretionary bandwidth limit,
+after which messages without proofs MUST be discarded by relay nodes.
+This bandwidth limit SHOULD be enforced using a [bandwidth validation mechanism](#free-bandwidth-exceeded)
+separate from a RLN rate-limiting.
+This implies that quality of service and
+reliability is significantly lower for messages without proofs
+and at times of high network utilization these messages may not be relayed at all.
+
+### RLN Parameters
+
+The Waku Network uses the following RLN parameters:
+
+* `rlnRelayUserMessageLimit=100`:
+Amount of messages that a membership is allowed to publish per epoch.
+Configurable between `0` and `MAX_MESSAGE_LIMIT`.
+* `rlnEpochSizeSec=600`: Size of the epoch in seconds.
+* `rlnRelayChainId=11155111`: Network in which the RLN contract is deployed,
+aka Sepolia.
+* `rlnRelayEthContractAddress=0xCB33Aa5B38d79E3D9Fa8B10afF38AA201399a7e3`:
+Network address where RLN memberships are stored.
+* `staked_fund=0`: In other words, the Waku Network does not use RLN staking.
+Registering a membership just requires to pay gas.
+* `MAX_MESSAGE_LIMIT=100`:
+Maximum amount of messages allowed per epoch for any membership.
+Enforced in the contract.
+* `max_epoch_gap=20`: Maximum allowed gap in seconds into the past or
+future compared to the validator's clock.
+
+Nodes MUST _reject_ messages not respecting any of these parameters.
+Nodes SHOULD use Network Time Protocol (NTP) to synchronize their own clocks,
+thereby ensuring valid timestamps for proof generation and validation.
+Publishers to the Waku Network SHOULD register an RLN membership.
+
+### RLN Proofs
+
+Each RLN member MUST generate and attach an RLN proof to every published message
+as described in [17/WAKU2-RLN-RELAY](../17/rln-relay.md/#publishing) and [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md).
+Slashing is not implemented for the Waku Network.
+Instead, validators will penalise peers forwarding messages exceeding the rate limit
+as specified for [the rate-limiting validation mechanism](#rate-limit-exceeded).
+This incentivizes all relay nodes to validate RLN proofs
+and reject messages violating rate limits
+in order to continue participating in the network.
+
+## Network traffic
+
+All payload on the Waku Network MUST be encapsulated in a [14/WAKU2-MESSAGE](../14/message.md)
+with rate limit proof extensions defined for [17/WAKU2-RLN-RELAY](../17/rln-relay.md/#payloads).
+Each message on the Waku Network SHOULD be validated by each relayer,
+according to the rules discussed under [message validation](#message-validation).
+
+### Message Attributes
+
+* The mandatory `payload` attribute MUST contain the message data payload
+as crafted by the application.
+* The mandatory `content_topic` attribute MUST specify a string identifier
+that can be used for content-based filtering.
+This is also crafted by the application.
+See [Autosharding](#autosharding) for more on the content topic format.
+* The optional `meta` attribute MAY be omitted.
+If present, will form part of the message uniqueness vector described in [14/WAKU2-MESSAGE](../14/message.md).
+* The optional `version` attribute SHOULD be set to `0`.
+It MUST be interpreted as `0` if not present.
+* The mandatory `timestamp` attribute MUST contain the Unix epoch time
+at which the message was generated by the application.
+The value MUST be in nanoseconds.
+It MAY contain a fudge factor of up to 1 seconds in either direction
+to improve resistance to timing attacks.
+* The optional `ephemeral` attribute MUST be set to `true`,
+if the message should not be persisted by the Waku Network.
+* The optional `rate_limit_proof` attribute SHOULD be populated with the RLN proof
+as set out in [RLN Proofs](#rln-proofs).
+Messages with this field unpopulated MAY be discarded from the network by relayers.
+This field MUST be populated if the message should be persisted by the Waku Network.
+
+### Message Size
+
+Any [14/WAKU2-MESSAGE](../14/message.md) published to the network
+MUST NOT exceed an absolute maximum size of `150` kilobytes.
+This limit applies to the entire message after protobuf serialization,
+including attributes.
+It is RECOMMENDED not to exceed an average size of `4` kilobytes
+for [14/WAKU2-MESSAGE](../14/message.md) published to the network.
+
+### Message Validation
+
+Relay nodes MUST apply [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators)
+to each relayed message and
+SHOULD apply all of the rules set out in the section below
+to determine the validity of a message.
+Validation has one of three outcomes,
+repeated here from the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators)
+for ease of reference:
+
+1. Accept - the message is considered valid and
+it MUST be delivered and forwarded to the network.
+2. Reject - the message is considered invalid, MUST be rejected and
+SHOULD trigger a gossipsub scoring penalty against the transmitting peer.
+3. Ignore - the message SHOULD NOT be delivered and forwarded to the network,
+but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer.
+
+The following validation rules are defined:
+
+#### Decoding failure
+
+If a message fails to decode as a valid [14/WAKU2-MESSAGE](../14/message.md),
+the relay node MUST _reject_ the message.
+This SHOULD trigger a penalty against the transmitting peer.
+
+#### Invalid timestamp
+
+If a message has a timestamp deviating by more than `20` seconds
+either into the past or the future
+when compared to the relay node's internal clock,
+the relay node MUST _reject_ the message.
+This allows for some deviation between internal clocks,
+network routing latency and
+an optional [fudge factor when timestamping new messages](#message-attributes).
+
+#### Free bandwidth exceeded
+
+If a message contains no RLN proof
+and the current bandwidth utilization on the shard the message was published to
+equals or exceeds `1` Mbps,
+the relay node SHOULD _ignore_ the message.
+
+#### Invalid RLN epoch
+
+If a message contains an RLN proof
+and the `epoch` attached to the proof deviates by more than `max_epoch_gap` seconds
+from the relay node's own `epoch`,
+the relay node MUST _reject_ the message.
+`max_epoch_gap` is [set to `20` seconds](#rln-parameters) for the Waku Network.
+
+#### Invalid RLN proof
+
+If a message contains an RLN proof
+and the zero-knowledge proof is invalid
+according to the verification process described in [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md),
+the relay node MUST _ignore_ the message.
+
+#### Rate limit exceeded
+
+If a message contains an RLN proof
+and the relay node detects double signaling
+according to the verification process described in [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md),
+the relay node MUST _reject_ the message
+for violating the agreed rate limit of `rlnRelayUserMessageLimit` messages
+every `rlnEpochSizeSec` second.
+This SHOULD trigger a penalty against the transmitting peer.
+
+## Autosharding
+
+Nodes in the Waku Network SHOULD allow encapsulating applications to use autosharding,
+as defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#automatic-sharding)
+by automatically determining the appropriate pubsub topic
+from the list [of defined Waku Network shards](#network-shards).
+This allows the application to omit the target pubsub topic
+when invoking any Waku protocol function.
+Applications using autosharding MUST use content topics in the format
+defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#content-topics-format-for-autosharding)
+and SHOULD use the short length format:
+
+```text
+/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}
+```
+
+When an encapsulating application makes use of autosharding
+the underlying node MUST determine the target pubsub topic(s)
+from the content topics provided by the application
+using the hashing mechanism defined in [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/#automatic-sharding).
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+* [10/WAKU2](../10/waku2.md)
+* [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
+* [11/WAKU2-RELAY](../11/relay.md)
+* [WAKU2-RELAY-SHARDING](../../core/relay-sharding.md)
+* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
+* [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459)
+* [33/WAKU2-DISCV5](../33/discv5.md)
+* [12/WAKU2-FILTER](../12/filter.md)
+* [13/WAKU2-STORE](../13/store.md)
+* [19/WAKU2-LIGHTPUSH](../19/lightpush.md)
+* [34/WAKU2-PEER-EXCHANGE](../../core/peer-exchange.md)
+* [32/RLN-V1](../../../../vac/32/rln-v1.md)
+* [RLN-V2](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/raw/rln-v2.md)
+* [14/WAKU2-MESSAGE](../14/message.md)
+* [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators)
+* [WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md/)
+* [The Waku Network Config](https://github.com/waku-org/nwaku/blob/master/waku/factory/networks_config.nim#L31)

waku/standards/core/66/metadata.md

@@ -0,0 +1,56 @@
+---
+slug: 66
+title: 66/WAKU2-METADATA
+name: Waku Metadata Protocol
+status: draft
+editor: Alvaro Revuelta <alrevuelta@status.im>
+contributors:
+---
+
+## Abstract
+
+This specification describes the metadata
+that can be associated with a [10/WAKU2](../10/waku2.md) node.
+
+## Metadata Protocol
+
+Waku specifies a req/resp protocol that provides information about the node's medatadata.
+Such metadata is meant to be used by the node to decide if a peer is worth connecting
+or not.
+The node that makes the request,
+includes its metadata so that the receiver is aware of it,
+without requiring an extra interaction.
+The parameters are the following:
+
+* `clusterId`: Unique identifier of the cluster that the node is running in.
+* `shards`: Shard indexes that the node is subscribed to.
+
+***Protocol Identifier***
+
+> /vac/waku/metadata/1.0.0
+
+### Request
+
+```protobuf
+message WakuMetadataRequest {
+  optional uint32 cluster_id = 1;
+  repeated uint32 shards = 2;
+}
+```
+
+### Response
+
+```protobuf
+message WakuMetadataResponse {
+  optional uint32 cluster_id = 1;
+  repeated uint32 shards = 2;
+}
+```
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+## References
+
+* [10/WAKU2](../10/waku2.md)

waku/standards/legacy/7/data.md

@@ -1,64 +1,81 @@
----
-slug: 7
-title: 7/WAKU-DATA
-name: Waku Envelope data field
-status: stable
-editor: Oskar Thorén <oskarth@titanproxy.com>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Kim De Mey <kimdemey@status.im>
----
-
-This specification describes the encryption, decryption and signing of the content in the [data field used in Waku](../../standards/core/6/waku1.md/#abnf-specification).
-
-## Specification
-
-The `data` field is used within the `waku envelope`, the field MUST contain the encrypted payload of the envelope.
-
-The fields that are concatenated and encrypted as part of the `data` field are:
- - flags
- - auxiliary field
- - payload
- - padding
- - signature
- 
-In case of symmetric encryption, a `salt`  (a.k.a. AES Nonce, 12 bytes) field MUST be appended. 
-
-### ABNF
-
-Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
-
-```abnf
-; 1 byte; first two bits contain the size of auxiliary field, 
-; third bit indicates whether the signature is present.
-flags           = 1OCTET
-
-; contains the size of payload.
-auxiliary-field = 4*OCTET
-
-; byte array of arbitrary size (may be zero)
-payload         = *OCTET
-
-; byte array of arbitrary size (may be zero).
-padding         = *OCTET
-
-; 65 bytes, if present.
-signature       = 65OCTET
-
-; 2 bytes, if present (in case of symmetric encryption).
-salt            = 2OCTET
-
-data        = flags auxiliary-field payload padding [signature] [salt]
-```
-
-### Signature
-
-Those unable to decrypt the envelope data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the `R`, `S` and `V` parameters of the SECP-256k1 ECDSA signature, in that order. `R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned. `V` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
-
-### Padding
-
-The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 7
+title: 7/WAKU-DATA
+name: Waku Envelope data field
+status: stable
+editor: Oskar Thorén <oskarth@titanproxy.com>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Kim De Mey <kimdemey@status.im>
+---
+
+This specification describes the encryption,
+decryption and signing of the content in the [data field used in Waku](../6/waku1.md/#abnf-specification).
+
+## Specification
+
+The `data` field is used within the `waku envelope`,
+the field MUST contain the encrypted payload of the envelope.
+
+The fields that are concatenated and encrypted as part of the `data` field are:
+
+- flags
+- auxiliary field
+- payload
+- padding
+- signature
+
+In case of symmetric encryption, a `salt`
+(a.k.a. AES Nonce, 12 bytes) field MUST be appended.
+
+### ABNF
+
+Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
+we have the following format:
+
+```abnf
+; 1 byte; first two bits contain the size of auxiliary field, 
+; third bit indicates whether the signature is present.
+flags           = 1OCTET
+
+; contains the size of payload.
+auxiliary-field = 4*OCTET
+
+; byte array of arbitrary size (may be zero)
+payload         = *OCTET
+
+; byte array of arbitrary size (may be zero).
+padding         = *OCTET
+
+; 65 bytes, if present.
+signature       = 65OCTET
+
+; 2 bytes, if present (in case of symmetric encryption).
+salt            = 2OCTET
+
+data        = flags auxiliary-field payload padding [signature] [salt]
+```
+
+### Signature
+
+Those unable to decrypt the envelope data are also unable to access the signature.
+The signature, if provided,
+is the ECDSA signature of the Keccak-256 hash of the unencrypted data
+using the secret key of the originator identity.
+The signature is serialized as the concatenation of the `R`, `S` and
+`V` parameters of the SECP-256k1 ECDSA signature, in that order.
+`R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned.
+`V` MUST be an 8-bit big-endian encoded,
+non-normalized and should be either 27 or 28.
+
+### Padding
+
+The padding field is used to align data size,
+since data size alone might reveal important metainformation.
+Padding can be arbitrary size.
+However, it is recommended that the size of Data Field (excluding the Salt)
+before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

waku/standards/legacy/8/mail.md

@@ -1,123 +1,169 @@
----
-slug: 8
-title: 8/WAKU-MAIL
-name: Waku Mailserver
-status: stable
-editor: Andrea Maria Piana <andreap@status.im>
-contributors:
-  - Adam Babik <adam@status.im>
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
----
-
-## Abstract
-
-In this specification, we describe Mailservers. These are nodes responsible for archiving envelopes and delivering them to peers on-demand.
-
-## Specification
-
-A node which wants to provide mailserver functionality MUST store envelopes from incoming Messages packets (Waku packet-code `0x01`). The envelopes can be stored in any format, however they MUST be serialized and deserialized to the Waku envelope format.
-
-A mailserver SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
-
-### Requesting Historic Envelopes
-
-In order to request historic envelopes, a node MUST send a packet P2P Request (`0x7e`) to a peer providing mailserver functionality. This packet requires one argument which MUST be a Waku envelope.
-
-In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
-
-```abnf
-; UNIX time in seconds; oldest requested envelope's creation time
-lower  = 4OCTET
-
-; UNIX time in seconds; newest requested envelope's creation time
-upper  = 4OCTET
-
-; array of Waku topics encoded in a bloom filter to filter envelopes
-bloom  = 64OCTET
-
-; unsigned integer limiting the number of returned envelopes
-limit  = 4OCTET
-
-; array of a cursor returned from the previous request (optional)
-cursor = *OCTET
-
-; List of topics interested in
-topics = "[" *1000topic "]"
-
-; 4 bytes of arbitrary data
-topic = 4OCTET
-
-payload-without-topic = "[" lower upper bloom limit [ cursor ] "]"
-
-payload-with-topic = "[" lower upper bloom limit cursor [ topics ] "]"
-
-payload = payload-with-topic | payload-without-topic
-```
-
-The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit` so that the requester can send another request using the obtained `Cursor` value. What exactly is in the `Cursor` is up to the implementation. The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request to another mailserver because the format or the result MAY be different.
-
-The envelope MUST be encrypted with a symmetric key agreed between the requester and Mailserver.
-
-If `Topics` is used the `Cursor` field MUST be specified for the argument order to be unambiguous. However, it MAY be set to `null`. `Topics` is used to specify which topics a node is interested in. If `Topics` is not empty, a mailserver MUST only send envelopes that belong to a topic from `Topics` list and `Bloom` value MUST be ignored.
-
-### Receiving Historic Envelopes
-
-Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes. A Mailserver MUST limit the amount of messages sent, either by the `Limit` specified in the request or limited to the maximum [RLPx packet size](./waku#maximum-packet-size), whichever limit comes first.
-
-In order to receive historic envelopes from a mailserver, a node MUST trust the selected mailserver, that is allow to receive expired packets with the P2P Message code. By default, such packets are discarded.
-
-Received envelopes MUST be passed through the Whisper envelope pipelines so that they are picked up by registered filters and passed to subscribers.
-
-For a requester, to know that all envelopes have been sent by mailserver, it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by a list with:
-
-```abnf
-; array with a Keccak-256 hash of the envelope containing the original request.
-request-id = 32OCTET
-
-; array with a Keccak-256 hash of the last sent envelope for the request. 
-last-envelope-hash = 32OCTET
-
-; array of a cursor returned from the previous request (optional)
-cursor = *OCTET
-
-payload = "[" request-id last-envelope-hash [ cursor ] "]"
-```
-
-If `Cursor` is not empty, it means that not all envelopes were sent due to the set `Limit` in the request. One or more consecutive requests MAY be sent with `Cursor` field filled in in order to receive the rest of the envelopes.
-
-### Security considerations
-
-There are several security considerations to take into account when running or interacting with Mailservers. Chief among them are: scalability, DDoS-resistance and privacy.
-
-**Mailserver High Availability requirement:**
-
-A mailserver has to be online to receive envelopes for other nodes, this puts a high availability requirement on it.
-
-**Mailserver client privacy:**
-
-A mailserver client fetches archival envelopes from a mailserver through a direct connection. 
-In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver.
-The collection of such information allows the mailserver to link clients' IP/IDs to their topic interests and build a profile for each client over time.
-As such, the mailserver client has to trust the mailserver with this level of information.
-A similar concern exists for the light nodes and their direct peers which is discussed in the security considerations of [6/WAKU1](/spec/7).
-
-**Mailserver trusted connection:**
-
-A mailserver has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning mailserver can overwhelm an individual node.
-
-## Changelog
-
-| Version                                                                                        | Comment |
-| :--------------------------------------------------------------------------------------------: | ------- |
-| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)       | marked stable as it is in use.                                                   |
-| 0.2.0                                                                                          | Add topic interest to reduce bandwidth usage |
-| [0.1.0](https://github.com/vacp2p/specs/blob/06d4c736c920526472a507e5d842212843a112ed/wms.md)  | Initial Release |
-
-### Difference between wms 0.1 and wms 0.2
-
-- `topics` option
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 8
+title: 8/WAKU-MAIL
+name: Waku Mailserver
+status: stable
+editor: Andrea Maria Piana <andreap@status.im>
+contributors:
+  - Adam Babik <adam@status.im>
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+## Abstract
+
+In this specification, we describe Mailservers.
+These are nodes responsible for archiving envelopes and
+delivering them to peers on-demand.
+
+## Specification
+
+A node which wants to provide mailserver functionality MUST store envelopes
+from incoming Messages packets (Waku packet-code `0x01`).
+The envelopes can be stored in any format,
+however they MUST be serialized and deserialized to the Waku envelope format.
+
+A mailserver SHOULD store envelopes for all topics
+to be generally useful for any peer,
+however for specific use cases it MAY store envelopes for a subset of topics.
+
+### Requesting Historic Envelopes
+
+In order to request historic envelopes,
+a node MUST send a packet P2P Request (`0x7e`)
+to a peer providing mailserver functionality.
+This packet requires one argument which MUST be a Waku envelope.
+
+In the Waku envelope's payload section,
+there MUST be RLP-encoded information about the details of the request:
+
+```abnf
+; UNIX time in seconds; oldest requested envelope's creation time
+lower  = 4OCTET
+
+; UNIX time in seconds; newest requested envelope's creation time
+upper  = 4OCTET
+
+; array of Waku topics encoded in a bloom filter to filter envelopes
+bloom  = 64OCTET
+
+; unsigned integer limiting the number of returned envelopes
+limit  = 4OCTET
+
+; array of a cursor returned from the previous request (optional)
+cursor = *OCTET
+
+; List of topics interested in
+topics = "[" *1000topic "]"
+
+; 4 bytes of arbitrary data
+topic = 4OCTET
+
+payload-without-topic = "[" lower upper bloom limit [ cursor ] "]"
+
+payload-with-topic = "[" lower upper bloom limit cursor [ topics ] "]"
+
+payload = payload-with-topic | payload-without-topic
+```
+
+The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and
+`Upper` is greater than `Limit` so that the requester can send another request
+using the obtained `Cursor` value.
+What exactly is in the `Cursor` is up to the implementation.
+The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request
+to another mailserver because the format or the result MAY be different.
+
+The envelope MUST be encrypted with a symmetric key agreed between the requester
+and Mailserver.
+
+If `Topics` is used the `Cursor` field MUST be specified
+for the argument order to be unambiguous.
+However, it MAY be set to `null`.
+`Topics` is used to specify which topics a node is interested in.
+If `Topics` is not empty,
+a mailserver MUST only send envelopes that belong to a topic from `Topics` list and
+`Bloom` value MUST be ignored.
+
+### Receiving Historic Envelopes
+
+Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`)
+followed by an array of Waku envelopes.
+A Mailserver MUST limit the amount of messages sent,
+either by the `Limit` specified in the request or
+limited to the maximum [RLPx packet size](./waku#maximum-packet-size),
+whichever limit comes first.
+
+In order to receive historic envelopes from a mailserver,
+a node MUST trust the selected mailserver,
+that is allow to receive expired packets with the P2P Message code.
+By default, such packets are discarded.
+
+Received envelopes MUST be passed through the Whisper envelope pipelines
+so that they are picked up by registered filters and passed to subscribers.
+
+For a requester, to know that all envelopes have been sent by mailserver,
+it SHOULD handle P2P Request Complete code (`0x7d`).
+This code is followed by a list with:
+
+```abnf
+; array with a Keccak-256 hash of the envelope containing the original request.
+request-id = 32OCTET
+
+; array with a Keccak-256 hash of the last sent envelope for the request. 
+last-envelope-hash = 32OCTET
+
+; array of a cursor returned from the previous request (optional)
+cursor = *OCTET
+
+payload = "[" request-id last-envelope-hash [ cursor ] "]"
+```
+
+If `Cursor` is not empty,
+it means that not all envelopes were sent due to the set `Limit` in the request.
+One or more consecutive requests MAY be sent with `Cursor` field filled
+in order to receive the rest of the envelopes.
+
+### Security considerations
+
+There are several security considerations to take into account when running or
+interacting with Mailservers.
+Chief among them are: scalability, DDoS-resistance and privacy.
+
+**Mailserver High Availability requirement:**
+
+A mailserver has to be online to receive envelopes for other nodes,
+this puts a high availability requirement on it.
+
+**Mailserver client privacy:**
+
+A mailserver client fetches archival envelopes from a mailserver
+through a direct connection.
+In this direct connection,
+the client discloses its IP/ID as well as the topics/ bloom filter
+it is interested in to the mailserver.
+The collection of such information allows the mailserver to link clients' IP/IDs
+to their topic interests and build a profile for each client over time.
+As such, the mailserver client has to trust the mailserver with this level of information.
+A similar concern exists for the light nodes and
+their direct peers which is discussed in the security considerations of [6/WAKU1](../6/waku1.md).
+
+**Mailserver trusted connection:**
+
+A mailserver has a direct TCP connection, which means they are trusted to send traffic.
+This means a malicious or malfunctioning mailserver can overwhelm an individual node.
+
+## Changelog
+
+| Version                                                                                        | Comment |
+| :--------------------------------------------------------------------------------------------: | ------- |
+| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)       | marked stable as it is in use.                                                   |
+| 0.2.0                                                                                          | Add topic interest to reduce bandwidth usage |
+| [0.1.0](https://github.com/vacp2p/specs/blob/06d4c736c920526472a507e5d842212843a112ed/wms.md)  | Initial Release |
+
+### Difference between wms 0.1 and wms 0.2
+
+- `topics` option
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

waku/standards/legacy/9/rpc.md

@@ -1,409 +1,459 @@
----
-slug: 9
-title: 9/WAKU-RPC
-name: Waku RPC API
-status: stable
-editor: Andrea Maria Piana <andreap@status.im>
-contributors:
-  - Dean Eigenmann <dean@status.im>
-  - Oskar Thorén <oskarth@titanproxy.com>
----
-
-This specification describes the RPC API that Waku nodes MAY adhere to. The unified API allows clients to easily
-be able to connect to any node implementation. The API described is privileged as a node stores the keys of clients. 
-
-## Introduction
-
-This API is based off the [Whisper V6 RPC API](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API).
-
-## Wire Protocol
-
-### Transport
-
-Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API that can be accessed. The JSON RPC version SHOULD be `2.0`. Below is an example request:
-
-```json
-{
-  "jsonrpc":"2.0",
-  "method":"waku_version",
-  "params":[],
-  "id":1
-}
-```
-
-#### Fields
-
-| Field     | Description                                         |
-| --------- | --------------------------------------------------- |
-| `jsonrpc` | Contains the used JSON RPC version (`Default: 2.0`) |
-| `method`  | Contains the JSON RPC method that is being called   |
-| `params`  | An array of parameters for the request              |
-| `id`      | The request ID                                      |
-
-### Objects
-
-In this section you will find objects used throughout the JSON RPC API.
-
-#### Message
-
-The message object represents a Waku message. Below you will find the description of the attributes contained in the message object. A message is the decrypted payload and padding of an [envelope](/spec/7) along with all of its metadata and other extra information such as the hash.
-
-| Field | Type | Description |
-| ----: | :--: | ----------- |
-| `sig` | string | Public Key that signed this message |
-| `recipientPublicKey` | string | The recipients public key |
-| `ttl` | number | Time-to-live in seconds |
-| `timestamp` | number | Unix timestamp of the message generation |
-| `topic` | string | 4 bytes, the message topic |
-| `payload` | string | Decrypted payload |
-| `padding` | string | Optional padding, byte array of arbitrary length |
-| `pow` | number | The proof of work value |
-| `hash` | string | Hash of the enveloped message |
-
-#### Filter
-
-The filter object represents filters that can be applied to retrieve messages. Below you will find the description of the attributes contained in the filter object.
-
-| Field | Type | Description |
-| ----: | :--: | ----------- |
-| `symKeyID` | string | ID of the symmetric key for message decryption |
-| `privateKeyID` | string | ID of private (asymmetric) key for message decryption |
-| `sig` | string | Public key of the signature |
-| `minPow` | number | Minimal PoW requirement for incoming messages |
-| `topics` | array | Array of possible topics, this can also contain partial topics |
-| `allowP2P` | boolean | Indicates if this filter allows processing of direct peer-to-peer messages |
-
-All fields are optional, however `symKeyID` or `privateKeyID` must be present, it cannot be both. Additionally, the `topics` field is only optional when an asymmetric key is used.
-
-### Methods
-
-#### `waku_version`
-
-The `waku_version` method returns the current version number.
-
-##### Parameters
-
-none
-
-##### Response
-
-- **string** - The version number.
-
-#### `waku_info`
-
-The `waku_info` method returns information about a Waku node.
-
-##### Parameters
-
-none
-
-##### Response
-
-The response is an `Object` containing the following fields:
-
-- **`minPow` [number]** - The current PoW requirement.
-- **`maxEnvelopeSize` [float]** - The current maximum envelope size in bytes.
-- **`memory` [number]** - The memory size of the floating messages in bytes.
-- **`envelopes` [number]** - The number of floating envelopes.
-
-#### `waku_setMaxEnvelopeSize`
-
-Sets the maximum envelope size allowed by this node. Any envelopes larger than this size both incoming and outgoing will be rejected. The envelope size can never exceed the underlying envelope size of `10mb`.
-
-##### Parameters
-
- - **number** - The message size in bytes.
-
-##### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_setMinPoW`
-
-Sets the minimal PoW required by this node.
-
-##### Parameters
-
- - **number** - The new PoW requirement.
-
-##### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_markTrustedPeer`
-
-Marks a specific peer as trusted allowing it to send expired messages.
-
-##### Parameters
-
-- **string** - `enode` of the peer.
-
-##### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_newKeyPair`
-
-Generates a keypair used for message encryption and decryption.
-
-##### Parameters
-
-none
-
-##### Response
-
- - **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_addPrivateKey`
-
-Stores a key and returns its ID.
-
-##### Parameters
-
-- **string** - Private key as hex bytes.
-
-##### Response
-
- - **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_deleteKeyPair`
-
-Deletes a specific key if it exists.
-
-##### Parameters
-
-- **string** - ID of the Key pair.
-
-##### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_hasKeyPair`
-
-Checks if the node has a private key of a key pair matching the given ID.
-
-##### Parameters
-
-- **string** - ID of the Key pair.
-
-##### Response
-
-- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_getPublicKey`
-
-Returns the public key for an ID.
-
-##### Parameters
-
-- **string** - ID of the Key.
-
-##### Response
-
- - **string** - The public key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_getPrivateKey`
-
-Returns the private key for an ID.
-
-##### Parameters
-
-- **string** - ID of the Key.
-
-##### Response
-
- - **string** - The private key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_newSymKey`
-
-Generates a random symmetric key and stores it under an ID. This key can be used to encrypt and decrypt messages where the key is known to both parties.
-
-##### Parameters
-
-none
-
-##### Response
-
- - **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_addSymKey`
-
-Stores the key and returns its ID.
-
-##### Parameters
-
-- **string** - The raw key for symmetric encryption hex encoded.
-
-##### Response
-
- - **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_generateSymKeyFromPassword`
-
-Generates the key from a password and stores it.
-
-##### Parameters
-
- - **string** - The password.
-
-##### Response
-
- - **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_hasSymKey`
-
-Returns whether there is a key associated with the ID.
-
-##### Parameters
-
-- **string** - ID of the Key.
-
-##### Response
-
-- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_getSymKey`
-
-Returns the symmetric key associated with an ID.
-
-##### Parameters
-
-- **string** - ID of the Key.
-
-##### Response
-
-- **string** - Raw key on success or an [error](https://www.jsonrpc.org/specification#error_object) of failure.
-
-#### `waku_deleteSymKey`
-
-Deletes the key associated with an ID.
-
-##### Parameters
-
-- **string** - ID of the Key.
-
-##### Response
-
-- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_subscribe`
-
-Creates and registers a new subscription to receive notifications for inbound Waku messages.
-
-##### Parameters
-
-The parameters for this request is an array containing the following fields:
-
- 1. **string** - The ID of the function call, in case of Waku this must contain the value "messages".
- 2. **object** - The [message filter](#filter).
-
-##### Response
-
-- **string** - ID of the subscription or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-###### Notifications
-
-Notifications received by the client contain a [message](#message) matching the filter. Below is an example notification:
-
-```json
-{
-  "jsonrpc": "2.0",
-  "method": "waku_subscription",
-  "params": {
-    "subscription": "02c1f5c953804acee3b68eda6c0afe3f1b4e0bec73c7445e10d45da333616412",
-    "result": {
-      "sig": "0x0498ac1951b9078a0549c93c3f6088ec7c790032b17580dc3c0c9e900899a48d89eaa27471e3071d2de6a1f48716ecad8b88ee022f4321a7c29b6ffcbee65624ff",
-      "recipientPublicKey": null,
-      "ttl": 10,
-      "timestamp": 1498577270,
-      "topic": "0xffaadd11",
-      "payload": "0xffffffdddddd1122",
-      "padding": "0x35d017b66b124dd4c67623ed0e3f23ba68e3428aa500f77aceb0dbf4b63f69ccfc7ae11e39671d7c94f1ed170193aa3e327158dffdd7abb888b3a3cc48f718773dc0a9dcf1a3680d00fe17ecd4e8d5db31eb9a3c8e6e329d181ecb6ab29eb7a2d9889b49201d9923e6fd99f03807b730780a58924870f541a8a97c87533b1362646e5f573dc48382ef1e70fa19788613c9d2334df3b613f6e024cd7aadc67f681fda6b7a84efdea40cb907371cd3735f9326d02854",
-      "pow": 0.6714754098360656,
-      "hash": "0x17f8066f0540210cf67ef400a8a55bcb32a494a47f91a0d26611c5c1d66f8c57"
-    }
-  }
-}
-```
-
-#### `waku_unsubscribe`
-
-Cancels and removes an existing subscription. The node MUST stop sending the client notifications.
-
-##### Parameters
-
- - **string** - The subscription ID.
-
-##### Response
-
-- **bool** - `true` or `false`
-
-#### `waku_newMessageFilter`
-
-Creates a new message filter within the node. This filter can be used to poll for new messages that match the criteria.
-
-##### Parameters
-
-The request must contain a [message filter](#filter) as its parameter. 
-
-##### Response
-
- - **string** - The ID of the filter.
-
-#### `waku_deleteMessageFilter`
-
-Removes a message filter from the node.
-
-##### Parameters
-
-- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
-
-##### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_getFilterMessages`
-
-Retrieves messages that match a filter criteria and were received after the last time this function was called.
-
-##### Parameters
-
-- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
-
-##### Response
-
-The response contains an array of [messages](#messages) or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-#### `waku_post`
-
-The `waku_post` method creates a waku envelope and propagates it to the network.
-
-##### Parameters
-
-The parameters is an `Object` containing the following fields:
- - **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
- - **`pubKey` [string]** `optional` - The public key for message encryption.
- - **`sig` [string]** `optional` - The ID of the signing key.
- - **`ttl` [number]** - The time-to-live in seconds.
- - **`topic` [string]** - 4 bytes message topic.
- - **`payload` [string]** - The payload to be encrypted.
- - **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
- - **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
- - **`powTarget` [number]** - Minimal PoW target required for this message.
- - **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
- 
-*Either the **`symKeyID`** or the **`pubKey`** need to be present. It can not be both.*
-
-#### Response
-
-- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
-
-## Changelog
-
-| Version                                                                                 | Comment          |
-| :--------------------------------------------------------------------------------------:| ---------------- |
-| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. | 
-
-## Copyright
-
-Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+---
+slug: 9
+title: 9/WAKU-RPC
+name: Waku RPC API
+status: stable
+editor: Andrea Maria Piana <andreap@status.im>
+contributors:
+  - Dean Eigenmann <dean@status.im>
+  - Oskar Thorén <oskarth@titanproxy.com>
+---
+
+This specification describes the RPC API that Waku nodes MAY adhere to.
+The unified API allows clients to easily
+be able to connect to any node implementation.
+The API described is privileged as a node stores the keys of clients.
+
+## Introduction
+
+This API is based off the [Whisper V6 RPC API](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API).
+
+## Wire Protocol
+
+### Transport
+
+Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API
+that can be accessed.
+The JSON RPC version SHOULD be `2.0`.
+Below is an example request:
+
+```json
+{
+  "jsonrpc":"2.0",
+  "method":"waku_version",
+  "params":[],
+  "id":1
+}
+```
+
+#### Fields
+
+| Field     | Description                                         |
+| --------- | --------------------------------------------------- |
+| `jsonrpc` | Contains the used JSON RPC version (`Default: 2.0`) |
+| `method`  | Contains the JSON RPC method that is being called   |
+| `params`  | An array of parameters for the request              |
+| `id`      | The request ID                                      |
+
+### Objects
+
+In this section you will find objects used throughout the JSON RPC API.
+
+#### Message
+
+The message object represents a Waku message.
+Below you will find the description of the attributes contained in the message object.
+A message is the decrypted payload and
+padding of an [envelope](../7/data.md) along with all of its metadata and
+other extra information such as the hash.
+
+| Field | Type | Description |
+| ----: | :--: | ----------- |
+| `sig` | string | Public Key that signed this message |
+| `recipientPublicKey` | string | The recipients public key |
+| `ttl` | number | Time-to-live in seconds |
+| `timestamp` | number | Unix timestamp of the message generation |
+| `topic` | string | 4 bytes, the message topic |
+| `payload` | string | Decrypted payload |
+| `padding` | string | Optional padding, byte array of arbitrary length |
+| `pow` | number | The proof of work value |
+| `hash` | string | Hash of the enveloped message |
+
+#### Filter
+
+The filter object represents filters that can be applied to retrieve messages.
+Below you will find the description of the attributes contained in the filter object.
+
+| Field | Type | Description |
+| ----: | :--: | ----------- |
+| `symKeyID` | string | ID of the symmetric key for message decryption |
+| `privateKeyID` | string | ID of private (asymmetric) key for message decryption |
+| `sig` | string | Public key of the signature |
+| `minPow` | number | Minimal PoW requirement for incoming messages |
+| `topics` | array | Array of possible topics, this can also contain partial topics |
+| `allowP2P` | boolean | Indicates if this filter allows processing of direct peer-to-peer messages |
+
+All fields are optional, however `symKeyID` or `privateKeyID` must be present,
+it cannot be both.
+Additionally, the `topics` field is only optional when an asymmetric key is used.
+
+### Methods
+
+#### `waku_version`
+
+The `waku_version` method returns the current version number.
+
+##### Parameters
+
+none
+
+##### Response
+
+- **string** - The version number.
+
+#### `waku_info`
+
+The `waku_info` method returns information about a Waku node.
+
+Parameters
+
+none
+
+Response
+
+The response is an `Object` containing the following fields:
+
+- **`minPow` [number]** - The current PoW requirement.
+- **`maxEnvelopeSize` [float]** - The current maximum envelope size in bytes.
+- **`memory` [number]** - The memory size of the floating messages in bytes.
+- **`envelopes` [number]** - The number of floating envelopes.
+
+#### `waku_setMaxEnvelopeSize`
+
+Sets the maximum envelope size allowed by this node.
+Any envelopes larger than this size both incoming and outgoing will be rejected.
+The envelope size can never exceed the underlying envelope size of `10mb`.
+
+Parameters
+
+- **number** - The message size in bytes.
+
+Response
+
+- **bool** -
+`true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_setMinPoW`
+
+Sets the minimal PoW required by this node.
+
+Parameters
+
+- **number** - The new PoW requirement.
+
+Response
+
+- **bool** -
+`true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_markTrustedPeer`
+
+Marks a specific peer as trusted allowing it to send expired messages.
+
+Parameters
+
+- **string** - `enode` of the peer.
+
+Response
+
+- **bool** -
+`true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_newKeyPair`
+
+Generates a keypair used for message encryption and decryption.
+
+Parameters
+
+none
+
+Response
+
+- **string** -
+Key ID on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_addPrivateKey`
+
+Stores a key and returns its ID.
+
+Parameters
+
+- **string** - Private key as hex bytes.
+
+Response
+
+- **string** - Key ID on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_deleteKeyPair`
+
+Deletes a specific key if it exists.
+
+Parameters
+
+- **string** - ID of the Key pair.
+
+Response
+
+- **bool** - `true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_hasKeyPair`
+
+Checks if the node has a private key of a key pair matching the given ID.
+
+Parameters
+
+- **string** - ID of the Key pair.
+
+Response
+
+- **bool** -
+`true` or
+`false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_getPublicKey`
+
+Returns the public key for an ID.
+
+Parameters
+
+- **string** - ID of the Key.
+
+Response
+
+- **string** - The public key or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_getPrivateKey`
+
+Returns the private key for an ID.
+
+Parameters
+
+- **string** - ID of the Key.
+
+Response
+
+- **string** - The private key or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_newSymKey`
+
+Generates a random symmetric key and stores it under an ID.
+This key can be used to encrypt and
+decrypt messages where the key is known to both parties.
+
+Parameters
+
+none
+
+Response
+
+- **string** - The key ID or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_addSymKey`
+
+Stores the key and returns its ID.
+
+Parameters
+
+- **string** - The raw key for symmetric encryption hex encoded.
+
+Response
+
+- **string** - The key ID or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_generateSymKeyFromPassword`
+
+Generates the key from a password and stores it.
+
+Parameters
+
+- **string** - The password.
+
+Response
+
+- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object)
+on failure.
+
+#### `waku_hasSymKey`
+
+Returns whether there is a key associated with the ID.
+
+Parameters
+
+- **string** - ID of the Key.
+
+Response
+
+- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object)
+on failure.
+
+#### `waku_getSymKey`
+
+Returns the symmetric key associated with an ID.
+
+Parameters
+
+- **string** - ID of the Key.
+
+Response
+
+- **string** - Raw key on success or
+an [error](https://www.jsonrpc.org/specification#error_object) of failure.
+
+#### `waku_deleteSymKey`
+
+Deletes the key associated with an ID.
+
+Parameters
+
+- **string** - ID of the Key.
+
+Response
+
+- **bool** -
+`true` or `false` or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_subscribe`
+
+Creates and
+registers a new subscription to receive notifications for inbound Waku messages.
+
+Parameters
+
+The parameters for this request is an array containing the following fields:
+
+ 1. **string** - The ID of the function call,
+in case of Waku this must contain the value "messages".
+ 2. **object** - The [message filter](#filter).
+
+Response
+
+- **string** - ID of the subscription or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+Notifications
+
+Notifications received by the client contain a [message](#message) matching the filter.
+Below is an example notification:
+
+```json
+{
+  "jsonrpc": "2.0",
+  "method": "waku_subscription",
+  "params": {
+    "subscription": "02c1f5c953804acee3b68eda6c0afe3f1b4e0bec73c7445e10d45da333616412",
+    "result": {
+      "sig": "0x0498ac1951b9078a0549c93c3f6088ec7c790032b17580dc3c0c9e900899a48d89eaa27471e3071d2de6a1f48716ecad8b88ee022f4321a7c29b6ffcbee65624ff",
+      "recipientPublicKey": null,
+      "ttl": 10,
+      "timestamp": 1498577270,
+      "topic": "0xffaadd11",
+      "payload": "0xffffffdddddd1122",
+      "padding": "0x35d017b66b124dd4c67623ed0e3f23ba68e3428aa500f77aceb0dbf4b63f69ccfc7ae11e39671d7c94f1ed170193aa3e327158dffdd7abb888b3a3cc48f718773dc0a9dcf1a3680d00fe17ecd4e8d5db31eb9a3c8e6e329d181ecb6ab29eb7a2d9889b49201d9923e6fd99f03807b730780a58924870f541a8a97c87533b1362646e5f573dc48382ef1e70fa19788613c9d2334df3b613f6e024cd7aadc67f681fda6b7a84efdea40cb907371cd3735f9326d02854",
+      "pow": 0.6714754098360656,
+      "hash": "0x17f8066f0540210cf67ef400a8a55bcb32a494a47f91a0d26611c5c1d66f8c57"
+    }
+  }
+}
+```
+
+#### `waku_unsubscribe`
+
+Cancels and removes an existing subscription.
+The node MUST stop sending the client notifications.
+
+Parameters
+
+- **string** - The subscription ID.
+
+Response
+
+- **bool** - `true` or `false`
+
+#### `waku_newMessageFilter`
+
+Creates a new message filter within the node.
+This filter can be used to poll for new messages that match the criteria.
+
+Parameters
+
+The request must contain a [message filter](#filter) as its parameter.
+
+Response
+
+- **string** - The ID of the filter.
+
+#### `waku_deleteMessageFilter`
+
+Removes a message filter from the node.
+
+Parameters
+
+- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newmessagefilter).
+
+Response
+
+- **bool** - `true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_getFilterMessages`
+
+Retrieves messages that match a filter criteria and
+were received after the last time this function was called.
+
+Parameters
+
+- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newmessagefilter).
+
+Response
+
+The response contains an array of [messages](#message) or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+#### `waku_post`
+
+The `waku_post` method creates a waku envelope and propagates it to the network.
+
+Parameters
+
+The parameters is an `Object` containing the following fields:
+
+- **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
+- **`pubKey` [string]** `optional` - The public key for message encryption.
+- **`sig` [string]** `optional` - The ID of the signing key.
+- **`ttl` [number]** - The time-to-live in seconds.
+- **`topic` [string]** - 4 bytes message topic.
+- **`payload` [string]** - The payload to be encrypted.
+- **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
+- **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
+- **`powTarget` [number]** - Minimal PoW target required for this message.
+- **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
+
+*Either the **`symKeyID`** or the **`pubKey`** need to be present.
+It can not be both.*
+
+Response
+
+- **bool** -
+`true` on success or
+an [error](https://www.jsonrpc.org/specification#error_object) on failure.
+
+## Changelog
+
+| Version                                                                                 | Comment          |
+| :--------------------------------------------------------------------------------------:| ---------------- |
+| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. |
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).