mirror of
https://github.com/vacp2p/rfc-index.git
synced 2026-01-10 08:08:17 -05:00
Compare commits
61 Commits
mls-rfc-up
...
vac-Stakin
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eb27ea421a | ||
|
|
9933b7ca74 | ||
|
|
3505da6bd6 | ||
|
|
3b968ccce3 | ||
|
|
536d31b5b7 | ||
|
|
4361e2958f | ||
|
|
88e326289a | ||
|
|
78323d890a | ||
|
|
24fe617c5f | ||
|
|
502d598be3 | ||
|
|
b60abdb2ff | ||
|
|
f44fc6eb93 | ||
|
|
36439eb400 | ||
|
|
1258e483fe | ||
|
|
882177064b | ||
|
|
4708ef71f3 | ||
|
|
4b333dae16 | ||
|
|
b768883f95 | ||
|
|
624cc8370b | ||
|
|
966b31d95e | ||
|
|
78be8ca331 | ||
|
|
860ffe6b11 | ||
|
|
dac06391c5 | ||
|
|
11258ec0b6 | ||
|
|
f31f41fec9 | ||
|
|
518717f1c6 | ||
|
|
5e3b4788ec | ||
|
|
711c9e656b | ||
|
|
170ad5dd51 | ||
|
|
36caaa621a | ||
|
|
97970270b1 | ||
|
|
dc4d1e5d65 | ||
|
|
db90adc94e | ||
|
|
062d16a02a | ||
|
|
5304b472f4 | ||
|
|
f829b12517 | ||
|
|
91c9679bc8 | ||
|
|
614348a498 | ||
|
|
5971166405 | ||
|
|
db365cb756 | ||
|
|
4df2d5f787 | ||
|
|
af7c413e64 | ||
|
|
7408956616 | ||
|
|
34aa3f3647 | ||
|
|
1b8b2ac70b | ||
|
|
3b152e44b5 | ||
|
|
cafa04fb93 | ||
|
|
805280880a | ||
|
|
517b63984c | ||
|
|
f08de10845 | ||
|
|
e8a3f8a77d | ||
|
|
8ee2a6d6b2 | ||
|
|
f4b34afd1a | ||
|
|
0277fd0c4d | ||
|
|
9bed57e4ad | ||
|
|
235c1d5aa6 | ||
|
|
718245979f | ||
|
|
1ddddc7994 | ||
|
|
a60a2c45de | ||
|
|
c3d5fe6f88 | ||
|
|
7a01711ffc |
@@ -1,17 +1,17 @@
|
||||
---
|
||||
slug: 38
|
||||
title: 38/CONSENSUS-CLARO
|
||||
title: CONSENSUS-CLARO
|
||||
name: Claro Consensus Protocol
|
||||
status: raw
|
||||
status: deprecated
|
||||
category: Standards Track
|
||||
tags: logos/consensus
|
||||
tags:
|
||||
- logos/consensus
|
||||
editor: Corey Petty <corey@status.im>
|
||||
created: 01-JUL-2022
|
||||
revised: <2022-08-26 Fri 13:11Z>
|
||||
uri: <https://rdf.logos.co/protocol/Claro/1/0/0#<2022-08-26%20Fri$2013:11Z>
|
||||
contributors:
|
||||
- Álvaro Castro-Castilla
|
||||
- Mark Evenson
|
||||
- Álvaro Castro-Castilla
|
||||
- Mark Evenson
|
||||
---
|
||||
|
||||
## Abstract
|
||||
@@ -307,13 +307,27 @@ message CommunityDescription {
|
||||
|
||||
Note: The usage of the clock is described in the [Clock](#clock) section.
|
||||
|
||||
### Pubsub topic or shard usage
|
||||
### Functional scope and shard assignment
|
||||
|
||||
Status network uses static sharding as explained in [Relay Sharding](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md#static-sharding)
|
||||
We define two special [functional scopes](../raw/status-app-protocols.md#functional-scope) for messages related to Status Communities:
|
||||
|
||||
All communities by default use the default shard `32` for most of their messages and use shard `64` for a few specific messages like community event messages.
|
||||
1. Global community control
|
||||
2. Global community content
|
||||
|
||||
A community can be assigned a dedicated shard (in the range of 1-1024) after creation in which case all messages of the community are sent on that specific shard except for the community event messages.
|
||||
All messages that relate to controlling communities MUST be assigned the _global community control_ scope.
|
||||
All messages that carry user-generated content for communities MUST be assigned the _global community content_ scope.
|
||||
|
||||
> **Note:** a previous iteration of Status Communities defined separate community-wide scopes for each community.
|
||||
However, this model was deprecated and all communities now operate on a global, shared scope.
|
||||
This implies that different communities will share shards on the routing layer.
|
||||
|
||||
The following [Waku transport layer](../raw/status-app-protocols.md#waku-transport-layer) allocations are reserved for communities:
|
||||
As per [STATUS-SIMPLE-SCALING](https://rfc.vac.dev/status/raw/simple-scaling/#relay-shards), communities use the default cluster ID `16`
|
||||
set aside for all Status app protocols.
|
||||
Within this cluster, the following [shards](../raw/status-app-protocols.md#pubsub-topics-and-sharding) are reserved for the community functional scopes:
|
||||
|
||||
1. All messages with a _global community control_ scope MUST be published to shard `128`
|
||||
2. All messages with a _global community content_ scope MUST be published to shard `256`
|
||||
|
||||
### Content topic level encryption
|
||||
|
||||
|
||||
127
status/deprecated/3rd-party.md
Normal file
127
status/deprecated/3rd-party.md
Normal file
@@ -0,0 +1,127 @@
|
||||
---
|
||||
title: 3RD-PARTY
|
||||
name: 3rd party
|
||||
status: deprecated
|
||||
description: This specification discusses 3rd party APIs that Status relies on.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Volodymyr Kozieiev <volodymyr@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification discusses 3rd party APIs that Status relies on.
|
||||
These APIs provide various capabilities, including:
|
||||
|
||||
- communicating with the Ethereum network,
|
||||
- allowing users to view address and transaction details on external websites,
|
||||
- retrieving fiat/crypto exchange rates,
|
||||
- obtaining information about collectibles,
|
||||
- hosting the privacy policy.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Description |
|
||||
|-------------------|-------------------------------------------------------------------------------------------------------|
|
||||
| Fiat money | Currency established as money, often by government regulation, but without intrinsic value. |
|
||||
| Full node | A computer, connected to the Ethereum network, that enforces all Ethereum consensus rules. |
|
||||
| Crypto-collectible| A unique, non-fungible digital asset, distinct from cryptocurrencies where tokens are identical. |
|
||||
|
||||
## Why 3rd Party APIs Can Be a Problem
|
||||
|
||||
Relying on 3rd party APIs conflicts with Status’s censorship-resistance principle.
|
||||
Since Status aims to avoid suppression of information,
|
||||
it is important to minimize reliance on 3rd parties that are critical to app functionality.
|
||||
|
||||
## 3rd Party APIs Used by the Current Status App
|
||||
|
||||
### Infura
|
||||
|
||||
**What is it?**
|
||||
Infura hosts a collection of Ethereum full nodes and provides an API
|
||||
to access the Ethereum and IPFS networks without requiring a full node.
|
||||
|
||||
**How Status Uses It**
|
||||
Since Status operates on mobile devices,
|
||||
it cannot rely on a local node.
|
||||
Therefore, all Ethereum network communication happens via Infura.
|
||||
|
||||
**Concerns**
|
||||
Making an HTTP request can reveal user metadata,
|
||||
which could be exploited in attacks if Infura is compromised.
|
||||
Infura uses centralized hosting providers;
|
||||
if these providers fail or cut off service,
|
||||
Ethereum-dependent features in Status would be affected.
|
||||
|
||||
### Etherscan
|
||||
|
||||
**What is it?**
|
||||
Etherscan is a service that allows users to explore the Ethereum blockchain
|
||||
for transactions, addresses, tokens, prices,
|
||||
and other blockchain activities.
|
||||
|
||||
**How Status Uses It**
|
||||
The Status Wallet allows users to view address and transaction details on Etherscan.
|
||||
|
||||
**Concerns**
|
||||
If Etherscan becomes unavailable,
|
||||
users won’t be able to view address or transaction details through Etherscan.
|
||||
However, in-app information will still be accessible.
|
||||
|
||||
### CryptoCompare
|
||||
|
||||
**What is it?**
|
||||
CryptoCompare provides live crypto prices, charts, and analysis from major exchanges.
|
||||
|
||||
**How Status Uses It**
|
||||
Status regularly fetches crypto prices from CryptoCompare,
|
||||
using this information to calculate fiat values
|
||||
for transactions or wallet assets.
|
||||
|
||||
**Concerns**
|
||||
HTTP requests can reveal metadata,
|
||||
which could be exploited if CryptoCompare is compromised.
|
||||
If CryptoCompare becomes unavailable,
|
||||
Status won’t be able to show fiat equivalents for crypto in the wallet.
|
||||
|
||||
### Collectibles
|
||||
|
||||
Various services provide information on collectibles:
|
||||
|
||||
- [Service 1](https://api.pixura.io/graphql)
|
||||
- [Service 2](https://www.etheremon.com/api)
|
||||
- [Service 3](https://us-central1-cryptostrikers-prod.cloudfunctions.net/cards/)
|
||||
- [Service 4](https://api.cryptokitties.co/)
|
||||
|
||||
**Concerns**
|
||||
HTTP requests can reveal metadata,
|
||||
which could be exploited if these services are compromised.
|
||||
|
||||
### Iubenda
|
||||
|
||||
**What is it?**
|
||||
Iubenda helps create compliance documents for websites and apps across jurisdictions.
|
||||
|
||||
**How Status Uses It**
|
||||
Status’s privacy policy is hosted on Iubenda.
|
||||
|
||||
**Concerns**
|
||||
If Iubenda becomes unavailable,
|
||||
users will be unable to view the app's privacy policy.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Version | Comment |
|
||||
|---------|-----------------|
|
||||
| 0.1.0 | Initial release |
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via CC0.
|
||||
|
||||
## References
|
||||
|
||||
- [GraphQL](https://api.pixura.io/graphql)
|
||||
- [Etheremon](https://www.etheremon.com/api)
|
||||
- [Cryptostrikers](https://us-central1-cryptostrikers-prod.cloudfunctions.net/cards/)
|
||||
- [Cryptokitties](https://api.cryptokitties.co/)
|
||||
138
status/deprecated/IPFS-gateway-for-sticker-Pack.md
Normal file
138
status/deprecated/IPFS-gateway-for-sticker-Pack.md
Normal file
@@ -0,0 +1,138 @@
|
||||
---
|
||||
title: IPFS-gateway-for-Sticker-Pack
|
||||
name: IPFS gateway for Sticker Pack
|
||||
status: deprecated
|
||||
description: This specification describes how Status uses the IPFS gateway to store stickers.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Gheorghe Pinzaru <gheorghe@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes how Status uses the IPFS gateway
|
||||
to store stickers.
|
||||
The specification explores image format,
|
||||
how a user uploads stickers,
|
||||
and how an end user can see them inside the Status app.
|
||||
|
||||
## Definition
|
||||
|
||||
| Term | Description |
|
||||
|------------------|----------------------------------------------------------------------------------------|
|
||||
| **Stickers** | A set of images which can be used to express emotions |
|
||||
| **Sticker Pack** | ERC721 token which includes the set of stickers |
|
||||
| **IPFS** | P2P network used to store and share data, in this case, the images for the stickerpack |
|
||||
|
||||
## Specification
|
||||
|
||||
### Image format
|
||||
|
||||
Accepted image file types are `PNG`, `JPG/JPEG` and `GIF`,
|
||||
with a maximum allowed size of 300kb.
|
||||
The minimum sticker image resolution is 512x512,
|
||||
and its background SHOULD be transparent.
|
||||
|
||||
### Distribution
|
||||
|
||||
The node implements sticker packs as [ERC721 token](https://eips.ethereum.org/EIPS/eip-721)
|
||||
and contain a set of stickers.
|
||||
The node stores these stickers inside the sticker pack as a set of hyperlinks pointing to IPFS storage.
|
||||
These hyperlinks are publicly available and can be accessed by any user inside the status chat.
|
||||
Stickers can be sent in chat only by accounts that own the sticker pack.
|
||||
|
||||
### IPFS gateway
|
||||
|
||||
At the moment of writing, the current main Status app uses the [Infura](https://infura.io/) gateway.
|
||||
However, clients could choose a different gateway or to run own IPFS node.
|
||||
Infura gateway is an HTTPS gateway,
|
||||
which based on an HTTP GET request with the multihash block will return the stored content at that block address.
|
||||
|
||||
The node requires the use of a gateway to enable easy access to the resources over HTTP.
|
||||
The node stores each image of a sticker inside IPFS using a unique address that is
|
||||
derived from the hash of the file.
|
||||
This ensures that a file can't be overridden,
|
||||
and an end-user of the IPFS will receive the same file at a given address.
|
||||
|
||||
### Security
|
||||
|
||||
The IPFS gateway acts as an end-user of the IPFS
|
||||
and allows users of the gateway to access IPFS without connection to the P2P network.
|
||||
Usage of a gateway introduces potential risk for the users of that gateway provider.
|
||||
In case of a compromise in the security of the provider, meta information such as IP address,
|
||||
User-Agent and other of its users can be leaked.
|
||||
If the provider servers are unavailable the node loses access through the gateway to the IPFS network.
|
||||
|
||||
### Status sticker usage
|
||||
|
||||
When the app shows a sticker, the Status app makes an HTTP GET request to IPFS gateway using the hyperlink.
|
||||
|
||||
To send a sticker in chat, a user of Status should buy or install a sticker pack.
|
||||
|
||||
To be available for installation a Sticker Pack should be submitted to Sticker market by an author.
|
||||
|
||||
#### Submit a sticker
|
||||
|
||||
To submit a sticker pack, the author should upload all assets to IPFS.
|
||||
Then generate a payload including name, author, thumbnail,
|
||||
preview and a list of stickers in the [EDN format](https://github.com/edn-format/edn). Following this structure:
|
||||
``
|
||||
{meta {:name "Sticker pack name"
|
||||
:author "Author Name"
|
||||
:thumbnail "e30101701220602163b4f56c747333f43775fdcbe4e62d6a3e147b22aaf6097ce0143a6b2373"
|
||||
:preview "e30101701220ef54a5354b78ef82e542bd468f58804de71c8ec268da7968a1422909357f2456"
|
||||
:stickers [{:hash "e301017012207737b75367b8068e5bdd027d7b71a25138c83e155d1f0c9bc5c48ff158724495"}
|
||||
{:hash "e301017012201a9cdea03f27cda1aede7315f79579e160c7b2b6a2eb51a66e47a96f47fe5284"}]}}
|
||||
``
|
||||
All asset fields, are contenthash fields as per [EIP 1577](https://eips.ethereum.org/EIPS/eip-1577).
|
||||
The node also uploads this payload to IPFS, and the node uses the IPFS address in the content field of the Sticker Market contract.
|
||||
See [Sticker Market spec](https://github.com/status-im/sticker-market/blob/651e88e5f38c690e57ecaad47f46b9641b8b1e27/docs/specification.md) for a detailed description of the contract.
|
||||
|
||||
#### Install a sticker pack
|
||||
|
||||
To install a sticker pack, the node fetches all sticker packs available in Sticker Market.
|
||||
The node needs the following steps to fetch all sticker packs:
|
||||
|
||||
#### 1. Get total number of sticker packs
|
||||
|
||||
Call `packCount()` on the sticker market contract, will return number of sticker pack registered as `uint256`.
|
||||
|
||||
#### 2. Get sticker pack by id
|
||||
|
||||
ID's are represented as `uint256` and are incremental from `0` to total number of sticker packs in the contract,
|
||||
received in the previous step.
|
||||
To get a sticker pack call `getPackData(sticker-pack-id)`, the return type is `["bytes4[]" "address" "bool" "uint256" "uint256" "bytes"]`
|
||||
which represents the following fields: `[category owner mintable timestamp price contenthash]`.
|
||||
Price is the SNT value in wei set by sticker pack owner.
|
||||
The contenthash is the IPFS address described in the [submit description](#submit-a-sticker) above.
|
||||
Other fields specification could be found in [Sticker Market spec](https://github.com/status-im/sticker-market/blob/651e88e5f38c690e57ecaad47f46b9641b8b1e27/docs/specification.md)
|
||||
|
||||
##### 3. Get owned sticker packs
|
||||
|
||||
The current Status app fetches owned sticker packs during the open of any sticker view
|
||||
(a screen which shows a sticker pack, or the list of sticker packs).
|
||||
To get owned packs, get all owned tokens for the current account address,
|
||||
by calling `balanceOf(address)` where address is the address for the current account.
|
||||
This method returns a `uint256` representing the count of available tokens. Using `tokenOfOwnerByIndex(address,uint256)` method,
|
||||
with the address of the user and ID in form of a `uint256`
|
||||
which is an incremented int from 0 to the total number of tokens, gives the token id.
|
||||
To get the sticker pack id from a token call`tokenPackId(uint256)` where `uint256` is the token id.
|
||||
This method will return an `uint256` which is the id of the owned sticker pack.
|
||||
|
||||
##### 4. Buy a sticker pack
|
||||
|
||||
To buy a sticker pack call `approveAndCall(address,uint256,bytes)`
|
||||
where `address` is the address of buyer,`uint256` is the price and third parameters `bytes` is the callback called if approved.
|
||||
In the callback, call `buyToken(uint256,address,uint256)`, first parameter is sticker pack id, second buyers address, and the last is the price.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [ERC721 Token Standard](https://eips.ethereum.org/EIPS/eip-721)
|
||||
- [Infura](https://infura.io/)
|
||||
- [EDN Format](https://github.com/edn-format/edn)
|
||||
- [EIP 1577](https://eips.ethereum.org/EIPS/eip-1577)
|
||||
- [Sticker Market Specification](https://github.com/status-im/sticker-market/blob/651e88e5f38c690e57ecaad47f46b9641b8b1e27/docs/specification.md)
|
||||
461
status/deprecated/account.md
Normal file
461
status/deprecated/account.md
Normal file
@@ -0,0 +1,461 @@
|
||||
---
|
||||
title: ACCOUNT
|
||||
name: Account
|
||||
status: deprecated
|
||||
description: This specification explains what a Status account is, and how a node establishes trust.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Samuel Hawksby-Robinson <samuel@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification explains what a Status account is,
|
||||
and how a node establishes trust.
|
||||
|
||||
## Introduction
|
||||
|
||||
The core concept of an account in Status is a set of cryptographic keypairs.
|
||||
Namely, the combination of the following:
|
||||
|
||||
1. a Whisper/Waku chat identity keypair
|
||||
1. a set of cryptocurrency wallet keypairs
|
||||
|
||||
The node verifies or derives everything else associated with the contact from the above items, including:
|
||||
|
||||
- Ethereum address (future verification, currently the same base keypair)
|
||||
- 3 word mnemonic name
|
||||
- identicon
|
||||
- message signatures
|
||||
|
||||
## Initial Key Generation
|
||||
|
||||
### Public/Private Keypairs
|
||||
|
||||
- An ECDSA (secp256k1 curve) public/private keypair MUST be generated via a [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki) derived path from a [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) mnemonic seed phrase.
|
||||
- The default paths are defined as such:
|
||||
- Whisper/Waku Chat Key (`IK`): `m/43'/60'/1581'/0'/0` (post Multiaccount integration)
|
||||
- following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
|
||||
<!-- WE CURRENTLY DO NOT IMPLEMENT ENCRYPTION KEY, FOR FUTURE - C.P. -->
|
||||
<!-- - DB encryption Key (`DBK`): `m/43'/60'/1581'/1'/0` (post Multiaccount integration) -->
|
||||
<!-- - following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md) -->
|
||||
- Status Wallet paths: `m/44'/60'/0'/0/i` starting at `i=0`
|
||||
- following [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
|
||||
- NOTE: this (`i=0`) is also the current (and only) path for Whisper/Waku key before Multiaccount integration
|
||||
|
||||
### X3DH Prekey bundle creation
|
||||
|
||||
- Status follows the X3DH prekey bundle scheme that [Open Whisper Systems](https://en.wikipedia.org/wiki/Signal_Messenger#2013%E2%80%932018:_Open_Whisper_Systems) (not to be confused with the Whisper sub-protocol) outlines [in their documentation](https://signal.org/docs/specifications/x3dh/#the-x3dh-protocol) with the following exceptions:
|
||||
|
||||
- Status does not publish one-time keys `OPK` or perform DH including them, because there are no central servers in the Status implementation.
|
||||
- A client MUST create X3DH prekey bundles, each defined by the following items:
|
||||
- Identity Key: `IK`
|
||||
- Signed prekey: `SPK`
|
||||
- Prekey signature: `Sig(IK, Encode(SPK))`
|
||||
- Timestamp
|
||||
- These bundles are made available in a variety of ways, as defined in section 2.1.
|
||||
|
||||
## Account Broadcasting
|
||||
|
||||
- A user is responsible for broadcasting certain information publicly so that others may contact them.
|
||||
|
||||
### X3DH Prekey bundles
|
||||
|
||||
- A client SHOULD regenerate a new X3DH prekey bundle every 24 hours. This MAY be done in a lazy way, such that a client that does not come online past this time period does not regenerate or broadcast bundles.
|
||||
- The current bundle SHOULD be broadcast on a Whisper/Waku topic specific to his Identity Key, `{IK}-contact-code`, intermittently. This MAY be done every 6 hours.
|
||||
- A bundle SHOULD accompany every message sent.
|
||||
- TODO: retrieval of long-time offline users bundle via `{IK}-contact-code`
|
||||
|
||||
## Optional Account additions
|
||||
|
||||
### ENS Username
|
||||
|
||||
- A user MAY register a public username on the Ethereum Name System (ENS). This username is a user-chosen subdomain of the `stateofus.eth` ENS registration that maps to their Whisper/Waku identity key (`IK`).
|
||||
|
||||
<!-- ### User Profile Picture
|
||||
- An account MAY edit the `IK` generated identicon with a chosen picture. This picture will become part of the publicly broadcast profile of the account. -->
|
||||
|
||||
<!-- TODO: Elaborate on wallet account and multiaccount -->
|
||||
<!-- TODO: Elaborate on security implications -->
|
||||
|
||||
## Trust establishment
|
||||
|
||||
**Trust establishment deals with users verifying they are communicating with who they think they are.**
|
||||
|
||||
### Terms Glossary
|
||||
|
||||
| term | description |
|
||||
| ------------------------- | ----------- |
|
||||
| privkey | ECDSA secp256k1 private key |
|
||||
| pubkey | ECDSA secp256k1 public key |
|
||||
| Whisper/Waku key | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
|
||||
|
||||
### Contact Discovery
|
||||
|
||||
#### Public channels
|
||||
|
||||
- Public group channels in Status are a broadcast/subscription system. All public messages are encrypted with a symmetric key derived from the channel name, `K_{pub,sym}`, which is publicly known.
|
||||
- A public group channel's symmetric key MUST creation must follow the [web3 API](https://web3js.readthedocs.io/en/1.0/web3-shh.html#generatesymkeyfrompassword)'s `web3.ssh.generateSymKeyFromPassword` function
|
||||
- In order to post to a public group channel, a client MUST have a valid account created.
|
||||
- In order to listen to a public group channel, a client must subscribe to the channel name.
|
||||
The sender of a message is derived from the message's signature.
|
||||
- Discovery of channel names is not currently part of the protocol, and is typically done out of band.
|
||||
If a channel name is used that has not been used, it will be created.
|
||||
- A client MUST sign the message otherwise it will be discarded by the recipients.
|
||||
- channel name specification:
|
||||
- matches `[a-z0-9\-]`
|
||||
- is not a public key
|
||||
|
||||
#### Private 1:1 messages
|
||||
|
||||
This can be done in the following ways:
|
||||
|
||||
1. scanning a user generated QR code
|
||||
1. discovery through the Status app
|
||||
1. asynchronous X3DH key exchange
|
||||
1. public key via public channel listening
|
||||
- `status-mobile/src/status_im/contact_code/core.cljs`
|
||||
1. contact codes
|
||||
1. decentralized storage (not implemented)
|
||||
1. Whisper/Waku
|
||||
|
||||
### Initial Key Exchange
|
||||
|
||||
#### Bundles
|
||||
|
||||
- An X3DH prekey bundle is defined as ([code](https://github.com/status-im/status-go/messaging/chat/protobuf/encryption.pb.go)):
|
||||
|
||||
```golang
|
||||
Identity // Identity key
|
||||
SignedPreKeys // a map of installation id to array of signed prekeys by that installation id
|
||||
Signature // Prekey signature
|
||||
Timestamp // When the bundle was lasted created locally
|
||||
```
|
||||
|
||||
- include BundleContainer
|
||||
- a new bundle SHOULD be created at least every 12 hours
|
||||
- a node only generates a bundle when it is used
|
||||
- a bundle SHOULD be distributed on the contact code channel. This is the Whisper and Waku topic `{IK}-contact-code`,
|
||||
where `IK` is the hex encoded public key of the user, prefixed with `0x`.
|
||||
The node encrypts the channel in the same way it encrypted public chats.
|
||||
|
||||
### Contact Verification
|
||||
|
||||
To verify that contact key information is as it should be, use the following.
|
||||
|
||||
#### Identicon
|
||||
|
||||
A low-poly identicon is deterministically generated from the Whisper/Waku chat public key.
|
||||
This can be compared out of band to ensure the receiver's public key is the one stored locally.
|
||||
|
||||
#### 3 word pseudonym / Whisper/Waku key fingerprint
|
||||
|
||||
Status generates a deterministic 3-word random pseudonym from the Whisper/Waku chat public key.
|
||||
This pseudonym acts as a human readable fingerprint to the Whisper/Waku chat public key.
|
||||
This name also shows when viewing a contact's public profile and in the chat UI.
|
||||
|
||||
- implementation: [gfycat](https://github.com/status-im/status-mobile/tree/develop/src/status_im/utils/gfycat)
|
||||
|
||||
#### ENS name
|
||||
|
||||
Status offers the ability to register a mapping of a human readable subdomain of `stateofus.eth` to their Whisper/Waku chat public key.
|
||||
The user purchases this registration (currently by staking 10 SNT)
|
||||
and the node stores it on the Ethereum mainnet blockchain for public lookup.
|
||||
|
||||
<!-- TODO: Elaborate on security implications -->
|
||||
|
||||
<!-- TODO: Incorporate or cut below into proper spec
|
||||
|
||||
### Possible Connection Breakdown
|
||||
|
||||
possible connections
|
||||
- client - client (not really ever, this is facilitated through all other connections)
|
||||
- personal chat
|
||||
- ratcheted with X3DH
|
||||
- private group chat
|
||||
- pairwise ratcheted with X3DH
|
||||
- public chat
|
||||
- client - mailserver (statusd + ???)
|
||||
- a mailserver identifies itself by an [enode address](https://github.com/ethereum/wiki/wiki/enode-url-format)
|
||||
- client - Whisper/Waku node (statusd)
|
||||
- a node identifies itself by an enode address
|
||||
- client - bootnode (go-ethereum)
|
||||
- a bootnode identifies itself by
|
||||
- an enode address
|
||||
- `NOTE: redezvous information here`
|
||||
- client - ENS registry (ethereum blockchain -> default to infura)
|
||||
- client - Ethereum RPC (custom go-ethereum RPC API -> default to infura API)
|
||||
- client - IPFS (Status hosted IPFS gateway -> defaults to ???)
|
||||
- we have a status hosted IPFS gateway for pinning but it currently isn't used much.
|
||||
|
||||
### Notes
|
||||
|
||||
A user in the system is a public-private key pair using the Elliptic-Curve Cryptography secp256k1 that Ethereum uses.
|
||||
- A 3-word random name is derived from the public key using the following package
|
||||
- `NOTE: need to find package`
|
||||
- This provides an associated human-readble fingerprint to the user's public key
|
||||
- A user can optionally add additional layers on top of this keypair
|
||||
- Chosen username
|
||||
- ENS username
|
||||
|
||||
All messages sent are encrypted with the public key of the destination and signed by the private key of the given user using the following scheme:
|
||||
- private chat
|
||||
- X3DH is used to define shared secrets which is then double ratcheted
|
||||
- private group chat
|
||||
- considered pairwise private chats
|
||||
- public group chat
|
||||
- the message is encrypted with a symmetric key derived from the chat name
|
||||
|
||||
-->
|
||||
|
||||
## Public Key Serialization
|
||||
|
||||
Idiomatically known as "public key compression" and "public key decompression".
|
||||
|
||||
The node SHOULD provide functionality for the serialization and deserialization of public / chat keys.
|
||||
|
||||
For maximum flexibility, when implementing this functionality, the node MUST support public keys encoded in a range of encoding formats, detailed below.
|
||||
|
||||
### Basic Serialization Example
|
||||
|
||||
In the example of a typical hexadecimal encoded elliptical curve (EC) public key (such as a secp256k1 pk),
|
||||
|
||||
```text
|
||||
0x04261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
|
||||
```
|
||||
|
||||
minor modification for compatibility and flexibility makes the key self-identifiable and easily parsable,
|
||||
|
||||
```text
|
||||
fe70104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
|
||||
```
|
||||
|
||||
EC serialization and compact encoding produces a much smaller string representation of the original key.
|
||||
|
||||
```text
|
||||
zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB
|
||||
```
|
||||
|
||||
### Public Key "Compression" Rationale
|
||||
|
||||
Serialized and compactly encoded ("compressed") public keys have a number of UI / UX advantages
|
||||
over non-serialized less densely encoded public keys.
|
||||
|
||||
Compressed public keys are smaller, and users may perceive them as less intimidating and less unnecessarily large.
|
||||
Compare the "compressed" and "uncompressed" version of the same public key from above example:
|
||||
|
||||
- `0xe70104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6`
|
||||
- `zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB`
|
||||
|
||||
The user can transmit and share the same data, but at one third of the original size.
|
||||
136 characters uncompressed vs 49 characters compressed, giving a significant character length reduction of 64%.
|
||||
|
||||
The user client app MAY use the compressed public keys throughout the user interface.
|
||||
For example in the `status-mobile` implementation of the user interface
|
||||
the following places could take advantage of a significantly smaller public key:
|
||||
|
||||
- `Onboarding` > `Choose a chat name`
|
||||
- `Profile` > `Header`
|
||||
- `Profile` > `Share icon` > `QR code popover`
|
||||
- `Invite friends` url from `Invite friends` button and `+ -button` > `Invite friends`
|
||||
- Other user `Profile details`
|
||||
- `Profile details` > `Share icon` > `QR code popover`
|
||||
|
||||
In the case of QR codes a compressed public key can reduce the complexity of the derived codes:
|
||||
|
||||
| Uncompressed |
|
||||
| --- |
|
||||
| |
|
||||
|
||||
| Compressed |
|
||||
| --- |
|
||||
| |
|
||||
|
||||
### Key Encoding
|
||||
|
||||
When implementing the pk de/serialization functionality, the node MUST use the [multiformats/multibase](https://github.com/multiformats/multibase)
|
||||
encoding protocol to interpret incoming key data and to return key data in a desired encoding.
|
||||
|
||||
The node SHOULD support the following `multibase` encoding formats.
|
||||
|
||||
```csv
|
||||
encoding, code, description, status
|
||||
identity, 0x00, 8-bit binary (encoder and decoder keeps data unmodified), default
|
||||
base2, 0, binary (01010101), candidate
|
||||
base8, 7, octal, draft
|
||||
base10, 9, decimal, draft
|
||||
base16, f, hexadecimal, default
|
||||
base16upper, F, hexadecimal, default
|
||||
base32hex, v, rfc4648 case-insensitive - no padding - highest char, candidate
|
||||
base32hexupper, V, rfc4648 case-insensitive - no padding - highest char, candidate
|
||||
base32hexpad, t, rfc4648 case-insensitive - with padding, candidate
|
||||
base32hexpadupper, T, rfc4648 case-insensitive - with padding, candidate
|
||||
base32, b, rfc4648 case-insensitive - no padding, default
|
||||
base32upper, B, rfc4648 case-insensitive - no padding, default
|
||||
base32pad, c, rfc4648 case-insensitive - with padding, candidate
|
||||
base32padupper, C, rfc4648 case-insensitive - with padding, candidate
|
||||
base32z, h, z-base-32 (used by Tahoe-LAFS), draft
|
||||
base36, k, base36 [0-9a-z] case-insensitive - no padding, draft
|
||||
base36upper, K, base36 [0-9a-z] case-insensitive - no padding, draft
|
||||
base58btc, z, base58 bitcoin, default
|
||||
base58flickr, Z, base58 flicker, candidate
|
||||
base64, m, rfc4648 no padding, default
|
||||
base64pad, M, rfc4648 with padding - MIME encoding, candidate
|
||||
base64url, u, rfc4648 no padding, default
|
||||
base64urlpad, U, rfc4648 with padding, default
|
||||
```
|
||||
|
||||
**Note** this specification RECOMMENDs that implementations extend the standard `multibase` protocol
|
||||
to parse strings prepended with `0x` as `f` hexadecimal encoded bytes.
|
||||
|
||||
Implementing this recommendation will allow the node to correctly interpret traditionally identified hexadecimal strings (e.g. `0x1337c0de`).
|
||||
|
||||
*Example:*
|
||||
|
||||
`0xe70102261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc`
|
||||
|
||||
SHOULD be interpreted as
|
||||
|
||||
`fe70102261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc`
|
||||
|
||||
This specification RECOMMENDs that the consuming service of the node uses a compact encoding type,
|
||||
such as base64 or base58 to allow for as short representations of the key as possible.
|
||||
|
||||
### Public Key Types
|
||||
|
||||
When implementing the pk de/serialization functionality, The node MUST support the [multiformats/multicodec](https://github.com/multiformats/multicodec) key type identifiers for the following public key type.
|
||||
|
||||
| Name | Tag | Code | Description |
|
||||
| ------------------ | --- | ------ | ------------------------------------ |
|
||||
| `secp256k1-pub` | key | `0xe7` | Secp256k1 public key |
|
||||
|
||||
For a public key to be identifiable to the node the public key data MUST be prepended with the relevant [multiformats/unsigned-varint](https://github.com/multiformats/unsigned-varint) formatted code.
|
||||
|
||||
*Example:*
|
||||
|
||||
Below is a representation of an deserialized secp256k1 public key.
|
||||
|
||||
```text
|
||||
04
|
||||
26 | 1c | 55 | 67 | 5e | 55 | ff | 25
|
||||
ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
|
||||
35 | f6 | 07 | 12 | d2 | 51 | cb | aa
|
||||
ab | 97 | bd | 50 | 05 | 4c | 6e | bc
|
||||
3c | d4 | e2 | 22 | 00 | c6 | 8d | af
|
||||
74 | 93 | e1 | f8 | da | 6a | 19 | 0a
|
||||
68 | a6 | 71 | e2 | d3 | 97 | 78 | 09
|
||||
61 | 24 | 24 | c7 | c3 | 88 | 8b | c6
|
||||
```
|
||||
|
||||
The `multicodec` code for a secp256k1 public key is `0xe7`.
|
||||
|
||||
After parsing the code `0xe7` as a `multiformats/uvarint`, the byte value is `0xe7 0x01`, prepending this to the public key results in the below representation.
|
||||
|
||||
```text
|
||||
e7 | 01 | 04
|
||||
26 | 1c | 55 | 67 | 5e | 55 | ff | 25
|
||||
ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
|
||||
35 | f6 | 07 | 12 | d2 | 51 | cb | aa
|
||||
ab | 97 | bd | 50 | 05 | 4c | 6e | bc
|
||||
3c | d4 | e2 | 22 | 00 | c6 | 8d | af
|
||||
74 | 93 | e1 | f8 | da | 6a | 19 | 0a
|
||||
68 | a6 | 71 | e2 | d3 | 97 | 78 | 09
|
||||
61 | 24 | 24 | c7 | c3 | 88 | 8b | c6
|
||||
```
|
||||
|
||||
### De/Serialization Process Flow
|
||||
|
||||
When implementing the pk de/serialization functionality, the node MUST be passed a `multicodec` identified public key,
|
||||
of the above supported types, encoded with a valid `multibase` identifier.
|
||||
|
||||
This specification RECOMMENDs that the node also accept an encoding type parameter to encode the output data.
|
||||
This provides for the case where the user requires the de/serialization key to be in a different encoding to the encoding of the given key.
|
||||
|
||||
#### Serialization Example
|
||||
|
||||
A hexadecimal encoded secp256k1 public chat key typically is represented as below:
|
||||
|
||||
```text
|
||||
0x04261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
|
||||
```
|
||||
|
||||
To be properly interpreted by the node for serialization the public key MUST be prepended with the `multicodec` `uvarint` code `0xea 0x01`
|
||||
and encoded with a valid `multibase` encoding, therefore giving the following:
|
||||
|
||||
```text
|
||||
fea0104261c55675e55ff25edb50b345cfb3a3f35f60712d251cbaaab97bd50054c6ebc3cd4e22200c68daf7493e1f8da6a190a68a671e2d3977809612424c7c3888bc6
|
||||
```
|
||||
|
||||
If adhering to the specification recommendation to provide the user with an output encoding parameter,
|
||||
the above string would be passed to the node with the following `multibase` encoding identifier.
|
||||
|
||||
In this example the output encoding is defined as `base58 bitcoin`.
|
||||
|
||||
```text
|
||||
z
|
||||
```
|
||||
|
||||
The return value in this case would be
|
||||
|
||||
```text
|
||||
zQ3shPyZJnxZK4Bwyx9QsaksNKDYTPmpwPvGSjMYVHoXHeEgB
|
||||
```
|
||||
|
||||
Which after `multibase` decoding can be represented in bytes as below:
|
||||
|
||||
```text
|
||||
e7 | 01 | 02
|
||||
26 | 1c | 55 | 67 | 5e | 55 | ff | 25
|
||||
ed | b5 | 0b | 34 | 5c | fb | 3a | 3f
|
||||
35 | f6 | 07 | 12 | d2 | 51 | cb | aa
|
||||
ab | 97 | bd | 50 | 05 | 4c | 6e | bc
|
||||
```
|
||||
|
||||
#### Deserialization Example
|
||||
|
||||
For the user, the deserialization process is exactly the same as serialization with the exception
|
||||
that the user MUST provide a serialized public key for deserialization. Else the deserialization algorithm will fail.
|
||||
|
||||
For further guidance on the implementation of public key de/serialization consult the [`status-go` implementation and tests](https://github.com/status-im/status-go/blob/c9772325f2dca76b3504191c53313663ca2efbe5/api/utils_test.go).
|
||||
|
||||
## Security Considerations
|
||||
|
||||
-
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.4
|
||||
|
||||
Released [June 24, 2020](https://github.com/status-im/specs/commit/e98a9b76b7d4e1ce93e0b692e1521c2d54f72c59)
|
||||
|
||||
- Added details of public key serialization and deserialization
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
- Added language to include Waku in all relevant places
|
||||
- Change to keep `Mailserver` term consistent
|
||||
- Added clarification to Open Whisper Systems
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [BIP43](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
|
||||
- [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
|
||||
- [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
|
||||
- [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
|
||||
- [Open Whisper Systems](https://en.wikipedia.org/wiki/Signal_Messenger#2013%E2%80%932018:_Open_Whisper_Systems)
|
||||
- [X3DH](https://signal.org/docs/specifications/x3dh/#the-x3dh-protocol)
|
||||
- [web3 API](https://web3js.readthedocs.io/en/1.0/web3-shh.html#generatesymkeyfrompassword)
|
||||
- [Protobuf encryption](https://github.com/status-im/status-go/messaging/chat/protobuf/encryption.pb.go)
|
||||
- [gfycat in Status](https://github.com/status-im/status-mobile/tree/develop/src/status_im/utils/gfycat)
|
||||
- [multiformats](https://github.com/multiformats/)
|
||||
- [status-go implementation and tests](https://github.com/status-im/status-go/blob/c9772325f2dca76b3504191c53313663ca2efbe5/api/utils_test.go)
|
||||
- [June 24, 2020 change commit](https://github.com/status-im/specs/commit/e98a9b76b7d4e1ce93e0b692e1521c2d54f72c59)
|
||||
- [May 22, 2020 change commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
427
status/deprecated/client.md
Normal file
427
status/deprecated/client.md
Normal file
@@ -0,0 +1,427 @@
|
||||
---
|
||||
title: CLIENT
|
||||
name: Client
|
||||
status: deprecated
|
||||
description: This specification describes how to write a Status client for communicating with other Status clients.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Samuel Hawksby-Robinson <samuel@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes how to write a Status client for communicating
|
||||
with other Status clients.
|
||||
This specification presents a reference implementation of the protocol
|
||||
used in a command-line client and a mobile app.
|
||||
|
||||
This document consists of two parts.
|
||||
The first outlines the specifications required to be a full Status client.
|
||||
The second provides a design rationale and answers some common questions.
|
||||
|
||||
## Introduction
|
||||
|
||||
### Protocol layers
|
||||
|
||||
Implementing a Status clients largely means implementing the following layers.
|
||||
Additionally, there are separate specifications for things like key management and account lifecycle.
|
||||
|
||||
Other aspects, such as how a node uses IPFS for stickers or how the browser works, are currently underspecified.
|
||||
These specifications facilitate the implementation of a Status client for basic private communication.
|
||||
|
||||
| Layer | Purpose | Technology |
|
||||
| ----------------- | ------------------------------ | ---------------------------- |
|
||||
| Data and payloads | End user functionality | 1:1, group chat, public chat |
|
||||
| Data sync | Data consistency | MVDS. |
|
||||
| Secure transport | Confidentiality, PFS, etc | Double Ratchet |
|
||||
| Transport privacy | Routing, Metadata protection | Waku / Whisper |
|
||||
| P2P Overlay | Overlay routing, NAT traversal | devp2p |
|
||||
|
||||
### Protobuf
|
||||
|
||||
[`protobuf`](https://developers.google.com/protocol-buffers/) is used in different layers, version `proto3` used is unless stated otherwise.
|
||||
|
||||
## Components
|
||||
|
||||
### P2P Overlay
|
||||
|
||||
Status clients run on a public, permissionless peer-to-peer network, as specified by the devP2P
|
||||
network protocols. devP2P provides a protocol for node discovery which is in
|
||||
draft mode
|
||||
[here](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md). See
|
||||
more on node discovery and management in the next section.
|
||||
|
||||
To communicate between Status nodes, the [RLPx Transport
|
||||
Protocol, v5](https://github.com/ethereum/devp2p/blob/master/rlpx.md) is used, which
|
||||
allows for TCP-based communication between nodes.
|
||||
|
||||
On top of this RLPx-based subprotocols are ran, the client
|
||||
SHOULD NOT use [Whisper V6](https://eips.ethereum.org/EIPS/eip-627), the client
|
||||
SHOULD use [Waku V1](/waku/standards/legacy/6/waku1.md)
|
||||
for privacy-preserving messaging and efficient usage of a node's bandwidth.
|
||||
|
||||
#### Node discovery and roles
|
||||
|
||||
There are four types of node roles:
|
||||
|
||||
1. `Bootstrap node`
|
||||
1. `Whisper/Waku relayer`
|
||||
1. `Mailserver` (servers and clients)
|
||||
1. `Mobile node` (Status Clients)
|
||||
|
||||
A standard Status client MUST implement both `Whisper/Waku relayer` and `Mobile node` node types. The
|
||||
other node types are optional, but it is RECOMMEND to implement a `Mailserver`
|
||||
client mode, otherwise the user experience is likely to be poor.
|
||||
|
||||
#### Bootstrapping
|
||||
|
||||
Bootstrap nodes allow Status nodes to discover and connect to other Status nodes
|
||||
in the network.
|
||||
|
||||
Currently, Status Gmbh provides the main bootstrap nodes, but anyone can
|
||||
run these provided they are connected to the rest of the Whisper/Waku network.
|
||||
|
||||
Status maintains a list of production fleet bootstrap nodes in the following locations:
|
||||
|
||||
**Hong Kong:**
|
||||
|
||||
- `enode://6e6554fb3034b211398fcd0f0082cbb6bd13619e1a7e76ba66e1809aaa0c5f1ac53c9ae79cf2fd4a7bacb10d12010899b370c75fed19b991d9c0cdd02891abad@47.75.99.169:443`
|
||||
- `enode://23d0740b11919358625d79d4cac7d50a34d79e9c69e16831c5c70573757a1f5d7d884510bc595d7ee4da3c1508adf87bbc9e9260d804ef03f8c1e37f2fb2fc69@47.52.106.107:443`
|
||||
|
||||
**Amsterdam:**
|
||||
|
||||
- `enode://436cc6f674928fdc9a9f7990f2944002b685d1c37f025c1be425185b5b1f0900feaf1ccc2a6130268f9901be4a7d252f37302c8335a2c1a62736e9232691cc3a@178.128.138.128:443`
|
||||
- `enode://5395aab7833f1ecb671b59bf0521cf20224fe8162fc3d2675de4ee4d5636a75ec32d13268fc184df8d1ddfa803943906882da62a4df42d4fccf6d17808156a87@178.128.140.188:443`
|
||||
|
||||
**Central US:**
|
||||
|
||||
- `enode://32ff6d88760b0947a3dee54ceff4d8d7f0b4c023c6dad34568615fcae89e26cc2753f28f12485a4116c977be937a72665116596265aa0736b53d46b27446296a@34.70.75.208:443`
|
||||
- `enode://5405c509df683c962e7c9470b251bb679dd6978f82d5b469f1f6c64d11d50fbd5dd9f7801c6ad51f3b20a5f6c7ffe248cc9ab223f8bcbaeaf14bb1c0ef295fd0@35.223.215.156:443`
|
||||
|
||||
These bootstrap nodes MAY change and are not guaranteed to stay this way forever
|
||||
and at some point circumstances might force them to change.
|
||||
|
||||
#### Discovery
|
||||
|
||||
A Status client MUST discover or have a list of peers to connect to. Status uses a
|
||||
light discovery mechanism based on a combination of [Discovery v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) and
|
||||
[Rendezvous Protocol](https://github.com/libp2p/specs/tree/master/rendezvous),
|
||||
(with some [modifications](https://github.com/status-im/rendezvous#differences-with-original-rendezvous)).
|
||||
Additionally, some static nodes MAY also be used.
|
||||
|
||||
A Status client MUST use at least one discovery method or use static nodes
|
||||
to communicate with other clients.
|
||||
|
||||
Discovery V5 uses bootstrap nodes to discover other peers. Bootstrap nodes MUST support
|
||||
Discovery V5 protocol as well in order to provide peers. It is kademlia-based discovery mechanism
|
||||
and it might consume significant (at least on mobile) amount of network traffic to operate.
|
||||
|
||||
In order to take advantage from simpler and more mobile-friendly peers discovery mechanism,
|
||||
i.e. Rendezvous protocol, one MUST provide a list of Rendezvous nodes which speak
|
||||
Rendezvous protocol. Rendezvous protocol is request-response discovery mechanism.
|
||||
It uses Ethereum Node Records (ENR) to report discovered peers.
|
||||
|
||||
Both peers discovery mechanisms use topics to provide peers with certain capabilities.
|
||||
There is no point in returning peers that do not support a particular protocol.
|
||||
Status nodes that want to be discovered MUST register to Discovery V5 and/or Rendezvous
|
||||
with the `whisper` topic. Status nodes that are `Mailservers` and want to
|
||||
be discoverable MUST additionally register with the `whispermail` topic.
|
||||
|
||||
It is RECOMMENDED to use both mechanisms but at the same time implement a structure
|
||||
called `PeerPool`. `PeerPool` is responsible for maintaining an optimal number of peers.
|
||||
For mobile nodes, there is no significant advantage to have more than 2-3 peers and one `Mailserver`.
|
||||
`PeerPool` can notify peers discovery protocol implementations that they should suspend
|
||||
their execution because the optimal number of peers is found. They should resume
|
||||
if the number of connected peers drops or a `Mailserver` disconnects.
|
||||
|
||||
It is worth noticing that an efficient caching strategy MAY be of great use, especially,
|
||||
on mobile devices. Discovered peers can be cached as they rarely change and used
|
||||
when the client starts again. In such a case, there might be no need to even start
|
||||
peers discovery protocols because cached peers will satisfy the optimal number of peers.
|
||||
|
||||
Alternatively, a client MAY rely exclusively on a list of static peers. This is the most efficient
|
||||
way because there are no peers discovery algorithm overhead introduced. The disadvantage
|
||||
is that these peers might be gone and without peers discovery mechanism, it won't be possible to find
|
||||
new ones.
|
||||
|
||||
The current list of static peers is published on <https://fleets.status.im/>. `eth.prod` is the current
|
||||
group of peers the official Status client uses. The others are test networks.
|
||||
|
||||
Finally, Waku node addresses can be retrieved by traversing
|
||||
the merkle tree found at [`fleets.status.im`](https://fleets.status.im), as described in [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459#client-protocol).
|
||||
|
||||
#### Mobile nodes
|
||||
|
||||
A `Mobile node` is a Whisper and/or Waku node which connects to part of the respective Whisper
|
||||
and/or Waku network(s). A `Mobile node` MAY relay messages. See next section for more details on how
|
||||
to use Whisper and/or Waku to communicate with other Status nodes.
|
||||
|
||||
### Transport privacy and Whisper / Waku usage
|
||||
|
||||
Once a Whisper and/or Waku node is up and running there are some specific settings required
|
||||
to communicate with other Status nodes.
|
||||
|
||||
See [WHISPER-USAGE](/status/deprecated/whisper-usage.md) and [WAKU-USAGE](/status/deprecated/waku-usage.md) for more details.
|
||||
|
||||
For providing an offline inbox, see the complementary [WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md) and [WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md).
|
||||
|
||||
### Secure Transport
|
||||
|
||||
In order to provide confidentiality, integrity, authentication and forward
|
||||
secrecy of messages the node implements a secure transport on top of Whisper and Waku. This is
|
||||
used in 1:1 chats and group chats, but not for public chats. See [SECURE-TRANSPORT](/status/deprecated/secure-transport.md) for more.
|
||||
|
||||
### Data Sync
|
||||
|
||||
[MVDS](/vac/2/mvds.md) is used for 1:1 and group chats, however it is currently not in use for public chats.
|
||||
[Status payloads](#payloads-and-clients) are serialized and then wrapped inside an
|
||||
MVDS message which is added to an [MVDS payload](/vac/2/mvds.md#payloads),
|
||||
the node encrypts this payload (if necessary for 1-to-1 / group-chats) and sends it using
|
||||
Whisper or Waku which also encrypts it.
|
||||
|
||||
### Payloads and clients
|
||||
|
||||
On top of secure transport, various types of data sync clients and
|
||||
the node uses payload formats for things like 1:1 chat, group chat and public chat. These have
|
||||
various degrees of standardization. Please refer to [PAYLOADS](/status/deprecated/payloads.md) for more details.
|
||||
|
||||
### BIPs and EIPs Standards support
|
||||
|
||||
For a list of EIPs and BIPs that SHOULD be supported by Status client, please
|
||||
see [EIPS](/status/deprecated/eips.md).
|
||||
|
||||
## Security Considerations
|
||||
|
||||
See [Appendix A](#appendix-a-security-considerations)
|
||||
|
||||
## Design Rationale
|
||||
|
||||
P2P Overlay
|
||||
|
||||
### Why devp2p? Why not use libp2p?
|
||||
|
||||
At the time Status developed the main Status clients, devp2p was the most
|
||||
mature. However, in the future libp2p is likely to be used, as it'll
|
||||
provide us with multiple transports, better protocol negotiation, NAT traversal,
|
||||
etc.
|
||||
|
||||
For very experimental bridge support, see the bridge between libp2p and devp2p
|
||||
in [Murmur](https://github.com/status-im/murmur).
|
||||
|
||||
### What about other RLPx subprotocols like LES, and Swarm?
|
||||
|
||||
Status is primarily optimized for resource restricted devices, and at present
|
||||
time light client support for these protocols are suboptimal. This is a work in
|
||||
progress.
|
||||
|
||||
For better Ethereum light client support, see [Re-enable LES as
|
||||
option](https://github.com/status-im/status-go/issues/1025). For better Swarm
|
||||
support, see [Swarm adaptive
|
||||
nodes](https://github.com/ethersphere/SWIPs/pull/12).
|
||||
|
||||
For transaction support, Status clients currently have to rely on Infura.
|
||||
|
||||
Status clients currently do not offer native support for file storage.
|
||||
|
||||
### Why do you use Whisper?
|
||||
|
||||
Whisper is one of the [three parts](http://gavwood.com/dappsweb3.html) of the
|
||||
vision of Ethereum as the world computer, Ethereum and Swarm being the other
|
||||
two. Status was started as an encapsulation of and a clear window to this world
|
||||
computer.
|
||||
|
||||
### Why do you use Waku?
|
||||
|
||||
Waku is a direct upgrade and replacement for Whisper, the main motivation for
|
||||
developing and implementing Waku can be found in the [Waku specs](/waku/).
|
||||
|
||||
>Waku was created to incrementally improve in areas that Whisper is lacking in,
|
||||
>with special attention to resource restricted devices. We specify the standard for
|
||||
>Waku messages in order to ensure forward compatibility of different Waku clients,
|
||||
>backwards compatibility with Whisper clients, as well as to allow multiple
|
||||
>implementations of Waku and its capabilities. We also modify the language to be more
|
||||
>unambiguous, concise and consistent.
|
||||
|
||||
Considerable work has gone into the active development of Ethereum, in contrast Whisper
|
||||
is not currently under active development, and it has several drawbacks. Among others:
|
||||
|
||||
- Whisper is very wasteful bandwidth-wise and doesn't appear to be scalable
|
||||
- Proof of work is a poor spam protection mechanism for heterogeneous devices
|
||||
- The privacy guarantees provided are not rigorous
|
||||
- There are no incentives to run a node
|
||||
|
||||
Finding a more suitable transport privacy is an ongoing research effort,
|
||||
together with [Vac](https://vac.dev/vac-overview) and other teams in the space.
|
||||
|
||||
### Why is PoW for Waku set so low?
|
||||
|
||||
A higher PoW would be desirable, but this kills the battery on mobile phones,
|
||||
which is a prime target for Status clients.
|
||||
|
||||
This means the network is currently vulnerable to DDoS attacks. Alternative
|
||||
methods of spam protection are currently being researched.
|
||||
|
||||
### Why do you not use Discovery v5 for node discovery?
|
||||
|
||||
At the time of implementing dynamic node discovery, Discovery v5 wasn't completed
|
||||
yet. Additionally, running a DHT on a mobile leads to slow node discovery, bad
|
||||
battery and poor bandwidth usage. Instead, each client can choose to turn on
|
||||
Discovery v5 for a short period until the node populates their peer list.
|
||||
|
||||
For some further investigation, see
|
||||
[here](https://github.com/status-im/swarms/blob/master/ideas/092-disc-v5-research.md).
|
||||
|
||||
### I heard something about `Mailservers` being trusted somehow?
|
||||
|
||||
In order to use a `Mailserver`, a given node needs to connect to it directly, i.e. add the `Mailserver`
|
||||
as its peer and mark it as trusted.
|
||||
This means that the `Mailserver` is able to send direct p2p messages to the node instead of broadcasting them.
|
||||
Effectively, it knows the bloom filter of the topics the node is interested in,
|
||||
when it is online as well as many metadata like IP address.
|
||||
|
||||
### Data sync
|
||||
|
||||
#### Why is MVDS not used for public chats?
|
||||
|
||||
Currently, public chats are broadcast-based, and there's no direct way of finding
|
||||
out who is receiving messages. Hence there's no clear group sync state context
|
||||
whereby participants can sync. Additionally, MVDS is currently not optimized for
|
||||
large group contexts, which means bandwidth usage will be a lot higher than
|
||||
reasonable. See [P2P Data Sync for Mobile](https://vac.dev/p2p-data-sync-for-mobile) for more.
|
||||
This is an active area of research.
|
||||
|
||||
## Footnotes
|
||||
|
||||
1. <https://github.com/status-im/status-protocol-go/>
|
||||
2. <https://github.com/status-im/status-console-client/>
|
||||
3. <https://github.com/status-im/status-mobile/>
|
||||
|
||||
## Appendix A: Security considerations
|
||||
|
||||
There are several security considerations to take into account when running Status.
|
||||
Chief among them are: scalability, DDoS-resistance and privacy.
|
||||
These also vary depending on what capabilities are used, such as `Mailserver`, light node, and so on.
|
||||
|
||||
### Scalability and UX
|
||||
|
||||
**Bandwidth usage:**
|
||||
|
||||
In version 1 of Status, bandwidth usage is likely to be an issue.
|
||||
In Status version 1.1 this is partially addressed with Waku usage, see [the theoretical scaling model](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
|
||||
|
||||
**`Mailserver` High Availability requirement:**
|
||||
|
||||
A `Mailserver` has to be online to receive messages for other nodes, this puts a high availability requirement on it.
|
||||
|
||||
**Gossip-based routing:**
|
||||
|
||||
Use of gossip-based routing doesn't necessarily scale.
|
||||
It means each node can see a message multiple times,
|
||||
and having too many light nodes can cause propagation probability that is too low.
|
||||
See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
|
||||
|
||||
**Lack of incentives:**
|
||||
|
||||
Status currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
|
||||
|
||||
### Privacy
|
||||
|
||||
**Light node privacy:**
|
||||
|
||||
The main privacy concern with light nodes is that directly connected peers will know that a message originates from them (as it are the only ones it sends). This means nodes can make assumptions about what messages (topics) their peers are interested in.
|
||||
|
||||
**Bloom filter privacy:**
|
||||
|
||||
A user reveals which messages they are interested in, by setting only the topics they are interested in on the bloom filter.
|
||||
This is a fundamental trade-off between bandwidth usage and privacy,
|
||||
though the trade-off space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
|
||||
|
||||
**`Mailserver client` privacy:**
|
||||
|
||||
A `Mailserver client` has to trust a `Mailserver`, which means they can send direct traffic. This reveals what topics / bloom filter a node is interested in, along with its peerID (with IP).
|
||||
|
||||
**Privacy guarantees not rigorous:**
|
||||
|
||||
Privacy for Whisper or Waku hasn't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
|
||||
|
||||
**Topic hygiene:**
|
||||
|
||||
Similar to bloom filter privacy, using a very specific topic reveals more information. See scalability model linked above.
|
||||
|
||||
### Spam resistance
|
||||
|
||||
**PoW bad for heterogeneous devices:**
|
||||
|
||||
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
|
||||
|
||||
**`Mailserver` trusted connection:**
|
||||
|
||||
A `Mailserver` has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning `Mailserver` can overwhelm an individual node.
|
||||
|
||||
### Censorship resistance
|
||||
|
||||
**Devp2p TCP port blockable:**
|
||||
|
||||
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
|
||||
|
||||
See <https://github.com/status-im/status-mobile/issues/6351> for some discussion.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Jacek Sieka
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
- Added that Waku SHOULD be used
|
||||
- Added that Whisper SHOULD NOT be used
|
||||
- Added language to include Waku in all relevant places
|
||||
- Change to keep `Mailserver` term consistent
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Protobuf](https://developers.google.com/protocol-buffers/)
|
||||
- [Discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
|
||||
- [RLPx Transport Protocol, v5](https://github.com/ethereum/devp2p/blob/master/rlpx.md)
|
||||
- [Whisper V6](https://eips.ethereum.org/EIPS/eip-627)
|
||||
- [Waku V1](/waku/standards/legacy/6/waku1.md)
|
||||
- [Rendezvous Protocol](https://github.com/libp2p/specs/tree/master/rendezvous)
|
||||
- [Rendezvous Protocol modifications](https://github.com/status-im/rendezvous#differences-with-original-rendezvous)
|
||||
- [Fleets Status](https://fleets.status.im)
|
||||
- [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459#client-protocol)
|
||||
- [WHISPER-USAGE](/status/deprecated/whisper-usage.md)
|
||||
- [WAKU-USAGE](/status/deprecated/waku-usage.md)
|
||||
- [WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md)
|
||||
- [WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md)
|
||||
- [SECURE-TRANSPORT](/status/deprecated/secure-transport.md)
|
||||
- [MVDS](/vac/2/mvds.md)
|
||||
- [PAYLOADS](/status/deprecated/payloads.md)
|
||||
- [EIPS](/status/deprecated/eips.md)
|
||||
- [Murmur](https://github.com/status-im/murmur)
|
||||
- [Re-enable LES as option](https://github.com/status-im/status-go/issues/1025)
|
||||
- [Swarm adaptive nodes](https://github.com/ethersphere/SWIPs/pull/12)
|
||||
- [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/)
|
||||
- [Waku specs](/waku/)
|
||||
- [Vac](https://vac.dev/vac-overview)
|
||||
- [theoretical scaling model](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability)
|
||||
- [Anonymity](https://eprint.iacr.org/2017/954.pdf)
|
||||
- [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf)
|
||||
- [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/)
|
||||
- [Discovery v5 research](https://github.com/status-im/swarms/blob/master/ideas/092-disc-v5-research.md)
|
||||
- [P2P Data Sync for Mobile](https://vac.dev/p2p-data-sync-for-mobile)
|
||||
- [Status protocol go](https://github.com/status-im/status-protocol-go/)
|
||||
- [Status console client](https://github.com/status-im/status-console-client/)
|
||||
- [Status mobile](https://github.com/status-im/status-mobile/)
|
||||
- [Status mobile issue 6351](https://github.com/status-im/status-mobile/issues/6351)
|
||||
135
status/deprecated/dapp-browser-API-usage.md
Normal file
135
status/deprecated/dapp-browser-API-usage.md
Normal file
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: Dapp browser API usage
|
||||
name: Dapp browser API usage
|
||||
status: deprecated
|
||||
description: This document describes requirements that an application must fulfill in order to provide a proper environment for Dapps running inside a browser.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This document describes requirements that an application must fulfill in order to provide a proper environment for Dapps running inside a browser.
|
||||
A description of the Status Dapp API is provided, along with an overview of bidirectional communication underlying the API implementation.
|
||||
The document also includes a list of EIPs that this API implements.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Description |
|
||||
|------------|-------------------------------------------------------------------------------------|
|
||||
| **Webview** | Platform-specific browser core implementation. |
|
||||
| **Ethereum Provider** | A JS object (`window.ethereum`) injected into each web page opened in the browser providing web3 compatible provider. |
|
||||
| **Bridge** | A set of facilities allow bidirectional communication between JS code and the application. |
|
||||
|
||||
## Overview
|
||||
|
||||
The application should expose an Ethereum Provider object (`window.ethereum`) to JS code running inside the browser.
|
||||
It is important to have the `window.ethereum` object available before the page loads, otherwise Dapps might not work correctly.
|
||||
|
||||
Additionally, the browser component should also provide bidirectional communication between JS code and the application.
|
||||
|
||||
## Usage in Dapps
|
||||
|
||||
Dapps can use the below properties and methods of `window.ethereum` object.
|
||||
|
||||
### Properties
|
||||
|
||||
#### `isStatus`
|
||||
|
||||
Returns true. Can be used by the Dapp to find out whether it's running inside Status.
|
||||
|
||||
#### `status`
|
||||
|
||||
Returns a `StatusAPI` object. For now it supports one method: `getContactCode` that sends a `contact-code` request to Status.
|
||||
|
||||
### Methods
|
||||
|
||||
#### `isConnected`
|
||||
|
||||
Similarly to Ethereum JS API [docs](https://github.com/ethereum/wiki/wiki/JavaScript-API#web3isconnected),
|
||||
it should be called to check if connection to a node exists. On Status, this fn always returns true, as once Status is up and running, node is automatically started.
|
||||
|
||||
#### `scanQRCode`
|
||||
|
||||
Sends a `qr-code` Status API request.
|
||||
|
||||
#### `request`
|
||||
|
||||
`request` method as defined by EIP-1193.
|
||||
|
||||
### Unused
|
||||
|
||||
Below are some legacy methods that some Dapps might still use.
|
||||
|
||||
#### `enable` (DEPRECATED)
|
||||
|
||||
Sends a `web3` Status API request. It returns a first entry in the list of available accounts.
|
||||
|
||||
Legacy `enable` method as defined by [EIP1102](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1102.md).
|
||||
|
||||
#### `send` (DEPRECATED)
|
||||
|
||||
Legacy `send` method as defined by [EIP1193](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1193.md).
|
||||
|
||||
#### `sendAsync` (DEPRECATED)
|
||||
|
||||
Legacy `sendAsync` method as defined by [EIP1193](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1193.md).
|
||||
|
||||
#### `sendSync` (DEPRECATED)
|
||||
|
||||
Legacy `send` method.
|
||||
|
||||
## Implementation
|
||||
|
||||
Status uses a [forked version](https://github.com/status-im/react-native-webview) of [react-native-webview](https://github.com/react-native-community/react-native-webview) to display web or dapps content.
|
||||
The fork provides an Android implementation of JS injection before page load.
|
||||
It is required in order to properly inject Ethereum Provider object.
|
||||
|
||||
Status injects two JS scripts:
|
||||
|
||||
- [provider.js](https://github.com/status-im/status-mobile/blob/develop/resources/js/provider.js): `window.ethereum` object
|
||||
- [webview.js](https://github.com/status-im/status-mobile/blob/develop/resources/js/webview.js): override for `history.pushState` used internally
|
||||
|
||||
Dapps running inside a browser communicate with Status Ethereum node by means of a *bridge* provided by react-native-webview library.
|
||||
The bridge allows for bidirectional communication between browser and Status. In order to do so, it injects a special `ReactNativeWebview` object into each page it loads.
|
||||
|
||||
On Status (React Native) end, `react-native-webview` library provides `WebView.injectJavascript` function
|
||||
on a webview component that allows to execute arbitrary code inside the webview.
|
||||
Thus it is possible to inject a function call passing Status node response back to the Dapp.
|
||||
|
||||
Below is the table briefly describing what functions/properties are used. More details available in package [docs](https://github.com/react-native-community/react-native-webview/blob/master/docs/Guide.md#communicating-between-js-and-native).
|
||||
|
||||
| Direction | Side | Method |
|
||||
|-----------|------|-----------|
|
||||
| Browser->Status | JS | `ReactNativeWebView.postMessage()`|
|
||||
| Browser->Status | RN | `WebView.onMessage()`|
|
||||
| Status->Browser | JS | `ReactNativeWebView.onMessage()`|
|
||||
| Status->Browser | RN | `WebView.injectJavascript()`|
|
||||
|
||||
## Compatibility
|
||||
|
||||
Status browser supports the following EIPs:
|
||||
|
||||
- [EIP1102](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1102.md): `eth_requestAccounts` support
|
||||
- [EIP1193](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1193.md): `connect`, `disconnect`, `chainChanged`, and `accountsChanged` event support is not implemented
|
||||
|
||||
## Changelog
|
||||
|
||||
| Version | Comment |
|
||||
| :-----: | ------- |
|
||||
| 0.1.0 | Initial Release |
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Ethereum JS API docs](https://github.com/ethereum/wiki/wiki/JavaScript-API#web3isconnected)
|
||||
- [EIP1102](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1102.md)
|
||||
- [EIP1193](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1193.md)
|
||||
- [forked version](https://github.com/status-im/react-native-webview)
|
||||
- [react-native-webview](https://github.com/react-native-community/react-native-webview)
|
||||
- [provider.js](https://github.com/status-im/status-mobile/blob/develop/resources/js/provider.js)
|
||||
- [webview.js](https://github.com/status-im/status-mobile/blob/develop/resources/js/webview.js)
|
||||
- [docs](https://github.com/react-native-community/react-native-webview/blob/master/docs/Guide.md#communicating-between-js-and-native)
|
||||
286
status/deprecated/eips.md
Normal file
286
status/deprecated/eips.md
Normal file
@@ -0,0 +1,286 @@
|
||||
---
|
||||
title: EIPS
|
||||
name: EIPS
|
||||
status: deprecated
|
||||
description: Status relation with the EIPs
|
||||
editor: Ricardo Guilherme Schmidt <ricardo3@status.im>
|
||||
contributors:
|
||||
-
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes how Status relates with EIPs.
|
||||
|
||||
## Introduction
|
||||
|
||||
Status should follow all standards as possible.
|
||||
Whenever the Status app needs a feature, it should be first checked if there is a standard for that,
|
||||
if not, Status should propose a standard.
|
||||
|
||||
### Support table
|
||||
|
||||
| | Status v0 | Status v1 | Other | State |
|
||||
|----------|-----------|-----------|----------| -------- |
|
||||
| BIP32 | N | Y | N | `stable` |
|
||||
| BIP39 | Y | Y | Y | `stable` |
|
||||
| BIP43 | N | Y | N | `stable` |
|
||||
| BIP44 | N | Y | N | `stable` |
|
||||
| EIP20 | Y | Y | Y | `stable` |
|
||||
| EIP55 | Y | Y | Y | `stable` |
|
||||
| EIP67 | P | P | N | `stable` |
|
||||
| EIP137 | P | P | N | `stable` |
|
||||
| EIP155 | Y | Y | Y | `stable` |
|
||||
| EIP165 | P | N | N | `stable` |
|
||||
| EIP181 | P | N | N | `stable` |
|
||||
| EIP191 | Y? | N | Y | `stable` |
|
||||
| EIP627 | Y | Y | N | `stable` |
|
||||
| EIP681 | Y | N | Y | `stable` |
|
||||
| EIP712 | P | P | Y | `stable` |
|
||||
| EIP721 | P | P | Y | `stable` |
|
||||
| EIP831 | N | Y | N | `stable` |
|
||||
| EIP945 | Y | Y | N | `stable` |
|
||||
| EIP1102 | Y | Y | Y | `stable` |
|
||||
| EIP1193 | Y | Y | Y | `stable` |
|
||||
| EIP1577 | Y | P | N | `stable` |
|
||||
| EIP1581 | N | Y | N | `stable` |
|
||||
| EIP1459 | N | | N | `raw` |
|
||||
|
||||
## Components
|
||||
|
||||
### BIP32 - Hierarchical Deterministic Wallets
|
||||
|
||||
Support: Dependency.
|
||||
[Reference](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)
|
||||
Description: Enable wallets to derive multiple private keys from the same seed.
|
||||
Used for: Dependency of BIP39 and BIP43.
|
||||
|
||||
### BIP39 - Mnemonic code for generating deterministic keys
|
||||
|
||||
Support: Dependency.
|
||||
[Reference](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
|
||||
Description: Enable wallet to create private key based on a safe seed phrase.
|
||||
Used for: Security and user experience.
|
||||
|
||||
### BIP43 - Purpose Field for Deterministic Wallets
|
||||
|
||||
Support: Dependency.
|
||||
[Reference](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
|
||||
Description: Enable wallet to create private keys branched for a specific purpose.
|
||||
Used for: Dependency of BIP44, uses "ethereum" coin.
|
||||
|
||||
### BIP44 - Multi-Account Hierarchy for Deterministic Wallets
|
||||
|
||||
Support: Dependency.
|
||||
[Reference](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
|
||||
Description: Enable wallet to derive multiple accounts in top of BIP39.
|
||||
Used for: Privacy.
|
||||
[Source code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/constants.cljs#L240)
|
||||
Observation: BIP44 don't solve privacy issues regarding the transparency of transactions, therefore directly connected addresses through a transactions can be identifiable by a "network reconnaissance attack" over transaction history, this attack together with leakage of information from centralized services, such as exchanges, would be fatal against the whole privacy of users, regardless of BIP44.
|
||||
|
||||
### EIP20 - Fungible Token
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-20)
|
||||
Description: Enable wallets to use tokens based on smart contracts compliant with this standard.
|
||||
Used for: Wallet feature.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs)
|
||||
|
||||
### EIP55 - Mixed-case checksum address encoding
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-55)
|
||||
Description: Checksum standard that uses lowercase and uppercase inside address hex value.
|
||||
Used for: Sanity check of forms using ethereum address.
|
||||
[Related](https://github.com/status-im/status-mobile/issues/4959) [Also](https://github.com/status-im/status-mobile/issues/8707)
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip55.cljs)
|
||||
|
||||
### EIP67 - Standard URI scheme with metadata, value and byte code
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://github.com/ethereum/EIPs/issues/67)
|
||||
Description: A standard way of creating Ethereum URIs for various use-cases.
|
||||
Used for: Legacy support.
|
||||
[Issue](https://github.com/status-im/status-mobile/issues/875)
|
||||
|
||||
### EIP137 - Ethereum Domain Name Service - Specification
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-137)
|
||||
Description: Enable wallets to lookup ENS names.
|
||||
Used for: User experience, as a wallet and identity feature, usernames.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/ens.cljs#L86)
|
||||
|
||||
### EIP155 - Simple replay attack protection
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-155)
|
||||
Description: Defined chainId parameter in the singed ethereum transaction payload.
|
||||
Used for: Signing transactions, crucial to safety of users against replay attacks.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/core.cljs)
|
||||
|
||||
### EIP165 - Standard Interface Detection
|
||||
|
||||
Support: Dependency/Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-165)
|
||||
Description: Standard interface for contract to answer if it supports other interfaces.
|
||||
Used for: Dependency of ENS and EIP721.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip165.cljs)
|
||||
|
||||
### EIP181 - ENS support for reverse resolution of Ethereum addresses
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-181)
|
||||
Description: Enable wallets to render reverse resolution of Ethereum addresses.
|
||||
Used for: Wallet feature.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/ens.cljs#L86)
|
||||
|
||||
### EIP191 - Signed Message
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-191)
|
||||
Description: Contract signature standard, adds an obligatory padding to signed message to differentiate from Ethereum Transaction messages.
|
||||
Used for: Dapp support, security, dependency of ERC712.
|
||||
|
||||
### EIP627 - Whisper Specification
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-627)
|
||||
Description: format of Whisper messages within the ÐΞVp2p Wire Protocol.
|
||||
Used for: Chat protocol.
|
||||
|
||||
### EIP681 - URL Format for Transaction Requests
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-681)
|
||||
Description: A link that pop up a transaction in the wallet.
|
||||
Used for: Useful as QR code data for transaction requests, chat transaction requests and for dapp links to transaction requests.
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip681.cljs)
|
||||
Related: [Issue #9183: URL Format for Transaction Requests (EIP681) is poorly supported](https://github.com/status-im/status-mobile/issues/9183) [Issue #9240](https://github.com/status-im/status-mobile/pull/9240) [Issue #9238](https://github.com/status-im/status-mobile/issues/9238) [Issue #7214](https://github.com/status-im/status-mobile/issues/7214) [Issue #7325](https://github.com/status-im/status-mobile/issues/7325) [Issue #8150](https://github.com/status-im/status-mobile/issues/8150)
|
||||
|
||||
### EIP712 - Typed Signed Message
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-712)
|
||||
Description: Standardize types for contract signature, allowing users to easily inspect whats being signed.
|
||||
Used for: User experience, security.
|
||||
Related: [Isse #5461](https://github.com/status-im/status-mobile/issues/5461) [Commit](https://github.com/status-im/status-mobile/commit/ba37f7b8d029d3358c7b284f6a2383b9ef9526c9)
|
||||
|
||||
### EIP721 - Non Fungible Token
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-721)
|
||||
Description: Enable wallets to use tokens based on smart contracts compliant with this standard.
|
||||
Used for: Wallet feature.
|
||||
Related: [Issue #8909](https://github.com/status-im/status-mobile/issues/8909)
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/erc721.cljs) [Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs)
|
||||
|
||||
### EIP945 - Web 3 QR Code Scanning API
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://github.com/ethereum/EIPs/issues/945)
|
||||
Used for: Sharing contactcode, reading transaction requests.
|
||||
Related: [Issue #5870](https://github.com/status-im/status-mobile/issues/5870)
|
||||
|
||||
### EIP1102 - Opt-in account exposure
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-1102)
|
||||
Description: Allow users to opt-in the exposure of their ethereum address to dapps they browse.
|
||||
Used for: Privacy, DApp support.
|
||||
Related: [Issue #7985](https://github.com/status-im/status-mobile/issues/7985)
|
||||
|
||||
### EIP1193 - Ethereum Provider JavaScript API
|
||||
|
||||
Support: Full.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-1193)
|
||||
Description: Allows dapps to recognize event changes on wallet.
|
||||
Used for: DApp support.
|
||||
Related: [Issue #7246](https://github.com/status-im/status-mobile/pull/7246)
|
||||
|
||||
### EIP1577 - contenthash field for ENS
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-1577)
|
||||
Description: Allows users browse ENS domains using contenthash standard.
|
||||
Used for: Browser, DApp support.
|
||||
Related: [Isse #6688](https://github.com/status-im/status-mobile/issues/6688)
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/utils/contenthash.cljs) [Sourcecode](https://github.com/status-im/status-mobile/blob/develop/test/cljs/status_im/test/utils/contenthash.cljs#L5)
|
||||
|
||||
### EIP1581 - Non-wallet usage of keys derived from BIP-32 trees
|
||||
|
||||
Support: Partial.
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-1581)
|
||||
Description: Allow wallet to derive keys that are less sensible (non wallet).
|
||||
Used for: Security (don't reuse wallet key) and user experience (don't request keycard every login).
|
||||
Related: [Issue #9096](https://github.com/status-im/status-mobile/issues/9088) [Issue #9096](https://github.com/status-im/status-mobile/pull/9096)
|
||||
[Sourcecode](https://github.com/status-im/status-mobile/blob/develop/src/status_im/constants.cljs#L242)
|
||||
|
||||
### EIP1459 - Node Discovery via DNS
|
||||
|
||||
Support: -
|
||||
[Reference](https://eips.ethereum.org/EIPS/eip-1459)
|
||||
Description: Allows the storing and retrieving of nodes through merkle trees stored in TXT records of a domain.
|
||||
Used for: Finding Waku nodes.
|
||||
Related: -
|
||||
Sourcecode: -
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [BIP32 - Hierarchical Deterministic Wallets](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)
|
||||
- [BIP39 - Mnemonic code for generating deterministic keys](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki)
|
||||
- [BIP43 - Purpose Field for Deterministic Wallets](https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki)
|
||||
- [BIP44 - Multi-Account Hierarchy for Deterministic Wallets](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
|
||||
- [BIP44 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/constants.cljs#L240)
|
||||
- [EIP20 - Fungible Token](https://eips.ethereum.org/EIPS/eip-20)
|
||||
- [EIP20 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs)
|
||||
- [EIP55 - Mixed-case checksum address encoding](https://eips.ethereum.org/EIPS/eip-55)
|
||||
- [EIP55 Related Issue 4959](https://github.com/status-im/status-mobile/issues/4959)
|
||||
- [EIP55 Related Issue 8707](https://github.com/status-im/status-mobile/issues/8707)
|
||||
- [EIP55 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip55.cljs)
|
||||
- [EIP67 - Standard URI scheme with metadata, value and byte code](https://github.com/ethereum/EIPs/issues/67)
|
||||
- [EIP67 Related Issue 875](https://github.com/status-im/status-mobile/issues/875)
|
||||
- [EIP137 - Ethereum Domain Name Service - Specification](https://eips.ethereum.org/EIPS/eip-137)
|
||||
- [EIP137 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/ens.cljs#L86)
|
||||
- [EIP155 - Simple replay attack protection](https://eips.ethereum.org/EIPS/eip-155)
|
||||
- [EIP155 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/core.cljs)
|
||||
- [EIP165 - Standard Interface Detection](https://eips.ethereum.org/EIPS/eip-165)
|
||||
- [EIP165 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip165.cljs)
|
||||
- [EIP181 - ENS support for reverse resolution of Ethereum addresses](https://eips.ethereum.org/EIPS/eip-181)
|
||||
- [EIP181 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/ens.cljs#L86)
|
||||
- [EIP191 - Signed Message](https://eips.ethereum.org/EIPS/eip-191)
|
||||
- [EIP627 - Whisper Specification](https://eips.ethereum.org/EIPS/eip-627)
|
||||
- [EIP681 - URL Format for Transaction Requests](https://eips.ethereum.org/EIPS/eip-681)
|
||||
- [EIP681 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/eip681.cljs)
|
||||
- [EIP681 Related Issue 9183](https://github.com/status-im/status-mobile/issues/9183)
|
||||
- [EIP681 Related Issue 9240](https://github.com/status-im/status-mobile/pull/9240)
|
||||
- [EIP681 Related Issue 9238](https://github.com/status-im/status-mobile/issues/9238)
|
||||
- [EIP681 Related Issue 7214](https://github.com/status-im/status-mobile/issues/7214)
|
||||
- [EIP681 Related Issue 7325](https://github.com/status-im/status-mobile/issues/7325)
|
||||
- [EIP681 Related Issue 8150](https://github.com/status-im/status-mobile/issues/8150)
|
||||
- [EIP712 - Typed Signed Message](https://eips.ethereum.org/EIPS/eip-712)
|
||||
- [EIP712 Related Issue 5461](https://github.com/status-im/status-mobile/issues/5461)
|
||||
- [EIP712 Related Commit](https://github.com/status-im/status-mobile/commit/ba37f7b8d029d3358c7b284f6a2383b9ef9526c9)
|
||||
- [EIP721 - Non Fungible Token](https://eips.ethereum.org/EIPS/eip-721)
|
||||
- [EIP721 Related Issue 8909](https://github.com/status-im/status-mobile/issues/8909)
|
||||
- [EIP721 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/erc721.cljs)
|
||||
- [EIP721 Source Code (Tokens)](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs)
|
||||
- [EIP945 - Web 3 QR Code Scanning API](https://github.com/ethereum/EIPs/issues/945)
|
||||
- [EIP945 Related Issue 5870](https://github.com/status-im/status-mobile/issues/5870)
|
||||
- [EIP1102 - Opt-in account exposure](https://eips.ethereum.org/EIPS/eip-1102)
|
||||
- [EIP1102 Related Issue 7985](https://github.com/status-im/status-mobile/issues/7985)
|
||||
- [EIP1193 - Ethereum Provider JavaScript API](https://eips.ethereum.org/EIPS/eip-1193)
|
||||
- [EIP1193 Related Issue 7246](https://github.com/status-im/status-mobile/pull/7246)
|
||||
- [EIP1577 - contenthash field for ENS](https://eips.ethereum.org/EIPS/eip-1577)
|
||||
- [EIP1577 Related Issue 6688](https://github.com/status-im/status-mobile/issues/6688)
|
||||
- [EIP1577 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/utils/contenthash.cljs)
|
||||
- [EIP1577 Test Source Code](https://github.com/status-im/status-mobile/blob/develop/test/cljs/status_im/test/utils/contenthash.cljs#L5)
|
||||
- [EIP1581 - Non-wallet usage of keys derived from BIP-32 trees](https://eips.ethereum.org/EIPS/eip-1581)
|
||||
- [EIP1581 Related Issue 9088](https://github.com/status-im/status-mobile/issues/9088)
|
||||
- [EIP1581 Related Issue 9096](https://github.com/status-im/status-mobile/pull/9096)
|
||||
- [EIP1581 Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/constants.cljs#L242)
|
||||
- [EIP1459 - Node Discovery via DNS](https://eips.ethereum.org/EIPS/eip-1459)
|
||||
237
status/deprecated/ethereum-usage.md
Normal file
237
status/deprecated/ethereum-usage.md
Normal file
@@ -0,0 +1,237 @@
|
||||
---
|
||||
title: ETHEREUM-USAGE
|
||||
name: Status interactions with the Ethereum blockchain
|
||||
status: deprecated
|
||||
description: All interactions that the Status client has with the Ethereum blockchain.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification documents all the interactions that the Status client has
|
||||
with the [Ethereum](https://ethereum.org/developers/) blockchain.
|
||||
|
||||
## Background
|
||||
|
||||
All the interactions are made through [JSON-RPC](https://github.com/ethereum/wiki/wiki/JSON-RPC).
|
||||
Currently [Infura](https://infura.io/) is used.
|
||||
The client assumes high-availability,
|
||||
otherwise it will not be able to interact with the Ethereum blockchain.
|
||||
Status nodes rely on these Infura nodes
|
||||
to validate the integrity of the transaction and report a consistent history.
|
||||
|
||||
Key handling is described [here](/status/deprecated/account.md)
|
||||
|
||||
1. [Wallet](#wallet)
|
||||
2. [ENS](#ens)
|
||||
|
||||
## Wallet
|
||||
|
||||
The wallet in Status has two main components:
|
||||
|
||||
1) Sending transactions
|
||||
2) Fetching balance
|
||||
|
||||
In the section below are described the `RPC` calls made the nodes, with a brief
|
||||
description of their functionality and how it is used by Status.
|
||||
|
||||
1.[Sending transactions](#sending-transactions)
|
||||
|
||||
- [EstimateGas](#estimategas)
|
||||
- [PendingNonceAt](#pendingnonceat)
|
||||
- [SuggestGasPrice](#suggestgasprice)
|
||||
- [SendTransaction](#sendtransaction)
|
||||
|
||||
2.[Fetching balance](#fetching-balance)
|
||||
|
||||
- [BlockByHash](#blockbyhash)
|
||||
- [BlockByNumber](#blockbynumber)
|
||||
- [FilterLogs](#filterlogs)
|
||||
- [HeaderByNumber](#headerbynumber)
|
||||
- [NonceAt](#nonceat)
|
||||
- [TransactionByHash](#transactionbyhash)
|
||||
- [TransactionReceipt](#transactionreceipt)
|
||||
|
||||
### Sending transactions
|
||||
|
||||
#### EstimateGas
|
||||
|
||||
EstimateGas tries to estimate the gas needed to execute a specific transaction
|
||||
based on the current pending state of the backend blockchain.
|
||||
There is no guarantee that this is the true gas limit requirement
|
||||
as other transactions may be added or removed by miners,
|
||||
but it should provide a basis for setting a reasonable default.
|
||||
|
||||
```go
|
||||
func (ec *Client) EstimateGas(ctx context.Context, msg ethereum.CallMsg) (uint64, error)
|
||||
```
|
||||
|
||||
[L499](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L499)
|
||||
|
||||
#### PendingNonceAt
|
||||
|
||||
`PendingNonceAt` returns the account nonce of the given account in the pending state.
|
||||
This is the nonce that should be used for the next transaction.
|
||||
|
||||
```go
|
||||
func (ec *Client) PendingNonceAt(ctx context.Context, account common.Address) (uint64, error)
|
||||
```
|
||||
|
||||
[L440](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L440)
|
||||
|
||||
#### SuggestGasPrice
|
||||
|
||||
`SuggestGasPrice` retrieves the currently suggested gas price to allow a timely
|
||||
execution of a transaction.
|
||||
|
||||
```go
|
||||
func (ec *Client) SuggestGasPrice(ctx context.Context) (*big.Int, error)
|
||||
```
|
||||
|
||||
[L487](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L487)
|
||||
|
||||
#### SendTransaction
|
||||
|
||||
`SendTransaction` injects a signed transaction into the pending pool for execution.
|
||||
|
||||
If the transaction was a contract creation use the TransactionReceipt method to get the
|
||||
contract address after the transaction has been mined.
|
||||
|
||||
```go
|
||||
func (ec *Client) SendTransaction(ctx context.Context, tx *types.Transaction) error
|
||||
```
|
||||
|
||||
[L512](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L512)
|
||||
|
||||
### Fetching balance
|
||||
|
||||
A Status node fetches the current and historical [ECR20](https://eips.ethereum.org/EIPS/eip-20) and ETH balance for the user wallet address.
|
||||
Collectibles following the [ERC-721](https://eips.ethereum.org/EIPS/eip-721) are also fetched if enabled.
|
||||
|
||||
A Status node supports by default the following [tokens](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs). Custom tokens can be added by specifying the `address`, `symbol` and `decimals`.
|
||||
|
||||
#### BlockByHash
|
||||
|
||||
`BlockByHash` returns the given full block.
|
||||
|
||||
It is used by status to fetch a given block which will then be inspected
|
||||
for transfers to the user address, both tokens and ETH.
|
||||
|
||||
```go
|
||||
func (ec *Client) BlockByHash(ctx context.Context, hash common.Hash) (*types.Block, error)
|
||||
```
|
||||
|
||||
[L78](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L78)
|
||||
|
||||
#### BlockByNumber
|
||||
|
||||
`BlockByNumber` returns a block from the current canonical chain. If number is nil, the
|
||||
latest known block is returned.
|
||||
|
||||
```go
|
||||
func (ec *Client) BlockByNumber(ctx context.Context, number *big.Int) (*types.Block, error)
|
||||
```
|
||||
|
||||
[L82](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L82)
|
||||
|
||||
#### FilterLogs
|
||||
|
||||
`FilterLogs` executes a filter query.
|
||||
|
||||
Status uses this function to filter out logs, using the hash of the block
|
||||
and the address of interest, both inbound and outbound.
|
||||
|
||||
```go
|
||||
func (ec *Client) FilterLogs(ctx context.Context, q ethereum.FilterQuery) ([]types.Log, error)
|
||||
```
|
||||
|
||||
[L377](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L377)
|
||||
|
||||
#### NonceAt
|
||||
|
||||
`NonceAt` returns the account nonce of the given account.
|
||||
|
||||
```go
|
||||
func (ec *Client) NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
|
||||
```
|
||||
|
||||
[L366](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L366)
|
||||
|
||||
#### TransactionByHash
|
||||
|
||||
`TransactionByHash` returns the transaction with the given hash,
|
||||
used to inspect those transactions made/received by the user.
|
||||
|
||||
```go
|
||||
func (ec *Client) TransactionByHash(ctx context.Context, hash common.Hash) (tx *types.Transaction, isPending bool, err error)
|
||||
```
|
||||
|
||||
[L202](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L202)
|
||||
|
||||
#### HeaderByNumber
|
||||
|
||||
`HeaderByNumber` returns a block header from the current canonical chain.
|
||||
|
||||
```go
|
||||
func (ec *Client) HeaderByNumber(ctx context.Context, number *big.Int) (*types.Header, error)
|
||||
```
|
||||
|
||||
[L172](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L172)
|
||||
|
||||
#### TransactionReceipt
|
||||
|
||||
`TransactionReceipt` returns the receipt of a transaction by transaction hash.
|
||||
It is used in status to check if a token transfer was made to the user address.
|
||||
|
||||
```go
|
||||
func (ec *Client) TransactionReceipt(ctx context.Context, txHash common.Hash) (*types.Receipt, error)
|
||||
```
|
||||
|
||||
[L270](https://github.com/ethereum/go-ethereum/blob/26d271dfbba1367326dec38068f9df828d462c61/ethclient/ethclient.go#L270)
|
||||
|
||||
## ENS
|
||||
|
||||
All the interactions with `ENS` are made through the [ENS contract](https://github.com/ensdomains/ens)
|
||||
|
||||
For the `stateofus.eth` username, one can be registered through these [contracts](https://github.com/status-im/ens-usernames)
|
||||
|
||||
### Registering, releasing and updating
|
||||
|
||||
- [Registering a username](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L113)
|
||||
- [Releasing a username](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L131)
|
||||
- [Updating a username](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L174)
|
||||
|
||||
### Slashing
|
||||
|
||||
Usernames MUST be in a specific format, otherwise they MAY be slashed:
|
||||
|
||||
- They MUST only contain alphanumeric characters
|
||||
- They MUST NOT be in the form `0x[0-9a-f]{5}.*` and have more than 12 characters
|
||||
- They MUST NOT be in the [reserved list](https://github.com/status-im/ens-usernames/blob/47c4c6c2058be0d80b7d678e611e166659414a3b/config/ens-usernames/reservedNames.js)
|
||||
- They MUST NOT be too short, this is dynamically set in the contract and can be checked against the [contract](https://github.com/status-im/ens-usernames/blob/master/contracts/registry/UsernameRegistrar.sol#L26)
|
||||
|
||||
- [Slash a reserved username](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L237)
|
||||
- [Slash an invalid username](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L261)
|
||||
- [Slash a username too similar to an address](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L215)
|
||||
- [Slash a username that is too short](https://github.com/status-im/ens-usernames/blob/77d9394d21a5b6213902473b7a16d62a41d9cd09/contracts/registry/UsernameRegistrar.sol#L200)
|
||||
|
||||
ENS names are propagated through `ChatMessage` and `ContactUpdate` [payload](/status/deprecated/payloads.md).
|
||||
A client SHOULD verify ens names against the public key of the sender on receiving the message against the [ENS contract](https://github.com/ensdomains/ens)
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Ethereum Developers](https://ethereum.org/developers/)
|
||||
- [JSON-RPC](https://github.com/ethereum/wiki/wiki/JSON-RPC)
|
||||
- [Infura](https://infura.io/)
|
||||
- [Key Handling](/status/deprecated/account.md)
|
||||
- [ERC-20 Token Standard](https://eips.ethereum.org/EIPS/eip-20)
|
||||
- [ERC-721 Non-Fungible Token Standard](https://eips.ethereum.org/EIPS/eip-721)
|
||||
- [Supported Tokens Source Code](https://github.com/status-im/status-mobile/blob/develop/src/status_im/ethereum/tokens.cljs)
|
||||
- [go-ethereum](https://github.com/ethereum/go-ethereum/)
|
||||
- [ENS Contract](https://github.com/ensdomains/ens)
|
||||
162
status/deprecated/group-chat.md
Normal file
162
status/deprecated/group-chat.md
Normal file
@@ -0,0 +1,162 @@
|
||||
---
|
||||
title: GROUP-CHAT
|
||||
name: Group Chat
|
||||
status: deprecated
|
||||
description: This document describes the group chat protocol used by the Status application.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Andrea Piana <andreap@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This document describes the group chat protocol used by the Status application.
|
||||
The node uses pairwise encryption among members so a message is exchanged
|
||||
between each participant, similarly to a one-to-one message.
|
||||
|
||||
## Membership updates
|
||||
|
||||
The node uses membership updates messages to propagate group chat membership changes.
|
||||
The protobuf format is described in the [PAYLOADS](/status/deprecated/payloads.md).
|
||||
Below describes each specific field.
|
||||
|
||||
The protobuf messages are:
|
||||
|
||||
```protobuf
|
||||
// MembershipUpdateMessage is a message used to propagate information
|
||||
// about group membership changes.
|
||||
message MembershipUpdateMessage {
|
||||
// The chat id of the private group chat
|
||||
string chat_id = 1;
|
||||
// A list of events for this group chat, first 65 bytes are the signature, then is a
|
||||
// protobuf encoded MembershipUpdateEvent
|
||||
repeated bytes events = 2;
|
||||
// An optional chat message
|
||||
ChatMessage message = 3;
|
||||
}
|
||||
|
||||
message MembershipUpdateEvent {
|
||||
// Lamport timestamp of the event as described in [Status Payload Specs](status-payload-specs.md#clock-vs-timestamp-and-message-ordering)
|
||||
uint64 clock = 1;
|
||||
// List of public keys of the targets of the action
|
||||
repeated string members = 2;
|
||||
// Name of the chat for the CHAT_CREATED/NAME_CHANGED event types
|
||||
string name = 3;
|
||||
// The type of the event
|
||||
EventType type = 4;
|
||||
|
||||
enum EventType {
|
||||
UNKNOWN = 0;
|
||||
CHAT_CREATED = 1; // See [CHAT_CREATED](#chat-created)
|
||||
NAME_CHANGED = 2; // See [NAME_CHANGED](#name-changed)
|
||||
MEMBERS_ADDED = 3; // See [MEMBERS_ADDED](#members-added)
|
||||
MEMBER_JOINED = 4; // See [MEMBER_JOINED](#member-joined)
|
||||
MEMBER_REMOVED = 5; // See [MEMBER_REMOVED](#member-removed)
|
||||
ADMINS_ADDED = 6; // See [ADMINS_ADDED](#admins-added)
|
||||
ADMIN_REMOVED = 7; // See [ADMIN_REMOVED](#admin-removed)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Payload
|
||||
|
||||
`MembershipUpdateMessage`:
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | chat-id | `string` | The chat id of the chat where the change is to take place |
|
||||
| 2 | events | See details | A list of events that describe the membership changes, in their encoded protobuf form |
|
||||
| 3 | message | `ChatMessage` | An optional message, described in [Message](/status/deprecated/payloads.md/#message) |
|
||||
|
||||
`MembershipUpdateEvent`:
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | The clock value of the event |
|
||||
| 2 | members | `[]string` | An optional list of hex encoded (prefixed with `0x`) public keys, the targets of the action |
|
||||
| 3 | name | `name` | An optional name, for those events that make use of it |
|
||||
| 4 | type | `EventType` | The type of event sent, described below |
|
||||
|
||||
### Chat ID
|
||||
|
||||
Each membership update MUST be sent with a corresponding `chatId`.
|
||||
The format of this chat ID MUST be a string of [UUID](https://tools.ietf.org/html/rfc4122),
|
||||
concatenated with the hex-encoded public key of the creator of the chat, joined by `-`.
|
||||
This chatId MUST be validated by all clients, and MUST be discarded if it does not follow these rules.
|
||||
|
||||
### Signature
|
||||
|
||||
The node calculates the signature for each event by encoding each `MembershipUpdateEvent` in its protobuf representation
|
||||
and prepending the bytes of the chatID, lastly the node signs the `Keccak256` of the bytes
|
||||
using the private key by the author and added to the `events` field of MembershipUpdateMessage.
|
||||
|
||||
### Group membership event
|
||||
|
||||
Any `group membership` event received MUST be verified by calculating the signature as per the method described above.
|
||||
The author MUST be extracted from it, if the verification fails the event MUST be discarded.
|
||||
|
||||
#### CHAT_CREATED
|
||||
|
||||
Chat `created event` is the first event that needs to be sent.
|
||||
Any event with a clock value lower than this MUST be discarded.
|
||||
Upon receiving this event a client MUST validate the `chatId`
|
||||
provided with the updates and create a chat with identified by `chatId` and named `name`.
|
||||
|
||||
#### NAME_CHANGED
|
||||
|
||||
`admins` use a `name changed` event to change the name of the group chat.
|
||||
Upon receiving this event a client MUST validate the `chatId` provided with the updates
|
||||
and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
|
||||
If the event is valid the chat name SHOULD be changed to `name`.
|
||||
|
||||
#### MEMBERS_ADDED
|
||||
|
||||
`admins` use a `members added` event to add members to the chat.
|
||||
Upon receiving this event a client MUST validate the `chatId`
|
||||
provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
|
||||
If the event is valid a client MUST update the list of members of the chat who have not joined, adding the `members` received.
|
||||
`members` is an array of hex encoded public keys.
|
||||
|
||||
#### MEMBER_JOINED
|
||||
|
||||
`members` use a `members joined` event to signal that they want to start receiving messages from this chat.
|
||||
Upon receiving this event a client MUST validate the `chatId` provided with the updates.
|
||||
If the event is valid a client MUST update the list of members of the chat who joined, adding the signer.
|
||||
Any `message` sent to the group chat should now include the newly joined member.
|
||||
|
||||
#### ADMINS_ADDED
|
||||
|
||||
`admins` use an `admins added` event to add make other admins in the chat.
|
||||
Upon receiving this event a client MUST validate the `chatId` provided with the updates,
|
||||
MUST ensure the author of the event is an admin of the chat
|
||||
and MUST ensure all `members` are already `members` of the chat, otherwise the event MUST be ignored.
|
||||
If the event is valid a client MUST update the list of admins of the chat, adding the `members` received.
|
||||
`members` is an array of hex encoded public keys.
|
||||
|
||||
#### MEMBER_REMOVED
|
||||
|
||||
`members` and/or `admins` use a `member-removed` event to leave or kick members of the chat.
|
||||
Upon receiving this event a client MUST validate the `chatId` provided with the updates, MUST ensure that:
|
||||
|
||||
- If the author of the event is an admin, target can only be themselves or a non-admin member.
|
||||
- If the author of the event is not an admin, the target of the event can only be themselves.
|
||||
|
||||
If the event is valid a client MUST remove the member from the list of `members`/`admins` of the chat,
|
||||
and no further message should be sent to them.
|
||||
|
||||
#### ADMIN_REMOVED
|
||||
|
||||
`Admins` use an `admin-removed` event to drop admin privileges.
|
||||
Upon receiving this event a client MUST validate the `chatId` provided with the updates,
|
||||
MUST ensure that the author of the event is also the target of the event.
|
||||
|
||||
If the event is valid a client MUST remove the member from the list of `admins` of the chat.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [PAYLOADS](/status/deprecated/payloads.md)
|
||||
- [UUID](https://tools.ietf.org/html/rfc4122)
|
||||
BIN
status/deprecated/images/qr-code1-accountmd.png
Normal file
BIN
status/deprecated/images/qr-code1-accountmd.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.4 KiB |
BIN
status/deprecated/images/qr-code2-accountmd.png
Normal file
BIN
status/deprecated/images/qr-code2-accountmd.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.1 KiB |
303
status/deprecated/keycard-usage-for-wallet-and-chat-keys.md
Normal file
303
status/deprecated/keycard-usage-for-wallet-and-chat-keys.md
Normal file
@@ -0,0 +1,303 @@
|
||||
---
|
||||
title: Keycard Usage for Wallet and Chat Keys
|
||||
name: Keycard Usage for Wallet and Chat Keys
|
||||
status: deprecated
|
||||
description: In this specification, we describe how Status communicates with Keycard to create, store and use multiaccount.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Roman Volosovskyi <roman@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
In this specification, we describe how Status communicates with Keycard to create, store and use multiaccount.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Description |
|
||||
| ------------------ | -------------------------------------------------------- |
|
||||
| Keycard Hardwallet | [https://keycard.tech/docs/](https://keycard.tech/docs/) |
|
||||
| | |
|
||||
|
||||
## Multiaccount creation/restoring
|
||||
|
||||
### Creation and restoring via mnemonic
|
||||
|
||||
1. `status-im.hardwallet.card/get-application-info`
|
||||
request: `nil`
|
||||
response: `{"initialized?" false}`
|
||||
2. `status-im.hardwallet.card/init-card`
|
||||
request: `{:pin 123123}`
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"password" "nEJXqf6VWbqeC5oN",
|
||||
"puk" "411810112887",
|
||||
"pin" "123123"}
|
||||
```
|
||||
|
||||
3. `status-im.hardwallet.card/get-application-info`
|
||||
request: `nil`
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"free-pairing-slots" 5,
|
||||
"app-version" "2.2",
|
||||
"secure-channel-pub-key" "04e70d7af7d91b8cd23adbefdfc242c096adee6c1b5ad27a4013a8f926864c1a4f816b338238dc4a04226ab42f23672585c6dca03627885530643f1656ee69b025",
|
||||
"key-uid" "",
|
||||
"instance-uid" "9f149d438988a7af5e1a186f650c9328",
|
||||
"paired?" false,
|
||||
"has-master-key?" false,
|
||||
"initialized?" true}
|
||||
```
|
||||
|
||||
4. `status-im.hardwallet.card/pair`
|
||||
params: `{:password "nEJXqf6VWbqeC5oN"}`
|
||||
response: `AAVefVX0kPGsxnvQV5OXRbRTLGI3k8/S27rpsq/lZrVR` (`pairing`)
|
||||
|
||||
5. `status-im.hardwallet.card/generate-and-load-keys`
|
||||
|
||||
```clojure
|
||||
{:mnemonic "lift mansion moment version card type uncle sunny lock gather nerve math",
|
||||
:pairing "AAVefVX0kPGsxnvQV5OXRbRTLGI3k8/S27rpsq/lZrVR",
|
||||
:pin "123123"}
|
||||
```
|
||||
response:
|
||||
```clojure
|
||||
{"whisper-address" "1f29a1a60c8a12f80c397a91c6ae0323f420e609",
|
||||
"whisper-private-key" "123123123123123",
|
||||
"wallet-root-public-key" "04eb9d01990a106a65a6dfaa48300f72aecfeabe502d9f4f7aeaccb146dc2f16e2dec81dcec0a1a52c1df4450f441a48c210e1a73777c0161030378df22e4ae015",
|
||||
"encryption-public-key" "045ee42f012d72be74b31a28ce320df617e0cd5b9b343fad34fcd61e2f5dfa89ab23d880473ba4e95401a191764c7f872b7af92ea0d8c39462147df6f3f05c2a11",
|
||||
"wallet-root-address" "132dd67ff47cc1c376879c474fd2afd0f1eee6de",
|
||||
"whisper-public-key" "0450ad84bb95f32c64f4e5027cc11d1b363a0566a0cfc475c5653e8af9964c5c9b0661129b75e6e1bc6e96ba2443238e53e7f49f2c5f2d16fcf04aca4826765d46",
|
||||
"address" "bf93eb43fea2ce94bf3a6463c16680b56aa4a08a",
|
||||
"wallet-address" "7eee1060d8e4722d36c99f30ff8291caa3cfc40c",
|
||||
"key-uid" "472d8436ccedb64bcbd897bed5895ec3458b306352e1bcee377df87db32ef2c2",
|
||||
"wallet-public-key" "0495ab02978ea1f8b059140e0be5a87aad9b64bb7d9706735c47dda6e182fd5ca41744ca37583b9a10c316b01d4321d6c85760c61301874089acab041037246294",
|
||||
"public-key" "0465d452d12171711f32bb931f9ea26fe1b88fe2511a7909a042b914fde10a99719136365d506e2d1694fc14627f9d557da33865efc6001da3942fc1d4d2469ca1",
|
||||
"instance-uid" "9f149d438988a7af5e1a186f650c9328"}
|
||||
```
|
||||
|
||||
### Multiaccount restoring via pairing
|
||||
This flow is required in case if a user want to pair a card with an existing multiaccount on it.
|
||||
1. `status-im.hardwallet.card/get-application-info`
|
||||
request: `nil`
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"free-pairing-slots" 4,
|
||||
"app-version" "2.2",
|
||||
"secure-channel-pub-key" "04e70d7af7d91b8cd23adbefdfc242c096adee6c1b5ad27a4013a8f926864c1a4f816b338238dc4a04226ab42f23672585c6dca03627885530643f1656ee69b025",
|
||||
"key-uid" "",
|
||||
"instance-uid" "9f149d438988a7af5e1a186f650c9328",
|
||||
"paired?" false,
|
||||
"has-master-key?" false,
|
||||
"initialized?" true}
|
||||
```
|
||||
2. `status-im.hardwallet.card/pair`
|
||||
params: `{:password "nEJXqf6VWbqeC5oN"}`
|
||||
response: `AAVefVX0kPGsxnvQV5OXRbRTLGI3k8/S27rpsq/lZrVR` (`pairing`)
|
||||
|
||||
3. `status-im.hardwallet.card/generate-and-load-keys`
|
||||
```clojure
|
||||
{:mnemonic "lift mansion moment version card type uncle sunny lock gather nerve math",
|
||||
:pairing "AAVefVX0kPGsxnvQV5OXRbRTLGI3k8/S27rpsq/lZrVR",
|
||||
:pin "123123"}
|
||||
```
|
||||
response:
|
||||
```clojure
|
||||
{"whisper-address" "1f29a1a60c8a12f80c397a91c6ae0323f420e609",
|
||||
"whisper-private-key" "123123123123123123123",
|
||||
"wallet-root-public-key" "04eb9d01990a106a65a6dfaa48300f72aecfeabe502d9f4f7aeaccb146dc2f16e2dec81dcec0a1a52c1df4450f441a48c210e1a73777c0161030378df22e4ae015",
|
||||
"encryption-public-key" "045ee42f012d72be74b31a28ce320df617e0cd5b9b343fad34fcd61e2f5dfa89ab23d880473ba4e95401a191764c7f872b7af92ea0d8c39462147df6f3f05c2a11",
|
||||
"wallet-root-address" "132dd67ff47cc1c376879c474fd2afd0f1eee6de",
|
||||
"whisper-public-key" "0450ad84bb95f32c64f4e5027cc11d1b363a0566a0cfc475c5653e8af9964c5c9b0661129b75e6e1bc6e96ba2443238e53e7f49f2c5f2d16fcf04aca4826765d46",
|
||||
"address" "bf93eb43fea2ce94bf3a6463c16680b56aa4a08a",
|
||||
"wallet-address" "7eee1060d8e4722d36c99f30ff8291caa3cfc40c",
|
||||
"key-uid" "472d8436ccedb64bcbd897bed5895ec3458b306352e1bcee377df87db32ef2c2",
|
||||
"wallet-public-key" "0495ab02978ea1f8b059140e0be5a87aad9b64bb7d9706735c47dda6e182fd5ca41744ca37583b9a10c316b01d4321d6c85760c61301874089acab041037246294",
|
||||
"public-key" "0465d452d12171711f32bb931f9ea26fe1b88fe2511a7909a042b914fde10a99719136365d506e2d1694fc14627f9d557da33865efc6001da3942fc1d4d2469ca1",
|
||||
"instance-uid" "9f149d438988a7af5e1a186f650c9328"}
|
||||
```
|
||||
|
||||
## Multiaccount unlocking
|
||||
|
||||
1. `status-im.hardwallet.card/get-application-info`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pairing nil, :on-success nil}
|
||||
```
|
||||
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"free-pairing-slots" 4,
|
||||
"app-version" "2.2",
|
||||
"secure-channel-pub-key" "04b079ac513d5e0ebbe9becbae1618503419f5cb59edddc7d7bb09ce0db069a8e6dec1fb40c6b8e5454f7e1fcd0bb4a0b9750256afb4e4390e169109f3ea3ba91d",
|
||||
"key-uid" "a5424fb033f5cc66dce9cbbe464426b6feff70ca40aa952c56247aaeaf4764a9",
|
||||
"instance-uid" "2268254e3ed7898839abe0b40e1b4200",
|
||||
"paired?" false,
|
||||
"has-master-key?" true,
|
||||
"initialized?" true}
|
||||
```
|
||||
|
||||
2. `status-im.hardwallet.card/get-keys`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pairing "ACEWbvUlordYWOE6M1Narn/AXICRltjyuKIAn4kkPXQG",
|
||||
:pin "123123"}
|
||||
```
|
||||
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"whisper-address" "ec83f7354ca112203d2ce3e0b77b47e6e33258aa",
|
||||
"whisper-private-key" "123123123123123123123123",
|
||||
"wallet-root-public-key" "0424a93fe62a271ad230eb2957bf221b4644670589f5c0d69bd11f3371034674bf7875495816095006c2c0d5f834d628b87691a8bbe3bcc2225269020febd65a19",
|
||||
"encryption-public-key" "0437eef85e669f800570f444e64baa2d0580e61cf60c0e9236b4108455ec1943f385043f759fcb5bd8348e32d6d6550a844cf24e57f68e9397a0f7c824a8caee2d",
|
||||
"wallet-root-address" "6ff915f9f31f365511b1b8c1e40ce7f266caa5ce",
|
||||
"whisper-public-key" "04b195df4336c596cca1b89555dc55dd6bb4c5c4491f352f6fdfae140a2349213423042023410f73a862aa188f6faa05c80b0344a1e39c253756cb30d8753f9f8324",
|
||||
"address" "73509a1bb5f3b74d0dba143705cd9b4b55b8bba1",
|
||||
"wallet-address" "2f0cc0e0859e7a05f319d902624649c7e0f48955",
|
||||
"key-uid" "a5424fb033f5cc66dce9cbbe464426b6feff70ca40aa952c56247aaeaf4764a9",
|
||||
"wallet-public-key" "04d6fab73772933215872c239787b2281f3b10907d099d04b88c861e713bd2b95883e0b1710a266830da29e76bbf6b87ed034ab139e36cc235a1b2a5b5ddfd4e91",
|
||||
"public-key" "0437eef85e669f800570f444e64baa2d0580e61cf60c0e9236b4108455ec1943f385043f759fcb5bd8348e32d6d6550a844cf24e57f68e9397a0f7c824a8caee2d",
|
||||
"instance-uid" "2268254e3ed7898839abe0b40e1b4200"}
|
||||
```
|
||||
|
||||
3. `status-im.hardwallet.card/get-application-info`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pairing "ACEWbvUlordYWOE6M1Narn/AXICRltjyuKIAn4kkPXQG"}
|
||||
```
|
||||
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"paired?" true,
|
||||
"has-master-key?" true,
|
||||
"app-version" "2.2",
|
||||
"free-pairing-slots" 4,
|
||||
"pin-retry-counter" 3,
|
||||
"puk-retry-counter" 5,
|
||||
"initialized?" true,
|
||||
"secure-channel-pub-key" "04b079ac513d5e0ebbe9becbae1618503419f5cb59edddc7d7bb09ce0db069a8e6dec1fb40c6b8e5454f7e1fcd0bb4a0b9750256afb4e4390e169109f3ea3ba91d",
|
||||
"key-uid" "a5424fb033f5cc66dce9cbbe464426b6feff70ca40aa952c56247aaeaf4764a9",
|
||||
"instance-uid" "2268254e3ed7898839abe0b40e1b4200"}
|
||||
```
|
||||
|
||||
## Transaction signing
|
||||
|
||||
1. `status-im.hardwallet.card/get-application-info`
|
||||
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pairing "ALecvegKyOW4szknl01yYWx60GLDK5gDhxMgJECRZ+7h",
|
||||
:on-success :hardwallet/sign}
|
||||
```
|
||||
|
||||
response:
|
||||
|
||||
```clojure
|
||||
{"paired?" true,
|
||||
"has-master-key?" true,
|
||||
"app-version" "2.2",
|
||||
"free-pairing-slots" 4,
|
||||
"pin-retry-counter" 3,
|
||||
"puk-retry-counter" 5,
|
||||
"initialized?" true,
|
||||
"secure-channel-pub-key" "0476d11f2ccdad4e7779b95a1ce063d7280cb6c09afe2c0e48ca0c64ab9cf2b3c901d12029d6c266bfbe227c73a802561302b2330ac07a3270fc638ad258fced4a",
|
||||
"key-uid" "d5c8cde8085e7a3fcf95aafbcbd7b3cfe32f61b85c2a8f662f60e76bdc100718",
|
||||
"instance-uid" "e20e27bfee115b431e6e81b8e9dcf04c"}
|
||||
```
|
||||
|
||||
2. `status-im.hardwallet.card/sign`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:hash "92fc7ef54c3e0c42de256b93fbf2c49dc6948ee089406e204dec943b7a0142a9",
|
||||
:pairing "ALecvegKyOW4szknl01yYWx60GLDK5gDhxMgJECRZ+7h",
|
||||
:pin "123123",
|
||||
:path "m/44'/60'/0'/0/0"}
|
||||
```
|
||||
|
||||
response: `5d2ca075593cf50aa34007a0a1df7df14a369534450fce4a2ae8d023a9d9c0e216b5e5e3f64f81bee91613318d01601573fdb15c11887a3b8371e3291e894de600`
|
||||
|
||||
## Account derivation
|
||||
|
||||
1. `status-im.hardwallet.card/verify-pin`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pin "123123",
|
||||
:pairing "ALecvegKyOW4szknl01yYWx60GLDK5gDhxMgJECRZ+7h"}
|
||||
```
|
||||
|
||||
response: `3`
|
||||
1. `status-im.hardwallet.card/export-key`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:pin "123123",
|
||||
:pairing "ALecvegKyOW4szknl01yYWx60GLDK5gDhxMgJECRZ+7h",
|
||||
:path "m/44'/60'/0'/0/1"}
|
||||
```
|
||||
|
||||
response: `046d1bcd2310a5e0094bc515b0ec995a8cb59e23d564094443af10011b6c00bdde44d160cdd32b4b6341ddd7dc83a4f31fdf60ec2276455649ccd7a22fa4ea01d8` (account's `public-key`)
|
||||
|
||||
## Reset pin
|
||||
|
||||
1. `status-im.hardwallet.card/change-pin`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:new-pin "111111",
|
||||
:current-pin "222222",
|
||||
:pairing "AA0sKxPkN+jMHXZZeI8Rgz04AaY5Fg0CzVbm9189Khob"}
|
||||
```
|
||||
|
||||
response:
|
||||
`true`
|
||||
|
||||
## Unblock pin
|
||||
|
||||
If user enters a wrong pin three times in a row a card gets blocked. The user can use puk code then to unblock the card and set a new pin.
|
||||
|
||||
1. `status-im.hardwallet.card/unblock-pin`
|
||||
params:
|
||||
|
||||
```clojure
|
||||
{:puk "120702722103",
|
||||
:new-pin "111111",
|
||||
:pairing "AIoQl0OtCL0/uSN7Ct1/FHRMEk/eM2Lrhn0bw7f8sgOe"}
|
||||
```
|
||||
|
||||
response
|
||||
`true`
|
||||
|
||||
## Status go calls
|
||||
|
||||
In order to use the card in the app we need to use some parts of status-go API, specifically:
|
||||
|
||||
1. [`SaveAccountAndLoginWithKeycard`](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go#L337) after multiaccount creation/restoring to store multiaccount and login into it
|
||||
2. [`LoginWithKeycard`](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go#L373) to log into already existing account
|
||||
3. [`HashTransaction`](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go#L492) and [`HashMessage`](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go#L520) for hashing transaction/message before signing
|
||||
4. [`SendTransactionWithSignature`](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go#L471) to send transaction
|
||||
|
||||
## Where are the keys stored?
|
||||
|
||||
1. When we create a regular multiaccount all its keys are stored on device and are encrypted via key which is derived from user's password. In case if account was created using keycard all keys are stored on the card and are retrieved from it during signing into multiaccount.
|
||||
2. When we create a regular multiaccount we also create a separate database for it and this database is encrypted using key which is derived from user's password. For a keycard account we use `encryption-public-key` (returned by `status-im.hardwallet.card/get-keys`/`status-im.hardwallet.card/generate-and-load-keys`) as a password.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Keycard Hardwallet Documentation](https://keycard.tech/docs/)
|
||||
- [Keycard Codebase](https://github.com/status-im/status-go/blob/b33ad8147d23a932064f241e575511d70a601dcc/mobile/status.go)
|
||||
52
status/deprecated/notifications.md
Normal file
52
status/deprecated/notifications.md
Normal file
@@ -0,0 +1,52 @@
|
||||
---
|
||||
title: NOTIFICATIONS
|
||||
name: Notifications
|
||||
status: deprecated
|
||||
description: A client should implement local notifications to offer notifications for any event in the app without the privacy cost and dependency on third party services.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Eric Dvorsak <eric@status.im>
|
||||
---
|
||||
|
||||
## Local Notifications
|
||||
|
||||
A client should implement local notifications to offer notifications
|
||||
for any event in the app without the privacy cost and dependency on third party services.
|
||||
This means that the client should run a background service to continuously or periodically check for updates.
|
||||
|
||||
### Android
|
||||
|
||||
Android allows running services on the device. When the user enables notifications,
|
||||
the client may start a ``Foreground Service`,
|
||||
and display a permanent notification indicating that the service is running,
|
||||
as required by Android guidelines.
|
||||
The service will simply keep the app from being killed by the system when it is in the background.
|
||||
The client will then be able to run in the background
|
||||
and display local notifications on events such as receiving a message in a one to one chat.
|
||||
|
||||
To facilitate the implementation of local notifications,
|
||||
a node implementation such as `status-go` may provide a specific `notification` signal.
|
||||
|
||||
Notifications are a separate process in Android, and interaction with a notification generates an `Intent`.
|
||||
To handle intents, the `NewMessageSignalHandler` may use a `BroadcastReceiver`,
|
||||
in order to update the state of local notifications when the user dismisses or tap a notification.
|
||||
If the user taps on a notification, the `BroadcastReceiver` generates a new intent to open the app should use universal links to get the user to the right place.
|
||||
|
||||
### iOS
|
||||
|
||||
We are not able to offer local notifications on iOS because there is no concept of services in iOS.
|
||||
It offers background updates but they’re not consistently triggered, and cannot be relied upon.
|
||||
The system decides when the background updates are triggered and the heuristics aren't known.
|
||||
|
||||
## Why is there no Push Notifications?
|
||||
|
||||
Push Notifications, as offered by Apple and Google are a privacy concern,
|
||||
they require a centralized service that is aware of who the notification needs to be delivered to.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- None
|
||||
386
status/deprecated/payloads.md
Normal file
386
status/deprecated/payloads.md
Normal file
@@ -0,0 +1,386 @@
|
||||
---
|
||||
title: PAYLOADS
|
||||
name: Payloads
|
||||
status: deprecated
|
||||
description: Payload of messages in Status, regarding chat and chat-related use cases.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes how the payload of each message in Status looks like.
|
||||
It is primarily centered around chat and chat-related use cases.
|
||||
|
||||
The payloads aims to be flexible enough to support messaging but also cases
|
||||
described in the [Status Whitepaper](https://status.im/whitepaper.pdf)
|
||||
as well as various clients created using different technologies.
|
||||
|
||||
## Introduction
|
||||
|
||||
This document describes the payload format and some special considerations.
|
||||
|
||||
## Payload wrapper
|
||||
|
||||
The node wraps all payloads in a [protobuf record](https://developers.google.com/protocol-buffers/):
|
||||
|
||||
```protobuf
|
||||
message ApplicationMetadataMessage {
|
||||
bytes signature = 1;
|
||||
bytes payload = 2;
|
||||
|
||||
Type type = 3;
|
||||
|
||||
enum Type {
|
||||
UNKNOWN = 0;
|
||||
CHAT_MESSAGE = 1;
|
||||
CONTACT_UPDATE = 2;
|
||||
MEMBERSHIP_UPDATE_MESSAGE = 3;
|
||||
PAIR_INSTALLATION = 4;
|
||||
SYNC_INSTALLATION = 5;
|
||||
REQUEST_ADDRESS_FOR_TRANSACTION = 6;
|
||||
ACCEPT_REQUEST_ADDRESS_FOR_TRANSACTION = 7;
|
||||
DECLINE_REQUEST_ADDRESS_FOR_TRANSACTION = 8;
|
||||
REQUEST_TRANSACTION = 9;
|
||||
SEND_TRANSACTION = 10;
|
||||
DECLINE_REQUEST_TRANSACTION = 11;
|
||||
SYNC_INSTALLATION_CONTACT = 12;
|
||||
SYNC_INSTALLATION_ACCOUNT = 13;
|
||||
SYNC_INSTALLATION_PUBLIC_CHAT = 14;
|
||||
CONTACT_CODE_ADVERTISEMENT = 15;
|
||||
PUSH_NOTIFICATION_REGISTRATION = 16;
|
||||
PUSH_NOTIFICATION_REGISTRATION_RESPONSE = 17;
|
||||
PUSH_NOTIFICATION_QUERY = 18;
|
||||
PUSH_NOTIFICATION_QUERY_RESPONSE = 19;
|
||||
PUSH_NOTIFICATION_REQUEST = 20;
|
||||
PUSH_NOTIFICATION_RESPONSE = 21;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`signature` is the bytes of the signed `SHA3-256` of the payload,
|
||||
signed with the key of the author of the message.
|
||||
The node needs the signature to validate authorship of the message,
|
||||
so that the message can be relayed to third parties.
|
||||
If a signature is not present, but an author is provided by a layer below,
|
||||
the message is not to be relayed to third parties, and it is considered plausibly deniable.
|
||||
|
||||
`payload` is the protobuf encoded content of the message, with the corresponding `type` set.
|
||||
|
||||
## Encoding
|
||||
|
||||
The node encodes the payload using [Protobuf](https://developers.google.com/protocol-buffers)
|
||||
|
||||
## Types of messages
|
||||
|
||||
### Message
|
||||
|
||||
The type `ChatMessage` represents a chat message exchanged between clients.
|
||||
|
||||
#### Payload
|
||||
|
||||
The protobuf description is:
|
||||
|
||||
```protobuf
|
||||
message ChatMessage {
|
||||
// Lamport timestamp of the chat message
|
||||
uint64 clock = 1;
|
||||
// Unix timestamps in milliseconds, currently not used as we use Whisper/Waku as more reliable, but here
|
||||
// so that we don't rely on it
|
||||
uint64 timestamp = 2;
|
||||
// Text of the message
|
||||
string text = 3;
|
||||
// Id of the message that we are replying to
|
||||
string response_to = 4;
|
||||
// Ens name of the sender
|
||||
string ens_name = 5;
|
||||
// Chat id, this field is symmetric for public-chats and private group chats,
|
||||
// but asymmetric in case of one-to-ones, as the sender will use the chat-id
|
||||
// of the received, while the receiver will use the chat-id of the sender.
|
||||
// Probably should be the concatenation of sender-pk & receiver-pk in alphabetical order
|
||||
string chat_id = 6;
|
||||
|
||||
// The type of message (public/one-to-one/private-group-chat)
|
||||
MessageType message_type = 7;
|
||||
// The type of the content of the message
|
||||
ContentType content_type = 8;
|
||||
|
||||
oneof payload {
|
||||
StickerMessage sticker = 9;
|
||||
}
|
||||
|
||||
enum MessageType {
|
||||
UNKNOWN_MESSAGE_TYPE = 0;
|
||||
ONE_TO_ONE = 1;
|
||||
PUBLIC_GROUP = 2;
|
||||
PRIVATE_GROUP = 3;
|
||||
// Only local
|
||||
SYSTEM_MESSAGE_PRIVATE_GROUP = 4;}
|
||||
enum ContentType {
|
||||
UNKNOWN_CONTENT_TYPE = 0;
|
||||
TEXT_PLAIN = 1;
|
||||
STICKER = 2;
|
||||
STATUS = 3;
|
||||
EMOJI = 4;
|
||||
TRANSACTION_COMMAND = 5;
|
||||
// Only local
|
||||
SYSTEM_MESSAGE_CONTENT_PRIVATE_GROUP = 6;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Payload
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | The clock of the chat|
|
||||
| 2 | timestamp | `uint64` | The sender timestamp at message creation |
|
||||
| 3 | text | `string` | The content of the message |
|
||||
| 4 | response_to | `string` | The ID of the message replied to |
|
||||
| 5 | ens_name | `string` | The ENS name of the user sending the message |
|
||||
| 6 | chat_id | `string` | The local ID of the chat the message is sent to |
|
||||
| 7 | message_type | `MessageType` | The type of message, different for one-to-one, public or group chats |
|
||||
| 8 | content_type | `ContentType` | The type of the content of the message |
|
||||
| 9 | payload | `Sticker\|nil` | The payload of the message based on the content type |
|
||||
|
||||
#### Content types
|
||||
|
||||
A node requires content types for a proper interpretation of incoming messages. Not each message is plain text but may carry different information.
|
||||
|
||||
The following content types MUST be supported:
|
||||
|
||||
* `TEXT_PLAIN` identifies a message which content is a plaintext.
|
||||
|
||||
There are other content types that MAY be implemented by the client:
|
||||
|
||||
* `STICKER`
|
||||
* `STATUS`
|
||||
* `EMOJI`
|
||||
* `TRANSACTION_COMMAND`
|
||||
|
||||
##### Mentions
|
||||
|
||||
A mention MUST be represented as a string with the `@0xpk` format, where `pk` is the public key of the [user account](/status/deprecated/account.md) to be mentioned,
|
||||
within the `text` field of a message with content_type `TEXT_PLAIN`.
|
||||
A message MAY contain more than one mention.
|
||||
This specification RECOMMENDs that the application does not require the user to enter the entire pk.
|
||||
This specification RECOMMENDs that the application allows the user to create a mention
|
||||
by typing @ followed by the related ENS or 3-word pseudonym.
|
||||
This specification RECOMMENDs that the application provides the user auto-completion functionality to create a mention.
|
||||
For better user experience, the client SHOULD display a known [ens name or the 3-word pseudonym corresponding to the key](/status/deprecated/account.md#contact-verification) instead of the `pk`.
|
||||
|
||||
##### Sticker content type
|
||||
|
||||
A `ChatMessage` with `STICKER` `Content/Type` MUST also specify the ID of the `Pack` and
|
||||
the `Hash` of the pack, in the `Sticker` field of `ChatMessage`
|
||||
|
||||
```protobuf
|
||||
message StickerMessage {
|
||||
string hash = 1;
|
||||
int32 pack = 2;
|
||||
}
|
||||
```
|
||||
|
||||
#### Message types
|
||||
|
||||
A node requires message types to decide how to encrypt a particular message
|
||||
and what metadata needs to be attached when passing a message to the transport layer.
|
||||
For more on this, see [WHISPER-USAGE](/status/deprecated/whisper-usage.md)
|
||||
and [WAKU-USAGE](/status/deprecated/waku-usage.md).
|
||||
|
||||
<!-- TODO: This reference is a bit odd, considering the layer payloads should interact with is Secure Transport, and not Whisper/Waku. This requires more detail -->
|
||||
|
||||
The following messages types MUST be supported:
|
||||
|
||||
* `ONE_TO_ONE` is a message to the public group
|
||||
* `PUBLIC_GROUP` is a private message
|
||||
* `PRIVATE_GROUP` is a message to the private group.
|
||||
|
||||
#### Clock vs Timestamp and message ordering
|
||||
|
||||
If a user sends a new message before the messages sent
|
||||
while the user was offline are received,
|
||||
the newmessage is supposed to be displayed last in a chat.
|
||||
This is where the basic algorithm of Lamport timestamp would fall short
|
||||
as it's only meant to order causally related events.
|
||||
|
||||
The status client therefore makes a "bid", speculating that it will beat the current chat-timestamp, s.t. the status client's
|
||||
Lamport timestamp format is: `clock = max({timestamp}, chat_clock + 1)`
|
||||
|
||||
This will satisfy the Lamport requirement, namely: a -> b then T(a) < T(b)
|
||||
|
||||
`timestamp` MUST be Unix time calculated, when the node creates the message, in milliseconds.
|
||||
This field SHOULD not be relied upon for message ordering.
|
||||
|
||||
`clock` SHOULD be calculated using the algorithm of [Lamport timestamps](https://en.wikipedia.org/wiki/Lamport_timestamps).
|
||||
When there are messages available in a chat,
|
||||
the node calculates `clock`'s value based on the last received message in a particular chat: `max(timeNowInMs, last-message-clock-value + 1)`.
|
||||
If there are no messages, `clock` is initialized with `timestamp`'s value.
|
||||
|
||||
Messages with a `clock` greater than `120` seconds over the Whisper/Waku timestamp SHOULD be discarded,
|
||||
in order to avoid malicious users to increase the `clock` of a chat arbitrarily.
|
||||
|
||||
Messages with a `clock` less than `120` seconds under the Whisper/Waku timestamp
|
||||
might indicate an attempt to insert messages in the chat history which is not distinguishable from a `datasync` layer re-transit event.
|
||||
A client MAY mark this messages with a warning to the user, or discard them.
|
||||
|
||||
The node uses `clock` value for the message ordering.
|
||||
The algorithm used, and the distributed nature of the system produces casual ordering, which might produce counter-intuitive results in some edge cases.
|
||||
For example, when a user joins a public chat and sends a message
|
||||
before receiving the exist messages, their message `clock` value might be lower
|
||||
and the message will end up in the past when the historical messages are fetched.
|
||||
|
||||
#### Chats
|
||||
|
||||
Chat is a structure that helps organize messages.
|
||||
It's usually desired to display messages only from a single recipient,
|
||||
or a group of recipients at a time and chats help to achieve that.
|
||||
|
||||
All incoming messages can be matched against a chat.
|
||||
The below table describes how to calculate a chat ID for each message type.
|
||||
|
||||
|Message Type|Chat ID Calculation|Direction|Comment|
|
||||
|------------|-------------------|---------|-------|
|
||||
|PUBLIC_GROUP|chat ID is equal to a public channel name; it should equal `chatId` from the message|Incoming/Outgoing||
|
||||
|ONE_TO_ONE|let `P` be a public key of the recipient; `hex-encode(P)` is a chat ID; use it as `chatId` value in the message|Outgoing||
|
||||
|user-message|let `P` be a public key of message's signature; `hex-encode(P)` is a chat ID; discard `chat-id` from message|Incoming|if there is no matched chat, it might be the first message from public key `P`; the node MAY discard the message or MAY create a new chat; Status official clients create a new chat|
|
||||
|PRIVATE_GROUP|use `chatId` from the message|Incoming/Outgoing|find an existing chat by `chatId`; if none is found, the user is not a member of that chat or the user hasn't joined that chat, the message MUST be discarded |
|
||||
|
||||
### Contact Update
|
||||
|
||||
`ContactUpdate` is a message exchange to notify peers that either the
|
||||
user has been added as a contact, or that information about the sending user have
|
||||
changed.
|
||||
|
||||
```protobuf
|
||||
message ContactUpdate {
|
||||
uint64 clock = 1;
|
||||
string ens_name = 2;
|
||||
string profile_image = 3;
|
||||
}
|
||||
```
|
||||
|
||||
Payload
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | The clock of the chat with the user |
|
||||
| 2 | ens_name | `string` | The ENS name if set |
|
||||
| 3 | profile_image | `string` | The base64 encoded profile picture of the user |
|
||||
|
||||
#### Contact update
|
||||
|
||||
A client SHOULD send a `ContactUpdate` to all the contacts each time:
|
||||
|
||||
* The ens_name has changed
|
||||
* A user edits the profile image
|
||||
|
||||
A client SHOULD also periodically send a `ContactUpdate` to all the contacts, the interval is up to the client,
|
||||
the Status official client sends these updates every 48 hours.
|
||||
|
||||
### SyncInstallationContact
|
||||
|
||||
The node uses `SyncInstallationContact` messages to synchronize in a best-effort the contacts to other devices.
|
||||
|
||||
```protobuf
|
||||
message SyncInstallationContact {
|
||||
uint64 clock = 1;
|
||||
string id = 2;
|
||||
string profile_image = 3;
|
||||
string ens_name = 4;
|
||||
uint64 last_updated = 5;
|
||||
repeated string system_tags = 6;
|
||||
}
|
||||
```
|
||||
|
||||
Payload
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | clock value of the chat |
|
||||
| 2 | id | `string` | id of the contact synced |
|
||||
| 3 | profile_image | `string` | `base64` encoded profile picture of the user |
|
||||
| 4 | ens_name | `string` | ENS name of the contact |
|
||||
| 5 | `array[string]` | Array of `system_tags` for the user, this can currently be: `":contact/added", ":contact/blocked", ":contact/request-received"`||
|
||||
|
||||
### SyncInstallationPublicChat
|
||||
|
||||
The node uses `SyncInstallationPublicChat` message to synchronize in a best-effort the public chats to other devices.
|
||||
|
||||
```protobuf
|
||||
message SyncInstallationPublicChat {
|
||||
uint64 clock = 1;
|
||||
string id = 2;
|
||||
}
|
||||
```
|
||||
|
||||
Payload
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | clock value of the chat |
|
||||
| 2 | id | `string` | id of the chat synced |
|
||||
|
||||
### PairInstallation
|
||||
|
||||
The node uses `PairInstallation` messages to propagate information about a device to its paired devices.
|
||||
|
||||
```protobuf
|
||||
message PairInstallation {
|
||||
uint64 clock = 1;
|
||||
string installation_id = 2;
|
||||
string device_type = 3;
|
||||
string name = 4;
|
||||
}
|
||||
```
|
||||
|
||||
Payload
|
||||
|
||||
| Field | Name | Type | Description |
|
||||
| ----- | ---- | ---- | ---- |
|
||||
| 1 | clock | `uint64` | clock value of the chat |
|
||||
| 2| installation_id | `string` | A randomly generated id that identifies this device |
|
||||
| 3 | device_type | `string` | The OS of the device `ios`,`android` or `desktop` |
|
||||
| 4 | name | `string` | The self-assigned name of the device |
|
||||
|
||||
### MembershipUpdateMessage and MembershipUpdateEvent
|
||||
|
||||
`MembershipUpdateEvent` is a message used to propagate information about group membership changes in a group chat.
|
||||
The details are in the [Group chats specs](/status/deprecated/group-chat.md).
|
||||
|
||||
## Upgradability
|
||||
|
||||
There are two ways to upgrade the protocol without breaking compatibility:
|
||||
|
||||
* A node always supports accretion
|
||||
* A node does not support deletion of existing fields or messages, which might break compatibility
|
||||
|
||||
## Security Considerations
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
* Added language to include Waku in all relevant places
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
[Status Whitepaper](https://status.im/whitepaper.pdf)
|
||||
[protobuf record](https://developers.google.com/protocol-buffers/)
|
||||
[Protobuf](https://developers.google.com/protocol-buffers)
|
||||
[Status user account](/status/deprecated/account.md)
|
||||
[ens name or the 3-word pseudonym corresponding to the key](deprecated/account/#contact-verification)
|
||||
[WHISPER-USAGE](/status/deprecated/whisper-usage.md)
|
||||
[WAKU-USAGE](/status/deprecated/waku-usage.md)
|
||||
[Lamport timestamps](https://en.wikipedia.org/wiki/Lamport_timestamps)
|
||||
[Group chats specs](/status/deprecated/group-chat.md)
|
||||
[May 22, 2020 change commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
753
status/deprecated/push-notification-server.md
Normal file
753
status/deprecated/push-notification-server.md
Normal file
@@ -0,0 +1,753 @@
|
||||
---
|
||||
title: PUSH-NOTIFICATION-SERVER
|
||||
name: Push notification server
|
||||
status: deprecated
|
||||
description: Status provides a set of Push notification services that can be used to achieve this functionality.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
---
|
||||
|
||||
## Reason
|
||||
|
||||
Push notifications for iOS devices and some Android devices can only be implemented by relying on [APN service](https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/APNSOverview.html#//apple_ref/doc/uid/TP40008194-CH8-SW1) for iOS or [Firebase](https://firebase.google.com/).
|
||||
|
||||
This is useful for Android devices that do not support foreground services
|
||||
or that often kill the foreground service.
|
||||
|
||||
iOS only allows certain kind of applications to keep a connection open when in the
|
||||
background, VoIP for example, which current status client does not qualify for.
|
||||
|
||||
Applications on iOS can also request execution time when they are in the [background](https://developer.apple.com/documentation/uikit/app_and_environment/scenes/preparing_your_ui_to_run_in_the_background/updating_your_app_with_background_app_refresh)
|
||||
but it has a limited set of use cases, for example it won't schedule any time
|
||||
if the application was force quit,
|
||||
and generally is not responsive enough to implement a push notification system.
|
||||
|
||||
Therefore Status provides a set of Push notification services
|
||||
that can be used to achieve this functionality.
|
||||
|
||||
Because this can't be safely implemented in a privacy preserving manner,
|
||||
clients MUST be given an option to opt-in to receiving and sending push notifications.
|
||||
They are disabled by default.
|
||||
|
||||
## Requirements
|
||||
|
||||
The party releasing the app MUST possess a certificate for the Apple Push Notification service
|
||||
and its has to run a [gorush](https://github.com/appleboy/gorush) publicly accessible server for sending the actual notification.
|
||||
The party releasing the app, Status in this case, needs to run its own [gorush](https://github.com/appleboy/gorush)
|
||||
|
||||
## Components
|
||||
|
||||
### Gorush instance
|
||||
|
||||
A [gorush](https://github.com/appleboy/gorush) instance MUST be publicly available,
|
||||
this will be used only by push notification servers.
|
||||
|
||||
### Push notification server
|
||||
|
||||
A push notification server used by clients to register for receiving and sending push notifications.
|
||||
|
||||
### Registering client
|
||||
|
||||
A Status client that wants to receive push notifications
|
||||
|
||||
### Sending client
|
||||
|
||||
A Status client that wants to send push notifications
|
||||
|
||||
## Registering with the push notification service
|
||||
|
||||
A client MAY register with one or more Push Notification services of their choice.
|
||||
|
||||
A `PNR message` (Push Notification Registration) MUST be sent to the [partitioned topic](status/deprecated/waku-usage/#partitioned-topic)
|
||||
for the public key of the node, encrypted with this key.
|
||||
|
||||
The message MUST be wrapped in a [`ApplicationMetadataMessage`](status/deprecated/payload/#payload-wrapper) with type set to `PUSH_NOTIFICATION_REGISTRATION`.
|
||||
|
||||
The marshaled protobuf payload MUST also be encrypted with AES-GCM
|
||||
using the Diffie–Hellman key generated from the client and server identity.
|
||||
|
||||
This is done in order to ensure that the extracted key from the signature will be
|
||||
considered invalid if it can't decrypt the payload.
|
||||
|
||||
The content of the message MUST contain the following [protobuf record](https://developers.google.com/protocol-buffers/):
|
||||
|
||||
```protobuf
|
||||
message PushNotificationRegistration {
|
||||
enum TokenType {
|
||||
UNKNOWN_TOKEN_TYPE = 0;
|
||||
APN_TOKEN = 1;
|
||||
FIREBASE_TOKEN = 2;
|
||||
}
|
||||
TokenType token_type = 1;
|
||||
string device_token = 2;
|
||||
string installation_id = 3;
|
||||
string access_token = 4;
|
||||
bool enabled = 5;
|
||||
uint64 version = 6;
|
||||
repeated bytes allowed_key_list = 7;
|
||||
repeated bytes blocked_chat_list = 8;
|
||||
bool unregister = 9;
|
||||
bytes grant = 10;
|
||||
bool allow_from_contacts_only = 11;
|
||||
string apn_topic = 12;
|
||||
bool block_mentions = 13;
|
||||
repeated bytes allowed_mentions_chat_list = 14;
|
||||
}
|
||||
```
|
||||
|
||||
A push notification server will handle the message according to the following rules:
|
||||
|
||||
- it MUST extract the public key of the sender from the signature and verify that
|
||||
the payload can be decrypted successfully.
|
||||
- it MUST verify that `token_type` is supported
|
||||
- it MUST verify that `device_token` is non empty
|
||||
- it MUST verify that `installation_id` is non empty
|
||||
- it MUST verify that `version` is non-zero and greater than the currently stored version for the public key and installation id of the sender, if any
|
||||
- it MUST verify that `grant` is non empty and according to the [specs](#server-grant)
|
||||
- it MUST verify that `access_token` is a valid [`uuid`](https://tools.ietf.org/html/rfc4122)
|
||||
- it MUST verify that `apn_topic` is set if `token_type` is `APN_TOKEN`
|
||||
|
||||
If the message can't be decrypted, the message MUST be discarded.
|
||||
|
||||
If `token_type` is not supported, a response MUST be sent with `error` set to
|
||||
`UNSUPPORTED_TOKEN_TYPE`.
|
||||
|
||||
If `token`,`installation_id`,`device_tokens`,`version` are empty, a response MUST
|
||||
be sent with `error` set to `MALFORMED_MESSAGE`.
|
||||
|
||||
If the `version` is equal or less than the currently stored version, a response MUST
|
||||
be sent with `error` set to `VERSION_MISMATCH`.
|
||||
|
||||
If any other error occurs the `error` should be set to `INTERNAL_ERROR`.
|
||||
|
||||
If the response is successful `success` MUST be set to `true` otherwise a response MUST be sent with `success` set to `false`.
|
||||
|
||||
`request_id` should be set to the `SHAKE-256` of the encrypted payload.
|
||||
|
||||
The response MUST be sent on the [partitioned topic](status/deprecated/waku-usage.md#partitioned-topic) of the sender
|
||||
and MUST not be encrypted using the [secure transport](status/deprecated/secure-transport) to facilitate the usage of ephemeral keys.
|
||||
|
||||
The payload of the response is:
|
||||
|
||||
```protobuf
|
||||
message PushNotificationRegistrationResponse {
|
||||
bool success = 1;
|
||||
ErrorType error = 2;
|
||||
bytes request_id = 3;
|
||||
|
||||
enum ErrorType {
|
||||
UNKNOWN_ERROR_TYPE = 0;
|
||||
MALFORMED_MESSAGE = 1;
|
||||
VERSION_MISMATCH = 2;
|
||||
UNSUPPORTED_TOKEN_TYPE = 3;
|
||||
INTERNAL_ERROR = 4;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The message MUST be wrapped in a [`ApplicationMetadataMessage`](status/deprecated/payloads.md#payload-wrapper) with type set to `PUSH_NOTIFICATION_REGISTRATION_RESPONSE`.
|
||||
|
||||
A client SHOULD listen for a response sent on the [partitioned topic](status/deprecated/waku-usage/#partitioned-topic)
|
||||
that the key used to register.
|
||||
|
||||
If `success` is `true` the client has registered successfully.
|
||||
|
||||
If `success` is `false`:
|
||||
|
||||
- If `MALFORMED_MESSAGE` is returned, the request SHOULD NOT be retried without ensuring that it is correctly formed.
|
||||
- If `INTERNAL_ERROR` is returned, the request MAY be retried, but the client MUST backoff exponentially
|
||||
|
||||
A client MAY register with multiple Push Notification Servers in order to increase availability.
|
||||
|
||||
A client SHOULD make sure that all the notification services they registered with have the same information about their tokens.
|
||||
|
||||
If no response is returned the request SHOULD be considered failed and MAY be retried with the same server or a different one, but clients MUST exponentially backoff after each trial.
|
||||
|
||||
If the request is successful the token SHOULD be [advertised](#advertising-a-push-notification-server) as described below
|
||||
|
||||
### Query topic
|
||||
|
||||
On successful registration the server MUST be listening to the topic derived from:
|
||||
|
||||
```protobuf
|
||||
0XHexEncode(Shake256(CompressedClientPublicKey))
|
||||
```
|
||||
|
||||
Using the topic derivation algorithm described [here](status/deprecated/waku-usage/#public-chats)
|
||||
and listen for client queries.
|
||||
|
||||
### Server grant
|
||||
|
||||
A push notification server needs to demonstrate to a client that it was authorized
|
||||
by the client to send them push notifications. This is done by building
|
||||
a grant which is specific to a given client-server pair.
|
||||
The grant is built as follow:
|
||||
|
||||
```protobuf
|
||||
Signature(Keccak256(CompressedPublicKeyOfClient . CompressedPublicKeyOfServer . AccessToken), PrivateKeyOfClient)
|
||||
```
|
||||
|
||||
When receiving a grant the server MUST be validate that the signature matches the registering client.
|
||||
|
||||
## Re-registering with the push notification server
|
||||
|
||||
A client SHOULD re-register with the node if the APN or FIREBASE token changes.
|
||||
|
||||
When re-registering a client SHOULD ensure that it has the most up-to-date
|
||||
`PushNotificationRegistration` and increment `version` if necessary.
|
||||
|
||||
Once re-registered, a client SHOULD advertise the changes.
|
||||
|
||||
## Changing options
|
||||
|
||||
This is handled in exactly the same way as re-registering above.
|
||||
|
||||
## Unregistering from push notifications
|
||||
|
||||
To unregister a client MUST send a `PushNotificationRegistration` request as described
|
||||
above with `unregister` set to `true`, or removing
|
||||
their device information.
|
||||
|
||||
The server MUST remove all data about this user if `unregistering` is `true`,
|
||||
apart from the `hash` of the public key and the `version` of the last options,
|
||||
in order to make sure that old messages are not processed.
|
||||
|
||||
A client MAY unregister from a server on explicit logout if multiple chat keys
|
||||
are used on a single device.
|
||||
|
||||
## Advertising a push notification server
|
||||
|
||||
Each user registered with one or more push notification servers SHOULD
|
||||
advertise periodically the push notification services that they have registered with for each device they own.
|
||||
|
||||
```protobuf
|
||||
message PushNotificationQueryInfo {
|
||||
string access_token = 1;
|
||||
string installation_id = 2;
|
||||
bytes public_key = 3;
|
||||
repeated bytes allowed_user_list = 4;
|
||||
bytes grant = 5;
|
||||
uint64 version = 6;
|
||||
bytes server_public_key = 7;
|
||||
}
|
||||
|
||||
message ContactCodeAdvertisement {
|
||||
repeated PushNotificationQueryInfo push_notification_info = 1;
|
||||
}
|
||||
```
|
||||
|
||||
The message MUST be wrapped in a [`ApplicationMetadataMessage`](status/deprecated/payloads/#payload-wrapper) with type set to `PUSH_NOTIFICATION_QUERY_INFO`.
|
||||
|
||||
If no filtering is done based on public keys,
|
||||
the access token SHOULD be included in the advertisement.
|
||||
Otherwise it SHOULD be left empty.
|
||||
|
||||
This SHOULD be advertised on the [contact code topic](/status/deprecated/waku-usage.md#contact-code-topic)
|
||||
and SHOULD be coupled with normal contact-code advertisement.
|
||||
|
||||
Every time a user register or re-register with a push notification service, their
|
||||
contact-code SHOULD be re-advertised.
|
||||
|
||||
Multiple servers MAY be advertised for the same `installation_id` for redundancy reasons.
|
||||
|
||||
## Discovering a push notification server
|
||||
|
||||
To discover a push notification service for a given user, their [contact code topic](status/deprecated/waku-usage/#contact-code-topic)
|
||||
SHOULD be listened to.
|
||||
A mailserver can be queried for the specific topic to retrieve the most up-to-date
|
||||
contact code.
|
||||
|
||||
## Querying the push notification server
|
||||
|
||||
If a token is not present in the latest advertisement for a user, the server
|
||||
SHOULD be queried directly.
|
||||
|
||||
To query a server a message:
|
||||
|
||||
```protobuf
|
||||
message PushNotificationQuery {
|
||||
repeated bytes public_keys = 1;
|
||||
}
|
||||
```
|
||||
|
||||
The message MUST be wrapped in a [`ApplicationMetadataMessage`](status/deprecated/payloads/#payload-wrapper) with type set to `PUSH_NOTIFICATION_QUERY`.
|
||||
|
||||
MUST be sent to the server on the topic derived from the hashed public key of the
|
||||
key we are querying, as [described above](#query-topic).
|
||||
|
||||
An ephemeral key SHOULD be used and SHOULD NOT be encrypted using the [secure transport](status/deprecated/secure-transport.md).
|
||||
|
||||
If the server has information about the client a response MUST be sent:
|
||||
|
||||
```protobuf
|
||||
message PushNotificationQueryInfo {
|
||||
string access_token = 1;
|
||||
string installation_id = 2;
|
||||
bytes public_key = 3;
|
||||
repeated bytes allowed_user_list = 4;
|
||||
bytes grant = 5;
|
||||
uint64 version = 6;
|
||||
bytes server_public_key = 7;
|
||||
}
|
||||
|
||||
message PushNotificationQueryResponse {
|
||||
repeated PushNotificationQueryInfo info = 1;
|
||||
bytes message_id = 2;
|
||||
bool success = 3;
|
||||
}
|
||||
```
|
||||
|
||||
A `PushNotificationQueryResponse` message MUST be wrapped in a [`ApplicationMetadataMessage`](status/deprecated/payloads.md#payload-wrapper) with type set to `PUSH_NOTIFICATION_QUERY_RESPONSE`.
|
||||
|
||||
Otherwise a response MUST NOT be sent.
|
||||
|
||||
If `allowed_key_list` is not set `access_token` MUST be set and `allowed_key_list` MUST NOT
|
||||
be set.
|
||||
|
||||
If `allowed_key_list` is set `allowed_key_list` MUST be set and `access_token` MUST NOT be set.
|
||||
|
||||
If `access_token` is returned, the `access_token` SHOULD be used to send push notifications.
|
||||
|
||||
If `allowed_key_list` are returned, the client SHOULD decrypt each
|
||||
token by generating an `AES-GCM` symmetric key from the Diffie–Hellman between the
|
||||
target client and itself
|
||||
If AES decryption succeeds it will return a valid [`uuid`](https://tools.ietf.org/html/rfc4122) which is what is used for access_token.
|
||||
The token SHOULD be used to send push notifications.
|
||||
|
||||
The response MUST be sent on the [partitioned topic](status/deprecated/waku-usage/#partitioned-topic) of the sender
|
||||
and MUST not be encrypted using the [secure transport](status/deprecated/secure-transport.md) to facilitate
|
||||
the usage of ephemeral keys.
|
||||
|
||||
On receiving a response a client MUST verify `grant` to ensure that the server
|
||||
has been authorized to send push notification to a given client.
|
||||
|
||||
## Sending a push notification
|
||||
|
||||
When sending a push notification, only the `installation_id` for the devices targeted
|
||||
by the message SHOULD be used.
|
||||
|
||||
If a message is for all the user devices, all the `installation_id` known to the client MAY be used.
|
||||
|
||||
The number of devices MAY be capped in order to reduce resource consumption.
|
||||
|
||||
At least 3 devices SHOULD be targeted, ordered by last activity.
|
||||
|
||||
For any device that a token is available, or that a token is successfully queried,
|
||||
a push notification message SHOULD be sent to the corresponding push notification server.
|
||||
|
||||
```protobuf
|
||||
message PushNotification {
|
||||
string access_token = 1;
|
||||
string chat_id = 2;
|
||||
bytes public_key = 3;
|
||||
string installation_id = 4;
|
||||
bytes message = 5;
|
||||
PushNotificationType type = 6;
|
||||
enum PushNotificationType {
|
||||
UNKNOWN_PUSH_NOTIFICATION_TYPE = 0;
|
||||
MESSAGE = 1;
|
||||
MENTION = 2;
|
||||
}
|
||||
bytes author = 7;
|
||||
}
|
||||
|
||||
message PushNotificationRequest {
|
||||
repeated PushNotification requests = 1;
|
||||
bytes message_id = 2;
|
||||
}
|
||||
```
|
||||
|
||||
A `PushNotificationRequest` message MUST be wrapped in a [`ApplicationMetadataMessage`](/status/deprecated/payloads.md#payload-wrapper) with type set to `PUSH_NOTIFICATION_REQUEST`.
|
||||
|
||||
Where `message` is the encrypted payload of the message and `chat_id` is the
|
||||
`SHAKE-256` of the `chat_id`.
|
||||
`message_id` is the id of the message
|
||||
`author` is the `SHAKE-256` of the public key of the sender.
|
||||
|
||||
If multiple server are available for a given push notification, only one notification
|
||||
MUST be sent.
|
||||
|
||||
If no response is received
|
||||
a client SHOULD wait at least 3 seconds, after which the request MAY be retried against a different server
|
||||
|
||||
This message SHOULD be sent using an ephemeral key.
|
||||
|
||||
On receiving the message, the push notification server MUST validate the access token.
|
||||
If the access token is valid, a notification MUST be sent to the gorush instance with the
|
||||
following data:
|
||||
|
||||
```protobuf
|
||||
{
|
||||
"notifications": [
|
||||
{
|
||||
"tokens": ["token_a", "token_b"],
|
||||
"platform": 1,
|
||||
"message": "You have a new message",
|
||||
"data": {
|
||||
"chat_id": chat_id,
|
||||
"message": message,
|
||||
"installation_ids": [installation_id_1, installation_id_2]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Where platform is `1` for IOS and `2` for Firebase, according to the [gorush documentation](https://github.com/appleboy/gorush)
|
||||
|
||||
A server MUST return a response message:
|
||||
|
||||
```protobuf
|
||||
message PushNotificationReport {
|
||||
bool success = 1;
|
||||
ErrorType error = 2;
|
||||
enum ErrorType {
|
||||
UNKNOWN_ERROR_TYPE = 0;
|
||||
WRONG_TOKEN = 1;
|
||||
INTERNAL_ERROR = 2;
|
||||
NOT_REGISTERED = 3;
|
||||
}
|
||||
bytes public_key = 3;
|
||||
string installation_id = 4;
|
||||
}
|
||||
|
||||
message PushNotificationResponse {
|
||||
bytes message_id = 1;
|
||||
repeated PushNotificationReport reports = 2;
|
||||
}
|
||||
```
|
||||
|
||||
A `PushNotificationResponse` message MUST be wrapped in a [`ApplicationMetadataMessage`](/status/deprecated/payloads.md#payload-wrapper) with type set to `PUSH_NOTIFICATION_RESPONSE`.
|
||||
|
||||
Where `message_id` is the `message_id` sent by the client.
|
||||
|
||||
The response MUST be sent on the [partitioned topic](/status/deprecated/waku-usage.md#partitioned-topic) of the sender
|
||||
and MUST not be encrypted using the [secure transport](/status/deprecated/secure-transport.md) to facilitate
|
||||
the usage of ephemeral keys.
|
||||
|
||||
If the request is accepted `success` MUST be set to `true`.
|
||||
Otherwise `success` MUST be set to `false`.
|
||||
|
||||
If `error` is `BAD_TOKEN` the client MAY query again the server for the token and
|
||||
retry the request.
|
||||
|
||||
If `error` is `INTERNAL_ERROR` the client MAY retry the request.
|
||||
|
||||
## Flow
|
||||
|
||||
### Registration process
|
||||
|
||||
- A client will generate a notification token through `APN` or `Firebase`.
|
||||
- The client will [register](#registering-with-the-push-notification-service) with one or more push notification server of their choosing.
|
||||
- The server should process the response and respond according to the success of the operation
|
||||
- If the request is not successful it might be retried, and adjusted according to the response. A different server can be also used.
|
||||
- Once the request is successful the client should [advertise](#advertising-a-push-notification-server) the new coordinates
|
||||
|
||||
### Sending a notification
|
||||
|
||||
- A client should prepare a message and extract the targeted installation-ids
|
||||
- It should retrieve the most up to date information for a given user, either by
|
||||
querying a push notification server, a mailserver if not listening already to the given topic, or checking
|
||||
the database locally
|
||||
- It should then [send](#sending-a-push-notification) a push notification according
|
||||
to the rules described
|
||||
- The server should then send a request to the gorush server including all the required
|
||||
information
|
||||
|
||||
### Receiving a push notification
|
||||
|
||||
- On receiving the notification, a client can open the right account by checking the
|
||||
`installation_id` included. The `chat_id` MAY be used to open the chat if present.
|
||||
- `message` can be decrypted and presented to the user. Otherwise messages can be pulled from the mailserver if the `message_id` is no already present.
|
||||
|
||||
## Protobuf description
|
||||
|
||||
### PushNotificationRegistration
|
||||
|
||||
`token_type`: the type of token. Currently supported is `APN_TOKEN` for Apple Push
|
||||
`device_token`: the actual push notification token sent by `Firebase` or `APN`
|
||||
and `FIREBASE_TOKEN` for firebase.
|
||||
`installation_id`: the [`installation_id`](/status/deprecated/account.md) of the device
|
||||
`access_token`: the access token that will be given to clients to send push notifications
|
||||
`enabled`: whether the device wants to be sent push notifications
|
||||
`version`: a monotonically increasing number identifying the current `PushNotificationRegistration`. Any time anything is changed in the record it MUST be increased by the client, otherwise the request will not be accepted.
|
||||
`allowed_key_list`: a list of `access_token` encrypted with the AES key generated
|
||||
by Diffie–Hellman between the publisher and the allowed
|
||||
contact.
|
||||
`blocked_chat_list`: a list of `SHA2-256` hashes of chat ids.
|
||||
Any chat id in this list will not trigger a notification.
|
||||
`unregister`: whether the account should be unregistered
|
||||
`grant`: the grant for this specific server
|
||||
`allow_from_contacts_only`: whether the client only wants push notifications from contacts
|
||||
`apn_topic`: the APN topic for the push notification
|
||||
`block_mentions`: whether the client does not want to be notified on mentions
|
||||
`allowed_mentions_chat_list`: a list of `SHA2-256` hashes of chat ids where we want to receive mentions
|
||||
|
||||
#### Data disclosed
|
||||
|
||||
- Type of device owned by a given user
|
||||
- The `FIREBASE` or `APN` push notification token
|
||||
- Hash of the chat_id a user is not interested in for notifications
|
||||
- The times a push notification record has been modified by the user
|
||||
- The number of contacts a client has, in case `allowed_key_list` is set
|
||||
|
||||
### PushNotificationRegistrationResponse
|
||||
|
||||
`success`: whether the registration was successful
|
||||
`error`: the error type, if any
|
||||
`request_id`: the `SHAKE-256` hash of the `signature` of the request
|
||||
`preferences`: the server stored preferences in case of an error
|
||||
|
||||
### ContactCodeAdvertisement
|
||||
|
||||
`push_notification_info`: the information for each device advertised
|
||||
|
||||
Data disclosed
|
||||
|
||||
- The chat key of the sender
|
||||
|
||||
### PushNotificationQuery
|
||||
|
||||
`public_keys`: the `SHAKE-256` of the public keys the client is interested in
|
||||
|
||||
Data disclosed
|
||||
|
||||
- The hash of the public keys the client is interested in
|
||||
|
||||
### PushNotificationQueryInfo
|
||||
|
||||
`access_token`: the access token used to send a push notification
|
||||
`installation_id`: the `installation_id` of the device associated with the `access_token`
|
||||
`public_key`: the `SHAKE-256` of the public key associated with this `access_token` and `installation_id`
|
||||
`allowed_key_list`: a list of encrypted access tokens to be returned
|
||||
to the client in case there's any filtering on public keys in place.
|
||||
`grant`: the grant used to register with this server.
|
||||
`version`: the version of the registration on the server.
|
||||
`server_public_key`: the compressed public key of the server.
|
||||
|
||||
### PushNotificationQueryResponse
|
||||
|
||||
`info`: a list of `PushNotificationQueryInfo`.
|
||||
`message_id`: the message id of the `PushNotificationQueryInfo` the server is replying to.
|
||||
`success`: whether the query was successful.
|
||||
|
||||
### PushNotification
|
||||
|
||||
`access_token`: the access token used to send a push notification.
|
||||
`chat_id`: the `SHAKE-256` of the `chat_id`.
|
||||
`public_key`: the `SHAKE-256` of the compressed public key of the receiving client.
|
||||
`installation_id`: the installation id of the receiving client.
|
||||
`message`: the encrypted message that is being notified on.
|
||||
`type`: the type of the push notification, either `MESSAGE` or `MENTION`
|
||||
`author`: the `SHAKE-256` of the public key of the sender
|
||||
|
||||
Data disclosed
|
||||
|
||||
- The `SHAKE-256` of the `chat_id` the notification is to be sent for
|
||||
- The cypher text of the message
|
||||
- The `SHAKE-256` of the public key of the sender
|
||||
- The type of notification
|
||||
|
||||
### PushNotificationRequest
|
||||
|
||||
`requests`: a list of `PushNotification`
|
||||
`message_id`: the [status message id](/status/deprecated/payloads.md)
|
||||
|
||||
Data disclosed
|
||||
|
||||
- The status message id for which the notification is for
|
||||
|
||||
### PushNotificationResponse
|
||||
|
||||
`message_id`: the `message_id` being notified on.
|
||||
`reports`: a list of `PushNotificationReport`
|
||||
|
||||
### PushNotificationReport
|
||||
|
||||
`success`: whether the push notification was successful.
|
||||
`error`: the type of the error in case of failure.
|
||||
`public_key`: the public key of the user being notified.
|
||||
`installation_id`: the installation id of the user being notified.
|
||||
|
||||
## Anonymous mode of operations
|
||||
|
||||
An anonymous mode of operations MAY be provided by the client, where the
|
||||
responsibility of propagating information about the user is left to the client,
|
||||
in order to preserve privacy.
|
||||
|
||||
A client in anonymous mode can register with the server using a key different
|
||||
from their chat key.
|
||||
This will hide their real chat key.
|
||||
|
||||
This public key is effectively a secret and SHOULD only be disclosed to clients that you the user wants to be notified by.
|
||||
|
||||
A client MAY advertise the access token on the contact-code topic of the key generated.
|
||||
A client MAY share their public key through [contact updates](/status/deprecated/payloads.md#contact-update)
|
||||
|
||||
A client receiving a push notification public key SHOULD listen to the contact code
|
||||
topic of the push notification public key for updates.
|
||||
|
||||
The method described above effectively does not share the identity of the sender
|
||||
nor the receiver to the server, but MAY result in missing push notifications as
|
||||
the propagation of the secret is left to the client.
|
||||
|
||||
This can be mitigated by [device syncing](/status/deprecated/payloads.md), but not completely
|
||||
addressed.
|
||||
|
||||
## Security considerations
|
||||
|
||||
If no anonymous mode is used, when registering with a push notification service a client discloses:
|
||||
|
||||
- The chat key
|
||||
- The devices that will receive notifications
|
||||
|
||||
A client MAY disclose:
|
||||
|
||||
- The hash of the chat_ids they want to filter out
|
||||
|
||||
When running in anonymous mode, the client's chat key is not disclosed.
|
||||
|
||||
When querying a push notification server a client will disclose:
|
||||
|
||||
- That it is interested in sending push notification to another client,
|
||||
but the querying client's chat key is not disclosed
|
||||
|
||||
When sending a push notification a client discloses:
|
||||
|
||||
- The `SHAKE-256` of the chat id
|
||||
|
||||
[//]: This section can be removed, for now leaving it here in order to help with the
|
||||
review process. Point can be integrated, suggestion welcome.
|
||||
|
||||
## FAQ
|
||||
|
||||
### Why having ACL done at the server side and not the client?
|
||||
|
||||
We looked into silent notification for
|
||||
[IOS](https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/pushing_background_updates_to_your_app) (android has no equivalent)
|
||||
but can't be used as it's expected to receive maximum 2/3 per hour, so not our use case. There
|
||||
are also issue when the user force quit the app.
|
||||
|
||||
### Why using an access token?
|
||||
|
||||
The access token is used to decouple the requesting information from the user from
|
||||
actually sending the push notification.
|
||||
|
||||
Some ACL is necessary otherwise it would be too easy to spam users (it's still fairly
|
||||
trivial, but with this method you could allow only contacts to send you push notifications).
|
||||
|
||||
Therefore your identity must be revealed to the server either when sending or querying.
|
||||
|
||||
By using an access token we increase deniability, as the server would know
|
||||
who requested the token but not necessarily who sent a push notification.
|
||||
Correlation between the two can be trivial in some cases.
|
||||
|
||||
This also allows a mode of use as we had before, where the server does not propagate
|
||||
info at all, and it's left to the user to propagate the token, through contact requests
|
||||
for example.
|
||||
|
||||
### Why advertise with the bundle?
|
||||
|
||||
Advertising with the bundle allows us to piggy-back on an already implemented behavior
|
||||
and save some bandwidth in cases where is not filtering by public keys
|
||||
|
||||
### What's the bandwidth impact for this?
|
||||
|
||||
Generally speaking, for each 1-to-1 message and group chat message you will sending
|
||||
1 and `number of participants` push notifications. This can be optimized if
|
||||
multiple users are using the same push notification server. Queries have also
|
||||
a bandwidth impact but they are made only when actually needed
|
||||
|
||||
### What's the information disclosed?
|
||||
|
||||
The data disclosed with each message sent by the client is above, but for a summary:
|
||||
|
||||
When you register with a push notification service you may disclose:
|
||||
|
||||
1) Your chat key
|
||||
2) Which devices you have
|
||||
3) The hash of the chat_ids you want to filter out
|
||||
4) The hash of the public keys you are interested/not interested in
|
||||
|
||||
When you query a notification service you may disclose:
|
||||
|
||||
1) Your chat key
|
||||
2) The fact that you are interested in sending push notification to a given user
|
||||
|
||||
Effectively this is fairly revealing if the user has a whitelist implemented.
|
||||
Therefore sending notification should be optional.
|
||||
|
||||
### What prevents a user from generating a random key and getting an access token and spamming?
|
||||
|
||||
Nothing really, that's the same as the status app as a whole. the only mechanism that prevents
|
||||
this is using a white-list as described above,
|
||||
but that implies disclosing your true identity to the push notification server.
|
||||
|
||||
### Why not 0-knowledge proofs/quantum computing
|
||||
|
||||
We start simple, we can iterate
|
||||
|
||||
### How to handle backward/forward compatibility
|
||||
|
||||
Most of the request have a target, so protocol negotiation can happen. We cannot negotiated
|
||||
the advertisement as that's effectively a broadcast, but those info should not change and we can
|
||||
always accrete the message.
|
||||
|
||||
### Why ack_key?
|
||||
|
||||
That's necessary to avoid duplicated push notifications and allow for the retry
|
||||
in case the notification is not successful.
|
||||
|
||||
Deduplication of the push notification is done on the client side, to reduce a bit
|
||||
of centralization and also in order not to have to modify gorush.
|
||||
|
||||
### Can I run my own node?
|
||||
|
||||
Sure, the methods allow that
|
||||
|
||||
### Can I register with multiple nodes for redundancy
|
||||
|
||||
Yep
|
||||
|
||||
### What does my node disclose?
|
||||
|
||||
Your node will disclose the IP address is running from, as it makes an HTTP post to
|
||||
gorush. A waku adapter could be used, but please not now.
|
||||
|
||||
### Does this have high-reliability requirements?
|
||||
|
||||
The gorush server yes, no way around it.
|
||||
|
||||
The rest, kind of, at least one node having your token needs to be up for you to receive notifications.
|
||||
But you can register with multiple servers (desktop, status, etc) if that's a concern.
|
||||
|
||||
### Can someone else (i.e not status) run this?
|
||||
|
||||
Push notification servers can be run by anyone. Gorush can be run by anyone I take,
|
||||
but we are in charge of the certificate, so they would not be able to notify status-clients.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.1
|
||||
|
||||
[Released](https://github.com/status-im/specs/commit/)
|
||||
|
||||
- Initial version
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [APN Service](https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/APNSOverview.html#//apple_ref/doc/uid/TP40008194-CH8-SW1)
|
||||
- [Background Execution on iOS](https://developer.apple.com/documentation/uikit/app_and_environment/scenes/preparing_your_ui_to_run_in_the_background/updating_your_app_with_background_app_refresh)
|
||||
- [Firebase](https://firebase.google.com/)
|
||||
- [Gorush](https://github.com/appleboy/gorush)
|
||||
- [UUID Specification](https://tools.ietf.org/html/rfc4122)
|
||||
- [Secure Transport](/status/deprecated/secure-transport.md)
|
||||
- [Silent Notifications on iOS](https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/pushing_background_updates_to_your_app)
|
||||
- [Waku Usage](/status/deprecated/waku-usage.md)
|
||||
- [ENS Contract](https://github.com/ensdomains/ens)
|
||||
- [Payloads](/status/deprecated/payloads.md)
|
||||
583
status/deprecated/secure-transport.md
Normal file
583
status/deprecated/secure-transport.md
Normal file
@@ -0,0 +1,583 @@
|
||||
---
|
||||
title: SECURE-TRANSPORT
|
||||
name: Secure Transport
|
||||
status: deprecated
|
||||
description: This document describes how Status provides a secure channel between two peers, providing confidentiality, integrity, authenticity, and forward secrecy.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Pedro Pombeiro <pedro@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This document describes how Status provides a secure channel between two peers,
|
||||
and thus provide confidentiality, integrity, authenticity and forward secrecy.
|
||||
It is transport-agnostic and works over asynchronous networks.
|
||||
|
||||
It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/) and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/) specifications, with some adaptations to operate in a decentralized environment.
|
||||
|
||||
## Introduction
|
||||
|
||||
This document describes how nodes establish a secure channel,
|
||||
and how various conversational security properties are achieved.
|
||||
|
||||
### Definitions
|
||||
|
||||
- **Perfect Forward Secrecy** is a feature of specific key-agreement protocols
|
||||
which provide assurances that session keys will not be compromised even if the private keys of the participants are compromised.
|
||||
Specifically, past messages cannot be decrypted by a third-party who manages to get a hold of a private key.
|
||||
|
||||
- **Secret channel** describes a communication channel where Double Ratchet algorithm is in use.
|
||||
|
||||
### Design Requirements
|
||||
|
||||
- **Confidentiality**: The adversary should not be able to learn what data is being exchanged between two Status clients.
|
||||
- **Authenticity**: The adversary should not be able to cause either endpoint of a Status 1:1 chat
|
||||
to accept data from any third party as though it came from the other endpoint.
|
||||
- **Forward Secrecy**: The adversary should not be able to learn
|
||||
what data was exchanged between two Status clients if, at some later time,
|
||||
the adversary compromises one or both of the endpoint devices.
|
||||
- **Integrity**: The adversary should not be able to cause either endpoint of a Status 1:1 chat
|
||||
to accept data that has been tampered with.
|
||||
|
||||
All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
|
||||
|
||||
### Conventions
|
||||
|
||||
Types used in this specification are defined using [Protobuf](https://developers.google.com/protocol-buffers/).
|
||||
|
||||
### Transport Layer
|
||||
|
||||
[Whisper](status/deprecated/whisper-usage) and [Waku](status/deprecated/waku-usage) serves as the transport layers for the Status chat protocol.
|
||||
|
||||
### User flow for 1-to-1 communications
|
||||
|
||||
#### Account generation
|
||||
|
||||
See [Account specification](status/deprecated/account)
|
||||
|
||||
#### Account recovery
|
||||
|
||||
If Alice later recovers her account, the Double Ratchet state information will not be available,
|
||||
so she is no longer able to decrypt any messages received from existing contacts.
|
||||
|
||||
If an incoming message (on the same Whisper/Waku topic) fails to decrypt,
|
||||
the node replies a message with the current bundle, so that the node notifies the other end of the new device.
|
||||
Subsequent communications will use this new bundle.
|
||||
|
||||
## Messaging
|
||||
|
||||
All 1:1 and group chat messaging in Status is subject to end-to-end encryption
|
||||
to provide users with a strong degree of privacy and security.
|
||||
Public chat messages are publicly readable by anyone since there's no permission model
|
||||
for who is participating in a public chat.
|
||||
|
||||
The rest of this document is purely about 1:1 and private group chat.
|
||||
Private group chat largely reduces to 1:1 chat, since there's a secure channel between each pair-wise participant.
|
||||
|
||||
### End-to-end encryption
|
||||
|
||||
End-to-end encryption (E2EE) takes place between two clients.
|
||||
The main cryptographic protocol is a [Status implementation](https://github.com/status-im/doubleratchet/) of the Double Ratchet protocol,
|
||||
which is in turn derived from the [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html), using a different ratchet.
|
||||
The transport protocol subsequently encrypt the message payload - Whisper/Waku (see section [Transport Layer](#transport-layer)) -, using symmetric key encryption.
|
||||
Furthermore, Status uses the concept of prekeys (through the use of [X3DH](https://signal.org/docs/specifications/x3dh/))
|
||||
to allow the protocol to operate in an asynchronous environment.
|
||||
It is not necessary for two parties to be online at the same time to initiate an encrypted conversation.
|
||||
|
||||
Status uses the following cryptographic primitives:
|
||||
|
||||
- Whisper/Waku
|
||||
- AES-256-GCM
|
||||
- ECIES
|
||||
- ECDSA
|
||||
- KECCAK-256
|
||||
- X3DH
|
||||
- Elliptic curve Diffie-Hellman key exchange (secp256k1)
|
||||
- KECCAK-256
|
||||
- ECDSA
|
||||
- ECIES
|
||||
- Double Ratchet
|
||||
- HMAC-SHA-256 as MAC
|
||||
- Elliptic curve Diffie-Hellman key exchange (Curve25519)
|
||||
- AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key
|
||||
|
||||
The node achieves key derivation using HKDF.
|
||||
|
||||
### Prekeys
|
||||
|
||||
Every client initially generates some key material which is stored locally:
|
||||
|
||||
- Identity keypair based on secp256k1 - `IK`
|
||||
- A signed prekey based on secp256k1 - `SPK`
|
||||
- A prekey signature - `Sig(IK, Encode(SPK))`
|
||||
|
||||
More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](/status/deprecated/account.md#x3dh-prekey-bundles).
|
||||
|
||||
Prekey bundles can be extracted from any user's messages,
|
||||
or found via searching for their specific topic, `{IK}-contact-code`.
|
||||
|
||||
TODO: See below on bundle retrieval, this seems like enhancement and parameter for recommendation
|
||||
|
||||
### Bundle retrieval
|
||||
|
||||
<!-- TODO: Potentially move this completely over to [Trust Establishment](./status-account-spec.md) -->
|
||||
|
||||
X3DH works by having client apps create and make available a bundle of prekeys (the X3DH bundle)
|
||||
that can later be requested by other interlocutors when they wish to start a conversation with a given user.
|
||||
|
||||
In the X3DH specification, nodes typically use a shared server
|
||||
to store bundles and allow other users to download them upon request.
|
||||
Given Status' goal of decentralization,
|
||||
Status chat clients cannot rely on the same type of infrastructure
|
||||
and must achieve the same result using other means.
|
||||
By growing order of convenience and security, the considered approaches are:
|
||||
|
||||
- contact codes;
|
||||
- public and one-to-one chats;
|
||||
- QR codes;
|
||||
- ENS record;
|
||||
- Decentralized permanent storage (e.g. Swarm, IPFS).
|
||||
- Whisper/Waku
|
||||
|
||||
<!-- TODO: Comment, it isn't clear what we actually _do_. It seems as if this is exploring the problem space. From a protocol point of view, it might make sense to describe the interface, and then have a recommendation section later on that specifies what we do. See e.g. Signal's specs where they specify specifics later on. -->
|
||||
|
||||
Currently, only public and one-to-one message exchanges and Whisper/Waku is used to exchange bundles.
|
||||
|
||||
Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys,
|
||||
the approach taken is to rotate more frequently the bundle (once every 24 hours),
|
||||
which will be propagated by the app through the channel available.
|
||||
|
||||
### 1:1 chat contact request
|
||||
|
||||
There are two phases in the initial negotiation of a 1:1 chat:
|
||||
|
||||
1. **Identity verification** (e.g., face-to-face contact exchange through QR code, Identicon matching).
|
||||
A QR code serves two purposes simultaneously - identity verification and initial bundle retrieval;
|
||||
1. **Asynchronous initial key exchange**, using X3DH.
|
||||
|
||||
For more information on account generation and trust establishment, see [2/ACCOUNT](/status/deprecated/account.md)
|
||||
|
||||
#### Initial key exchange flow (X3DH)
|
||||
|
||||
[Section 3 of the X3DH protocol](https://signal.org/docs/specifications/x3dh/#sending-the-initial-message) describes the initial key exchange flow, with some additional context:
|
||||
|
||||
- The users' identity keys `IK_A` and `IK_B` correspond to their respective Status chat public keys;
|
||||
- Since it is not possible to guarantee that a prekey will be used only once in a decentralized world,
|
||||
the one-time prekey `OPK_B` is not used in this scenario;
|
||||
- Nodes do not send Bundles to a centralized server, but instead served in a decentralized way as described in [bundle retrieval](#bundle-retrieval).
|
||||
|
||||
Alice retrieves Bob's prekey bundle, however it is not specific to Alice. It contains:
|
||||
|
||||
([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12))
|
||||
|
||||
``` protobuf
|
||||
// X3DH prekey bundle
|
||||
message Bundle {
|
||||
|
||||
bytes identity = 1;
|
||||
|
||||
map<string,SignedPreKey> signed_pre_keys = 2;
|
||||
|
||||
bytes signature = 4;
|
||||
|
||||
int64 timestamp = 5;
|
||||
}
|
||||
|
||||
```golang
|
||||
- `identity`: Identity key `IK_B`
|
||||
- `signed_pre_keys`: Signed prekey `SPK_B` for each device, indexed by `installation-id`
|
||||
- `signature`: Prekey signature <i>Sig(`IK_B`, Encode(`SPK_B`))</i>
|
||||
- `timestamp`: When the bundle was created locally
|
||||
|
||||
([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L5))
|
||||
|
||||
``` protobuf
|
||||
message SignedPreKey {
|
||||
bytes signed_pre_key = 1;
|
||||
uint32 version = 2;
|
||||
}
|
||||
```
|
||||
|
||||
The `signature` is generated by sorting `installation-id` in lexicographical order, and concatenating the `signed-pre-key` and `version`:
|
||||
|
||||
`installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2`
|
||||
|
||||
#### Double Ratchet
|
||||
|
||||
Having established the initial shared secret `SK` through X3DH, it can be used to seed a Double Ratchet exchange between Alice and Bob.
|
||||
|
||||
Please refer to the [Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/) for more details.
|
||||
|
||||
The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage` ([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L65))
|
||||
containing a map of `DirectMessageProtocol` indexed by `installation-id` ([protobuf](https://github.com/status-im/status-go/blob/1ac9dd974415c3f6dee95145b6644aeadf02f02c/services/shhext/chat/encryption.proto#L56)):
|
||||
|
||||
``` protobuf
|
||||
message ProtocolMessage {
|
||||
|
||||
string installation_id = 2;
|
||||
|
||||
repeated Bundle bundles = 3;
|
||||
|
||||
// One to one message, encrypted, indexed by installation_id
|
||||
map<string,DirectMessageProtocol> direct_message = 101;
|
||||
|
||||
// Public chats, not encrypted
|
||||
bytes public_message = 102;
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
- `bundles`: a sequence of bundles
|
||||
- `installation_id`: the installation id of the sender
|
||||
- `direct_message` is a map of `DirectMessageProtocol` indexed by `installation-id`
|
||||
- `public_message`: unencrypted public chat message.
|
||||
|
||||
``` protobuf
|
||||
message DirectMessageProtocol {
|
||||
X3DHHeader X3DH_header = 1;
|
||||
DRHeader DR_header = 2;
|
||||
DHHeader DH_header = 101;
|
||||
// Encrypted payload
|
||||
bytes payload = 3;
|
||||
}
|
||||
```
|
||||
|
||||
```protobuf
|
||||
- `X3DH_header`: the `X3DHHeader` field in `DirectMessageProtocol` contains:
|
||||
|
||||
([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
|
||||
``` protobuf
|
||||
message X3DHHeader {
|
||||
bytes key = 1;
|
||||
bytes id = 4;
|
||||
}
|
||||
```
|
||||
|
||||
- `key`: Alice's ephemeral key `EK_A`;
|
||||
- `id`: Identifier stating which of Bob's prekeys Alice used, in this case Bob's bundle signed prekey.
|
||||
|
||||
Alice's identity key `IK_A` is sent at the transport layer level (Whisper/Waku);
|
||||
|
||||
- `DR_header`: Double ratchet header ([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)). Used when Bob's public bundle is available:
|
||||
``` protobuf
|
||||
message DRHeader {
|
||||
bytes key = 1;
|
||||
uint32 n = 2;
|
||||
uint32 pn = 3;
|
||||
bytes id = 4;
|
||||
}
|
||||
```
|
||||
- `key`: Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet));
|
||||
- `n`: number of the message in the sending chain;
|
||||
- `pn`: length of the previous sending chain;
|
||||
- `id`: Bob's bundle ID.
|
||||
|
||||
- `DH_header`: Diffie-Helman header (used when Bob's bundle is not available):
|
||||
([protobuf](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
|
||||
``` protobuf
|
||||
message DHHeader {
|
||||
bytes key = 1;
|
||||
}
|
||||
```
|
||||
- `key`: Alice's compressed ephemeral public key.
|
||||
|
||||
- `payload`:
|
||||
- if a bundle is available, contains payload encrypted with the Double Ratchet algorithm;
|
||||
- otherwise, payload encrypted with output key of DH exchange (no Perfect Forward Secrecy).
|
||||
```
|
||||
<!-- TODO: A lot of links to status-go, seems likely these should be updated to status-protocol-go -->
|
||||
|
||||
## Security Considerations
|
||||
|
||||
The same considerations apply as in [section 4 of the X3DH spec](https://signal.org/docs/specifications/x3dh/#security-considerations) and [section 6 of the Double Ratchet spec](https://signal.org/docs/specifications/doubleratchet/#security-considerations), with some additions detailed below.
|
||||
|
||||
<!-- TODO: Add any additional context here not covered in the X3DH and DR specs -->
|
||||
|
||||
<!--
|
||||
TODO: description here
|
||||
|
||||
### --- Security and Privacy Features
|
||||
#### Confidentiality (YES)
|
||||
> Only the intended recipients are able to read a message. Specifically, the message must not be readable by a server operator that is not a conversation participant
|
||||
|
||||
- Yes.
|
||||
- There's a layer of encryption at Whisper as well as above with Double Ratchet
|
||||
- Relay nodes and Mailservers can only read a topic of a Whisper message, and nothing within the payload.
|
||||
|
||||
#### Integrity (YES)
|
||||
> No honest party will accept a message that has been modified in transit.
|
||||
|
||||
- Yes.
|
||||
- Assuming a user validates (TODO: Check this assumption) every message they are able to decrypt and validate its signature from the sender, then it is not able to be altered in transit.
|
||||
* [igorm] i'm really not sure about it, Whisper provides a signature, but I'm not sure we check it anywhere (simple grepping didn't give anything)
|
||||
* [andrea] Whisper checks the signature and a public key is derived from it, we check the public key is a meaningful public key. The pk itself is not in the content of the message for public chats/1-to-1 so potentially you could send a message from a random account without having access to the private key, but that would not be much of a deal, as you might just as easily create a random account)
|
||||
|
||||
#### Authentication (YES)
|
||||
> Each participant in the conversation receives proof of possession of a known long-term secret from all other participants that they believe to be participating in the conversation. In addition, each participant is able to verify that a message was sent from the claimed source
|
||||
|
||||
- 1:1 --- one-to-one messages are encrypted with the recipient's public key, and digitally signed by the sender's. In order to provide Perfect Forward Secrecy, we build on the X3DH and Double Ratchet specifications from Open Whisper Systems, with some adaptations to operate in a decentralized environment.
|
||||
- group --- group chat is pairwise
|
||||
- public --- A user subscribes to a public channel topic and the decryption key is derived from the topic name
|
||||
|
||||
**TODO:** Need to verify that this is actually the case
|
||||
**TODO:** Fill in explicit details here
|
||||
|
||||
#### Participant Consistency (YES?)
|
||||
> At any point when a message is accepted by an honest party, all honest parties are guaranteed to have the same view of the participant list
|
||||
|
||||
- **TODO:** Need details here
|
||||
|
||||
#### Destination Validation (YES?)
|
||||
> When a message is accepted by an honest party, they can verify that they were included in the set of intended recipients for the message.
|
||||
|
||||
- Users are aware of the topic that a message was sent to, and that they have the ability to decrypt it.
|
||||
-
|
||||
|
||||
#### Forward Secrecy (PARTIAL)
|
||||
> Compromising all key material does not enable decryption of previously encrypted data
|
||||
|
||||
- After first back and forth between two contacts with PFS enabled, yes.
|
||||
|
||||
#### Backward Secrecy (YES)
|
||||
> Compromising all key material does not enable decryption of succeeding encrypted data
|
||||
|
||||
- PFS requires both backward and forwards secrecy
|
||||
[Andrea: This is not true, (Perfect) Forward Secrecy does not imply backward secrecy (which is also called post-compromise security, as signal calls it, or future secrecy, it's not well defined). Technically this is a NO , double ratchet offers good Backward secrecy, but not perfect. Effectively if all the key material is compromised, any future message received will be also compromised (due to the hash ratchet), until a DH ratchet step is completed (i.e. the compromised party generate a new random key and ratchet)]
|
||||
|
||||
#### Anonymity Preserving (PARTIAL)
|
||||
> Any anonymity features provided by the underlying transport privacy architecture are not undermined (e.g., if the transport privacy system provides anonymity, the conversation security level does not de-anonymize users by linking key identifiers).
|
||||
|
||||
- by default, yes
|
||||
- ENS Naming system attaches an identifier to a given public key
|
||||
|
||||
#### Speaker Consistency (PARTIAL)
|
||||
> All participants agree on the sequence of messages sent by each participant. A protocol might perform consistency checks on blocks of messages during the protocol, or after every message is sent.
|
||||
|
||||
- We use Lamport timestamps for ordering of events.
|
||||
- In addition to this, we use local timestamps to attempt a more intuitive ordering. [Andrea: currently this was introduced as a regression during performance optimization and might result in out-of-order messages if sent across day boundaries, so I consider it a bug and not part of the specs (it does not make the order more intuitive, quite the opposite as it might result in causally related messages being out-of-order, but helps dividing the messages in days)]
|
||||
- Fundamentally, there's no single source of truth, nor consensus process for global ordering [Andrea: Global ordering does not need a consensus process i.e. if you order messages alphabetically, and you break ties consistently, you have global ordering, as all the participants will see the same ordering (as opposed to say order by the time the message was received locally), of course is not useful, you want to have causal + global to be meaningful]
|
||||
|
||||
TODO: Understand how this is different from Global Transcript
|
||||
[Andrea: This is basically Global transcript for a single participants, we offer global transcript]
|
||||
|
||||
#### Causality Preserving (PARTIAL)
|
||||
> Implementations can avoid displaying a message before messages that causally precede it
|
||||
|
||||
- Not yet, but in pipeline (data sync layer)
|
||||
|
||||
[Andrea: Messages are already causally ordered, we don't display messages that are causally related out-of-order, that's already granted by lamport timestamps]
|
||||
|
||||
TODO: Verify if this can be done already by looking at Lamport clock difference
|
||||
|
||||
#### Global Transcript (PARTIAL)
|
||||
> All participants see all messages in the same order
|
||||
|
||||
- See directly above
|
||||
|
||||
[Andrea: messages are globally (total) ordered, so all participants see the same ordering]
|
||||
|
||||
#### Message Unlinkability (NO)
|
||||
> If a judge is convinced that a participant authored one message in the conversation, this does not provide evidence that they authored other messages
|
||||
|
||||
- Currently, the Status software signs every messages sent with the user's public key, thus making it unable to provide unlinkability.
|
||||
- This is not necessary though, and could be built in to have an option to not sign.
|
||||
- Side note: moot account allows for this but is a function of the anonymity set that uses it. The more people that use this account the stronger the unlinkability.
|
||||
|
||||
#### Message Repudiation (NO)
|
||||
> Given a conversation transcript and all cryptographic keys, there is no evidence that a given message was authored by any particular user
|
||||
|
||||
- All messages are digitally signed by their sender.
|
||||
- The underlying transport, Whisper/Waku, does allow for unsigned messages, but we don't use it.
|
||||
|
||||
#### Participant Repudiation (NO)
|
||||
> Given a conversation transcript and all cryptographic key material for all but one accused (honest) participant, there is no evidence that the honest participant was in a conversation with any of the other participants.
|
||||
|
||||
### --- Group related features
|
||||
#### Computational Equality (YES)
|
||||
> All chat participants share an equal computational load
|
||||
|
||||
- One a message is sent, all participants in a group chat perform the same steps to retrieve and decrypt it.
|
||||
- If proof of work is actually used at the Whisper layer (basically turned off in Status) then the sender would have to do additional computational work to send messages.
|
||||
|
||||
#### Trust Equality (PARTIAL)
|
||||
> No participant is more trusted or takes on more responsibility than any other
|
||||
|
||||
- 1:1 chats and public chats are equal
|
||||
- group chats have admins (on purpose)
|
||||
- Private Group chats have Administrators and Members. Upon construction, the creator is made an admin. These groups have the following privileges:
|
||||
- Admins:
|
||||
- Add group members
|
||||
- Promote group members to admin
|
||||
- Change group name
|
||||
- Members:
|
||||
- Accept invitation to group
|
||||
- Leave group
|
||||
- Non-Members:
|
||||
- Invited by admins show up as "invited" in group; this leaks contact information
|
||||
- Invited people don't opt-in to being invited
|
||||
|
||||
TODO: Group chat dynamics should have a documented state diagram
|
||||
TODO: create issues for identity leak of invited members as well as current members of a group showing up who have not accepted yet [Andrea: that's an interesting point, didn't think of that. Currently we have this behavior for 2 reasons, backward compatibility with previous releases, which had no concept of joining, and also because we rely on other peers to propagate group info, so we don't have a single-message point of failure (the invitation), the first can be addressed easily, the second is trickier, without giving up the propagation mechanism (if we choose to give this up, then it's trivial)]
|
||||
|
||||
#### Subgroup Messaging (NO)
|
||||
> Messages can be sent to a subset of participants without forming a new conversation
|
||||
|
||||
- This would require a new topic and either a new public chat or a new group chat
|
||||
[Andrea: This is a YES, as messages are pairwise encrypted, and client-side fanout, so anyone could potentially send a message only to a subset of the group]
|
||||
|
||||
#### Contractible Membership (PARTIAL)
|
||||
> After the conversation begins, participants can leave without restarting the protocol
|
||||
|
||||
- For 1:1, there is no way to ignore or block a user from sending you a message. This is currently in the pipeline.
|
||||
- For public chats, Yes. A member simply stops subscribing to a specific topic and will no longer receive messages.
|
||||
- For group chats: this assumes pairwise encryption OR key is renegotiated
|
||||
- This only currently works on the identity level, and not the device level. A ghost device will have access to anything other devices have.
|
||||
[Andrea: For group chats, that's possible as using pairwise encryption, also with group chats (which use device-to-device encryption), ghost devices is a bit more complicated, in general, they don't have access to the messages you send, i.e. If I send a message from device A1 to the group chat and there is a ghost device A2, it will not be able to decrypt the content, but will see that a message has been sent (as only paired devices are kept in sync, and those are explicitly approved by the user). Messages that you receive are different, so a ghost device (A2) will potentially be able to decrypt the message, but A1 can detect the ghost device (in most cases, it's complicated :), the pfs docs describe multi-device support), for one-to-one ghost devices are undetectable]
|
||||
|
||||
#### Expandable Membership (PARTIAL)
|
||||
> After the conversation begins, participants can join without restarting the protocol.
|
||||
|
||||
- 1:1: no, only 1:1
|
||||
- private group: yes, since it is pair-wise, each person in the group just creates a pair with the new member
|
||||
- public: yes, as members of a public chat are only subscribing to a topic and receiving anyone sending messages to it.
|
||||
|
||||
### --- Usability and Adoption
|
||||
|
||||
#### Out-of-Order Resilient (PARTIAL)
|
||||
> If a message is delayed in transit, but eventually arrives, its contents are accessible upon arrival
|
||||
|
||||
- Due to asynchronous forward secrecy and no additional services, private keys might be rotated
|
||||
|
||||
[Andrea: That's correct, in some cases if the message is delayed for too long, or really out-of-order, the specific message key might have been deleted, as we only keep the last 3000 message keys]
|
||||
[Igor: TTL of a Whisper message can expire, so any node-in-transit will drop it. Also, I believe we ignore messages with skewed timestamps]
|
||||
|
||||
#### Dropped Message Resilient (PARTIAL)
|
||||
> Messages can be decrypted without receipt of all previous messages. This is desirable for asynchronous and unreliable network services
|
||||
|
||||
- Public chats: yes, users are able to decrypt any message received at any time.
|
||||
- 1-to-1/group chat also, this is a YES in my opinion
|
||||
|
||||
#### Asynchronous (PARTIAL)
|
||||
> Messages can be sent securely to disconnected recipients and received upon their next connection
|
||||
|
||||
- The semantics around message reliability are currently poor
|
||||
* [Igor: messages are stored on mailservers for way longer than TTL (30 days), but that requires Status infrastructure]
|
||||
- There's a TTL in Whisper and mailserver can deliver messages after the fact
|
||||
|
||||
TODO: this requires more detail
|
||||
|
||||
#### Multi-Device Support (YES)
|
||||
> A user can participate in the conversation using multiple devices at once. Each device must be able to send and receive messages. Ideally, all devices have identical views of the conversation. The devices might use a synchronized long-term key or distinct keys.
|
||||
|
||||
- Yes
|
||||
- There is currently work being done to improve the syncing process between a user's devices.
|
||||
|
||||
#### No Additional Service (NO)
|
||||
> The protocol does not require any infrastructure other than the protocol participants. Specifically, the protocol must not require additional servers for relaying messages or storing any kind of key material.
|
||||
|
||||
- The protocol requires Whisper/Waku relay servers and mailservers currently.
|
||||
- The larger the number of Whisper/Waku relay servers, the better the transport security but there might be potential scaling problems.
|
||||
- Mailservers act to provide asynchronicity so users can retrieve messages after coming back from an offline period.
|
||||
|
||||
-->
|
||||
|
||||
## Session management
|
||||
|
||||
A node identifies a peer by two pieces of data:
|
||||
|
||||
1) An `installation-id` which is generated upon creating a new account in the `Status` application
|
||||
2) Their identity Whisper/Waku key
|
||||
|
||||
### Initialization
|
||||
|
||||
A node initializes a new session once a successful X3DH exchange has taken place. Subsequent messages will use the established session until re-keying is necessary.
|
||||
|
||||
### Concurrent sessions
|
||||
|
||||
If a node creates two sessions concurrently between two peers, the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
|
||||
|
||||
### Re-keying
|
||||
|
||||
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
|
||||
|
||||
### Multi-device support
|
||||
|
||||
Multi-device support is quite challenging as there is not a central place
|
||||
where information on which and how many devices (identified by their respective `installation-id`) belongs to a whisper-identity / waku-identity.
|
||||
|
||||
Furthermore, account recovery always needs to be taken into consideration,
|
||||
where a user wipes clean the whole device and the nodes loses all the information about any previous sessions.
|
||||
|
||||
Taking these considerations into account, the way the network propagates multi-device information using x3dh bundles,
|
||||
which will contain information about paired devices as well as information about the sending device.
|
||||
|
||||
This means that every time a new device is paired, the bundle needs to be updated and propagated with the new information,
|
||||
the user has the responsibility to make sure the pairing is successful.
|
||||
|
||||
The method is loosely based on [Sesame](https://signal.org/docs/specifications/sesame/).
|
||||
|
||||
### Pairing
|
||||
|
||||
When a user adds a new account in the `Status` application, a new `installation-id` will be generated.
|
||||
The device should be paired as soon as possible if other devices are present.
|
||||
Once paired the contacts will be notified of the new device and it will be included in further communications.
|
||||
|
||||
If a bundle received from the `IK` is different to the `installation-id`,
|
||||
the device will be shown to the user and will have to be manually approved, to a maximum of 3.
|
||||
Once that is done any message sent by one device will also be sent to any other enabled device.
|
||||
|
||||
Once a user enables a new device, a new bundle will be generated which will include pairing information.
|
||||
|
||||
The bundle will be propagated to contacts through the usual channels.
|
||||
|
||||
Removal of paired devices is a manual step that needs to be applied on each device,
|
||||
and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
|
||||
|
||||
### Sending messages to a paired group
|
||||
|
||||
When sending a message, the peer will send a message to other `installation-id` that they have seen.
|
||||
The node caps the number of devices to 3, ordered by last activity.
|
||||
The node sends messages using pairwise encryption, including their own devices.
|
||||
|
||||
Account recovery
|
||||
|
||||
Account recovery is no different from adding a new device, and it is handled in exactly the same way.
|
||||
|
||||
### Partitioned devices
|
||||
|
||||
In some cases (i.e. account recovery when no other pairing device is available, device not paired),
|
||||
it is possible that a device will receive a message that is not targeted to its own `installation-id`.
|
||||
In this case an empty message containing bundle information is sent back,
|
||||
which will notify the receiving end of including this device in any further communication.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
- Added language to include Waku in all relevant places
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [X3DH](https://signal.org/docs/specifications/x3dh/)
|
||||
- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
|
||||
- [Protobuf](https://developers.google.com/protocol-buffers/)
|
||||
- [Whisper](/status/deprecated/whisper-usage.md)
|
||||
- [Waku](/status/deprecated/waku-usage.md)
|
||||
- [Account specification](/status/deprecated/account.md)
|
||||
- [Status implementation](https://github.com/status-im/doubleratchet/)
|
||||
- [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html)
|
||||
- [X3DH](https://signal.org/docs/specifications/x3dh/)
|
||||
- [ACCOUNT](/status/deprecated/account.md)
|
||||
- [Sesame](https://signal.org/docs/specifications/sesame/)
|
||||
- [May 22, 2020 commit change](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
138
status/deprecated/waku-mailserver.md
Normal file
138
status/deprecated/waku-mailserver.md
Normal file
@@ -0,0 +1,138 @@
|
||||
---
|
||||
title: WAKU-MAILSERVER
|
||||
name: Waku Mailserver
|
||||
status: deprecated
|
||||
description: Waku Mailserver is a specification that allows messages to be stored permanently and to allow the stored messages to be delivered to requesting client nodes, regardless if the messages are not available in the network due to the message TTL expiring.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Samuel Hawksby-Robinson <samuel@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
Being mostly offline is an intrinsic property of mobile clients.
|
||||
They need to save network transfer and battery consumption to avoid spending too much money or constant charging.
|
||||
Waku protocol, on the other hand, is an online protocol.
|
||||
Messages are available in the Waku network only for short period of time calculate in seconds.
|
||||
|
||||
Waku Mailserver is a specification that allows messages to be stored permanently
|
||||
and allows the stored messages to be delivered to requesting client nodes,
|
||||
regardless if the messages are not available in the network due to the message TTL expiring.
|
||||
|
||||
## `Mailserver`
|
||||
|
||||
From the network perspective, a `Mailserver` is just like any other Waku node.
|
||||
The only difference is that a `Mailserver` has the capability of archiving messages
|
||||
and delivering them to its peers on-demand.
|
||||
|
||||
It is important to notice that a `Mailserver` will only handle requests from its direct peers
|
||||
and exchanged packets between a `Mailserver` and a peer are p2p messages.
|
||||
|
||||
### Archiving messages
|
||||
|
||||
A node which wants to provide `Mailserver` functionality MUST store envelopes from
|
||||
incoming message packets (Waku packet-code `0x01`). The envelopes can be stored in any
|
||||
format, however they MUST be serialized and deserialized to the Waku envelope format.
|
||||
|
||||
A `Mailserver` SHOULD store envelopes for all topics to be generally useful for any peer,
|
||||
however for specific use cases it MAY store envelopes for a subset of topics.
|
||||
|
||||
### Requesting messages
|
||||
|
||||
In order to request historic messages, a node MUST send a packet P2P Request (`0x7e`) to a peer providing `Mailserver` functionality.
|
||||
This packet requires one argument which MUST be a Waku envelope.
|
||||
|
||||
In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
|
||||
|
||||
```golang
|
||||
[ Lower, Upper, Bloom, Limit, Cursor ]
|
||||
```
|
||||
|
||||
`Lower`: 4-byte wide unsigned integer (UNIX time in seconds; oldest requested envelope's creation time)
|
||||
`Upper`: 4-byte wide unsigned integer (UNIX time in seconds; newest requested envelope's creation time)
|
||||
`Bloom`: 64-byte wide array of Waku topics encoded in a bloom filter to filter envelopes
|
||||
`Limit`: 4-byte wide unsigned integer limiting the number of returned envelopes
|
||||
`Cursor`: an array of a cursor returned from the previous request (optional)
|
||||
|
||||
The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit`
|
||||
so that the requester can send another request using the obtained `Cursor` value.
|
||||
What exactly is in the `Cursor` is up to the implementation.
|
||||
The requester SHOULD NOT use a `Cursor` obtained from one `Mailserver` in a request to another `Mailserver` because the format or the result MAY be different.
|
||||
|
||||
The envelope MUST be encrypted with a symmetric key agreed between the requester and the `Mailserver`.
|
||||
|
||||
### Receiving historic messages
|
||||
|
||||
Historic messages MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes.
|
||||
|
||||
In order to receive historic messages from a `Mailserver`, a node MUST trust the selected `Mailserver`,
|
||||
that is allowed to send packets with the P2P Message code. By default, the node discards such packets.
|
||||
|
||||
Received envelopes MUST be passed through the Waku envelope pipelines
|
||||
so that they are picked up by registered filters and passed to subscribers.
|
||||
|
||||
For a requester, to know that all messages have been sent by a `Mailserver`,
|
||||
it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by the following parameters:
|
||||
|
||||
```golang
|
||||
[ RequestID, LastEnvelopeHash, Cursor ]
|
||||
```
|
||||
|
||||
* `RequestID`: 32-byte wide array with a Keccak-256 hash of the envelope containing the original request
|
||||
* `LastEnvelopeHash`: 32-byte wide array with a Keccak-256 hash of the last sent envelope for the request
|
||||
* `Cursor`: an array of a cursor returned from the previous request (optional)
|
||||
|
||||
If `Cursor` is not empty, it means that not all messages were sent due to the set `Limit` in the request.
|
||||
One or more consecutive requests MAY be sent with `Cursor` field filled in order to receive the rest of the messages.
|
||||
|
||||
## Security considerations
|
||||
|
||||
### Confidentiality
|
||||
|
||||
The node encrypts all Waku envelopes. A `Mailserver` node can not inspect their contents.
|
||||
|
||||
### Altruistic and centralized operator risk
|
||||
|
||||
In order to be useful, a `Mailserver` SHOULD be online most of time.
|
||||
That means users either have to be a bit tech-savvy to run their own node,
|
||||
or rely on someone else to run it for them.
|
||||
|
||||
Currently, one of Status's legal entities provides `Mailservers` in an altruistic manner,
|
||||
but this is suboptimal from a decentralization, continuance and risk point of view.
|
||||
Coming up with a better system for this is ongoing research.
|
||||
|
||||
A Status client SHOULD allow the `Mailserver` selection to be customizable.
|
||||
|
||||
### Privacy concerns
|
||||
|
||||
In order to use a `Mailserver`, a given node needs to connect to it directly,
|
||||
i.e. add the `Mailserver` as its peer and mark it as trusted.
|
||||
This means that the `Mailserver` is able to send direct p2p messages to the node instead of broadcasting them.
|
||||
Effectively, it will have access to the bloom filter of topics that the user is interested in,
|
||||
when it is online as well as many metadata like IP address.
|
||||
|
||||
### Denial-of-service
|
||||
|
||||
Since a `Mailserver` is delivering expired envelopes and has a direct TCP connection with the recipient,
|
||||
the recipient is vulnerable to DoS attacks from a malicious `Mailserver` node.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.1
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
* Created document
|
||||
* Forked from [whisper-mailserver](/status/deprecated/whisper-mailserver.md)
|
||||
* Change to keep `Mailserver` term consistent
|
||||
* Replaced Whisper references with Waku
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
* [May 22, 2020 change commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
392
status/deprecated/waku-usage.md
Normal file
392
status/deprecated/waku-usage.md
Normal file
@@ -0,0 +1,392 @@
|
||||
---
|
||||
title: WAKU-USAGE
|
||||
name: Waku Usage
|
||||
status: deprecated
|
||||
description: Status uses Waku to provide privacy-preserving routing and messaging on top of devP2P.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Samuel Hawksby-Robinson <samuel@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
Status uses [Waku](/waku/standards/legacy/6/waku1.md) to provide privacy-preserving routing
|
||||
and messaging on top of devP2P.
|
||||
Waku uses topics to partition its messages,
|
||||
and these are leveraged for all chat capabilities.
|
||||
In the case of public chats, the channel name maps directly to its Waku topic.
|
||||
This allows anyone to listen on a single channel.
|
||||
|
||||
Additionally, since anyone can receive Waku envelopes,
|
||||
it relies on the ability to decrypt messages to decide who is the correct recipient.
|
||||
Status nodes do not rely upon this property,
|
||||
and implement another secure transport layer on top of Whisper.
|
||||
|
||||
## Reason
|
||||
|
||||
Provide routing, metadata protection, topic-based multicasting and basic
|
||||
encryption properties to support asynchronous chat.
|
||||
|
||||
## Terminology
|
||||
|
||||
* *Waku node*: an Ethereum node with Waku V1 enabled
|
||||
* *Waku network*: a group of Waku nodes connected together through the internet connection and forming a graph
|
||||
* *Message*: a decrypted Waku message
|
||||
* *Offline message*: an archived envelope
|
||||
* *Envelope*: an encrypted message with metadata like topic and Time-To-Live
|
||||
|
||||
## Waku packets
|
||||
|
||||
| Packet Name | Code | References |
|
||||
| -------------------- | ---: | --- |
|
||||
| Status | 0 | [Status](status), [WAKU-1](/waku/standards/legacy/6/waku1.md#status) |
|
||||
| Messages | 1 | [WAKU-1](/waku/standards/legacy/6/waku1.md#messages) |
|
||||
| Batch Ack | 11 | Undocumented. Marked for Deprecation |
|
||||
| Message Response | 12 | [WAKU-1](/waku/standards/legacy/6/waku1.md#batch-ack-and-message-response) |
|
||||
| Status Update | 22 | [WAKU-1](/waku/standards/legacy/6/waku1.md#status-update) |
|
||||
| P2P Request Complete | 125 | [4/WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md) |
|
||||
| P2P Request | 126 | [4/WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md), [WAKU-1](/waku/standards/legacy/6/waku1.md#p2p-request) |
|
||||
| P2P Messages | 127 | [4/WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md), [WAKU-1](/waku/standards/legacy/6/waku1.md#p2p-request-complete) |
|
||||
|
||||
## Waku node configuration
|
||||
|
||||
A Waku node must be properly configured to receive messages from Status clients.
|
||||
|
||||
Nodes use Waku's Proof Of Work algorithm to deter denial of service and various spam/flood attacks against the Whisper network.
|
||||
The sender of a message must perform some work which in this case means processing time.
|
||||
Because Status' main client is a mobile client, this easily leads to battery draining and poor performance of the app itself.
|
||||
Hence, all clients MUST use the following Whisper node settings:
|
||||
|
||||
* proof-of-work requirement not larger than `0.002` for payloads less than 50,000 bytes
|
||||
* proof-of-work requirement not larger than `0.000002` for payloads greater than or equal to 50,000 bytes
|
||||
* time-to-live not lower than `10` (in seconds)
|
||||
|
||||
## Status
|
||||
|
||||
Handshake is a RLP-encoded packet sent to a newly connected peer. It MUST start with a Status Code (`0x00`) and follow up with items:
|
||||
|
||||
```golang
|
||||
[
|
||||
[ pow-requirement-key pow-requirement ]
|
||||
[ bloom-filter-key bloom-filter ]
|
||||
[ light-node-key light-node ]
|
||||
[ confirmations-enabled-key confirmations-enabled ]
|
||||
[ rate-limits-key rate-limits ]
|
||||
[ topic-interest-key topic-interest ]
|
||||
]
|
||||
```
|
||||
|
||||
| Option Name | Key | Type | Description | References |
|
||||
| ----------------------- | ------ | -------- | ----------- | --- |
|
||||
| `pow-requirement` | `0x00` | `uint64` | minimum PoW accepted by the peer | [WAKU-1#pow-requirement](/waku/standards/legacy/6/waku1.md#pow-requirement-field) |
|
||||
| `bloom-filter` | `0x01` | `[]byte` | bloom filter of Waku topic accepted by the peer | [WAKU-1#bloom-filter](/waku/standards/legacy/6/waku1.md#bloom-filter-field) |
|
||||
| `light-node` | `0x02` | `bool` | when true, the peer won't forward envelopes through the Messages packet. | [WAKU-1#light-node](/waku/standards/legacy/6/waku1.md#light-node) |
|
||||
| `confirmations-enabled` | `0x03` | `bool` | when true, the peer will send message confirmations | [WAKU-1#confirmations-enabled-field](/waku/standards/legacy/6/waku1.md#confirmations-enabled-field) |
|
||||
| `rate-limits` | `0x04` | | See [Rate limiting](/waku/standards/legacy/6/waku1.md#rate-limits-field) | [WAKU-1#rate-limits](/waku/standards/legacy/6/waku1.md#rate-limits-field) |
|
||||
| `topic-interest` | `0x05` | `[10000][4]byte` | Topic interest is used to share a node's interest in envelopes with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics. | [WAKU-1#topic-interest](/waku/standards/legacy/6/waku1.md#topic-interest-field), [the theoretical scaling model](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability) |
|
||||
|
||||
<!-- TODO Add `light-node` and `confirmations-enabled` links when https://github.com/vacp2p/specs/pull/128 is merged -->
|
||||
|
||||
## Rate limiting
|
||||
|
||||
In order to provide an optional very basic Denial-of-Service attack protection, each node SHOULD define its own rate limits.
|
||||
The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
|
||||
|
||||
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
|
||||
|
||||
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
|
||||
|
||||
Each node SHOULD broadcast its rate limits to its peers using `rate limits` in `status-options` via packet code `0x00` or `0x22`. The rate limits is RLP-encoded information:
|
||||
|
||||
```golang
|
||||
[ IP limits, PeerID limits, Topic limits ]
|
||||
```
|
||||
|
||||
`IP limits`: 4-byte wide unsigned integer
|
||||
`PeerID limits`: 4-byte wide unsigned integer
|
||||
`Topic limits`: 4-byte wide unsigned integer
|
||||
|
||||
The rate limits MAY also be sent as an optional parameter in the handshake.
|
||||
|
||||
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits.
|
||||
If the limit gets exceeded, the connection MAY be dropped by the peer.
|
||||
|
||||
## Keys management
|
||||
|
||||
The protocol requires a key (symmetric or asymmetric) for the following actions:
|
||||
|
||||
* signing & verifying messages (asymmetric key)
|
||||
* encrypting & decrypting messages (asymmetric or symmetric key).
|
||||
|
||||
As nodes require asymmetric keys and symmetric keys to process incoming messages,
|
||||
they must be available all the time and are stored in memory.
|
||||
|
||||
Keys management for PFS is described in [5/SECURE-TRANSPORT](/status/deprecated/secure-transport.md).
|
||||
|
||||
The Status protocols uses a few particular Waku topics to achieve its goals.
|
||||
|
||||
### Contact code topic
|
||||
|
||||
Nodes use the contact code topic to facilitate the discovery of X3DH bundles so that the first message can be PFS-encrypted.
|
||||
|
||||
Each user publishes periodically to this topic. If user A wants to contact user B, she SHOULD look for their bundle on this contact code topic.
|
||||
|
||||
Contact code topic MUST be created following the algorithm below:
|
||||
|
||||
```golang
|
||||
contactCode := "0x" + hexEncode(activePublicKey) + "-contact-code"
|
||||
|
||||
var hash []byte = keccak256(contactCode)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
### Partitioned topic
|
||||
|
||||
Waku is broadcast-based protocol. In theory, everyone could communicate using a single topic but that would be extremely inefficient.
|
||||
Opposite would be using a unique topic for each conversation, however,
|
||||
this brings privacy concerns because it would be much easier to detect whether and when two parties have an active conversation.
|
||||
|
||||
Nodes use partitioned topics to broadcast private messages efficiently.
|
||||
By selecting a number of topic, it is possible to balance efficiency and privacy.
|
||||
|
||||
Currently, nodes set the number of partitioned topics to `5000`. They MUST be generated following the algorithm below:
|
||||
|
||||
```golang
|
||||
var partitionsNum *big.Int = big.NewInt(5000)
|
||||
var partition *big.Int = big.NewInt(0).Mod(publicKey.X, partitionsNum)
|
||||
|
||||
partitionTopic := "contact-discovery-" + strconv.FormatInt(partition.Int64(), 10)
|
||||
|
||||
var hash []byte = keccak256(partitionTopic)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
### Public chats
|
||||
|
||||
A public chat MUST use a topic derived from a public chat name following the algorithm below:
|
||||
|
||||
```golang
|
||||
var hash []byte
|
||||
hash = keccak256(name)
|
||||
|
||||
topicLen = 4
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
<!-- NOTE: commented out as it is currently not used. In code for potential future use. - C.P. Oct 8, 2019
|
||||
### Personal discovery topic
|
||||
|
||||
Personal discovery topic is used to ???
|
||||
|
||||
A client MUST implement it following the algorithm below:
|
||||
```golang
|
||||
personalDiscoveryTopic := "contact-discovery-" + hexEncode(publicKey)
|
||||
|
||||
var hash []byte = keccak256(personalDiscoveryTopic)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
Each Status Client SHOULD listen to this topic in order to receive ??? -->
|
||||
|
||||
### Group chat topic
|
||||
|
||||
Group chats does not have a dedicated topic.
|
||||
All group chat messages (including membership updates) are sent as one-to-one messages to multiple recipients.
|
||||
|
||||
### Negotiated topic
|
||||
|
||||
When a client sends a one to one message to another client, it MUST listen to their negotiated topic.
|
||||
This is computed by generating a diffie-hellman key exchange between two members
|
||||
and taking the first four bytes of the `SHA3-256` of the key generated.
|
||||
|
||||
```golang
|
||||
|
||||
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
|
||||
ecies.ImportECDSAPublic(theirPublicKey),
|
||||
16,
|
||||
16,
|
||||
)
|
||||
|
||||
|
||||
hexEncodedKey := hex.EncodeToString(sharedKey)
|
||||
|
||||
var hash []byte = keccak256(hexEncodedKey)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
A client SHOULD send to the negotiated topic only if it has received a message from all the devices included in the conversation.
|
||||
|
||||
### Flow
|
||||
|
||||
To exchange messages with client `B`, a client `A` SHOULD:
|
||||
|
||||
* Listen to client's `B` Contact Code Topic to retrieve their bundle information, including a list of active devices
|
||||
* Send a message on client's `B` partitioned topic
|
||||
* Listen to the Negotiated Topic between `A` & `B`
|
||||
* Once client `A` receives a message from `B`, the Negotiated Topic SHOULD be used
|
||||
|
||||
## Message encryption
|
||||
|
||||
Even though, the protocol specifies an encryption layer that encrypts messages before passing them to the transport layer,
|
||||
Waku protocol requires each Waku message to be encrypted anyway.
|
||||
|
||||
The node encrypts public and group messages using symmetric encryption, and creates the key from a channel name string.
|
||||
The implementation is available in [`shh_generateSymKeyFromPassword`](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API#shh_generatesymkeyfrompassword) JSON-RPC method of go-ethereum Whisper implementation.
|
||||
|
||||
The node encrypts one-to-one messages using asymmetric encryption.
|
||||
|
||||
## Message confirmations
|
||||
|
||||
Sending a message is a complex process where many things can go wrong.
|
||||
Message confirmations tell a node that a message originating from it has been seen by its direct peers.
|
||||
|
||||
A node MAY send a message confirmation for any batch of messages received in a packet Messages Code (`0x01`).
|
||||
|
||||
A node sends a message confirmation using Batch Acknowledge packet (`0x0b`) or Message Response packet (`0x0c`).
|
||||
|
||||
The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data (raw bytes).
|
||||
|
||||
The Message Response packet is more complex and is followed by a Versioned Message Response:
|
||||
|
||||
```golang
|
||||
[ Version, Response]
|
||||
```
|
||||
|
||||
`Version`: a version of the Message Response, equal to `1`,
|
||||
`Response`: `[ Hash, Errors ]` where `Hash` is a keccak256 hash of the envelopes batch data (raw bytes)
|
||||
for which the confirmation is sent and `Errors` is a list of envelope errors when processing the batch.
|
||||
A single error contains `[ Hash, Code, Description ]` where `Hash` is a hash of the processed envelope,
|
||||
`Code` is an error code and `Description` is a descriptive error message.
|
||||
|
||||
The supported codes:
|
||||
`1`: means time sync error which happens when an envelope is too old
|
||||
or created in the future (the root cause is no time sync between nodes).
|
||||
|
||||
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message,
|
||||
one or more peers broadcast a corresponding confirmation. To limit that, both Batch Acknowledge packet (`0x0b`)
|
||||
and Message Response packet (`0x0c`) are not broadcast to peers of the peers, i.e. they do not follow epidemic spread.
|
||||
|
||||
In the current Status network setup, only `Mailservers` support message confirmations.
|
||||
A client posting a message to the network and after receiving a confirmation can be sure that the message got processed by the `Mailserver`.
|
||||
If additionally, sending a message is limited to non-`Mailserver` peers,
|
||||
it also guarantees that the message got broadcast through the network and it reached the selected `Mailserver`.
|
||||
|
||||
## Waku V1 extensions
|
||||
|
||||
### Request historic messages
|
||||
|
||||
Sends a request for historic messages to a `Mailserver`.
|
||||
The `Mailserver` node MUST be a direct peer and MUST be marked as trusted (using `waku_markTrustedPeer`).
|
||||
|
||||
The request does not wait for the response.
|
||||
It merely sends a peer-to-peer message to the `Mailserver` and it's up to `Mailserver` to process it and start sending historic messages.
|
||||
|
||||
The drawback of this approach is that it is impossible to tell which historic messages are the result of which request.
|
||||
|
||||
It's recommended to return messages from newest to oldest.
|
||||
To move further back in time, use `cursor` and `limit`.
|
||||
|
||||
#### wakuext_requestMessages
|
||||
|
||||
**Parameters**:
|
||||
|
||||
* Object - The message request object:
|
||||
* `mailServerPeer` - `String`: `Mailserver`'s enode address.
|
||||
* `from` - `Number` (optional): Lower bound of time range as unix timestamp, default is 24 hours back from now.
|
||||
* `to` - `Number` (optional): Upper bound of time range as unix timestamp, default is now.
|
||||
* `limit` - `Number` (optional): Limit the number of messages sent back, default is no limit.
|
||||
* `cursor` - `String` (optional): Used for paginated requests.
|
||||
* `topics` - `Array`: hex-encoded message topics.
|
||||
* `symKeyID` - `String`: an ID of a symmetric key used to authenticate with the `Mailserver`, derived from the `Mailserver` password.
|
||||
|
||||
**Returns**:
|
||||
`Boolean` - returns `true` if the request was sent.
|
||||
|
||||
The above `topics` is then converted into a bloom filter and then and sent to the `Mailserver`.
|
||||
|
||||
<!-- TODO: Clarify actual request with bloom filter to mailserver -->
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.1
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
* Created document
|
||||
* Forked from [3-whisper-usage](3-whisper-usage.md)
|
||||
* Change to keep `Mailserver` term consistent
|
||||
* Replaced Whisper references with Waku
|
||||
* Added [Status options](#status) section
|
||||
* Updated [Waku packets](#waku-packets) section to match Waku
|
||||
* Added that `Batch Ack` is marked for deprecation
|
||||
* Changed `shh_generateSymKeyFromPassword` to `waku_generateSymKeyFromPassword`
|
||||
* [Exists here](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/waku/api.go#L172-L175)
|
||||
* [Exists here](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/eth-node/bridge/geth/public_waku_api.go#L33-L36)
|
||||
* Changed `shh_markTrustedPeer` to `waku_markTrustedPeer`
|
||||
* [Exists here](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/waku/api.go#L100-L108)
|
||||
* Changed `shhext_requestMessages` to `wakuext_requestMessages`
|
||||
* [Exists here](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/services/wakuext/api.go#L76-L139)
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
* [Waku](waku)
|
||||
* [WAKU1](/waku/standards/legacy/6/waku1.md)
|
||||
* [WAKU-MAILSERVER](/status/deprecated/waku-mailserver.md)
|
||||
* [The theoretical scaling model](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability)
|
||||
* [SECURE-TRANSPORT](/status/deprecated/secure-transport.md)
|
||||
* [May 22, 2020 commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
* [`shh_generateSymKeyFromPassword`](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API#shh_generatesymkeyfrompassword)
|
||||
* [Key Change #1](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/waku/api.go#L172-L175)
|
||||
* [Key Change #2](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/eth-node/bridge/geth/public_waku_api.go#L33-L36)
|
||||
* [Key Change #3](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/waku/api.go#L100-L108)
|
||||
* [Key Change #4](https://github.com/status-im/status-go/blob/2d13ccf5ec3db7e48d7a96a7954be57edb96f12f/services/wakuext/api.go#L76-L139)
|
||||
148
status/deprecated/whisper-mailserver.md
Normal file
148
status/deprecated/whisper-mailserver.md
Normal file
@@ -0,0 +1,148 @@
|
||||
---
|
||||
title: WHISPER-MAILSERVER
|
||||
name: Whisper mailserver
|
||||
status: deprecated
|
||||
description: Whisper Mailserver is a Whisper extension that allows to store messages permanently and deliver them to the clients even though they are already not available in the network and expired.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
Being mostly offline is an intrinsic property of mobile clients.
|
||||
They need to save network transfer and battery consumption
|
||||
to avoid spending too much money or constant charging.
|
||||
Whisper protocol, on the other hand, is an online protocol.
|
||||
Messages are available in the Whisper network only for short period of time calculate in seconds.
|
||||
|
||||
Whisper `Mailserver` is a Whisper extension that allows to store messages permanently
|
||||
and deliver them to the clients even though they are already not available in the network and expired.
|
||||
|
||||
## `Mailserver`
|
||||
|
||||
From the network perspective, `Mailserver` is just like any other Whisper node.
|
||||
The only difference is that it has a capability of archiving messages and delivering them to its peers on-demand.
|
||||
|
||||
It is important to notice that `Mailserver` will only handle requests from its direct peers
|
||||
and exchanged packets between `Mailserver` and a peer are p2p messages.
|
||||
|
||||
### Archiving messages
|
||||
|
||||
A node which wants to provide `Mailserver` functionality MUST store envelopes
|
||||
from incoming message packets (Whisper packet-code `0x01`).
|
||||
The envelopes can be stored in any format,
|
||||
however they MUST be serialized and deserialized to the Whisper envelope format.
|
||||
|
||||
A `Mailserver` SHOULD store envelopes for all topics to be generally useful for any peer,
|
||||
however for specific use cases it MAY store envelopes for a subset of topics.
|
||||
|
||||
### Requesting messages
|
||||
|
||||
In order to request historic messages, a node MUST send a packet P2P Request (`0x7e`) to a peer providing `Mailserver` functionality.
|
||||
This packet requires one argument which MUST be a Whisper envelope.
|
||||
|
||||
In the Whisper envelope's payload section, there MUST be RLP-encoded information about the details of the request:
|
||||
|
||||
```golang
|
||||
[ Lower, Upper, Bloom, Limit, Cursor ]
|
||||
```
|
||||
|
||||
`Lower`: 4-byte wide unsigned integer (UNIX time in seconds; oldest requested envelope's creation time)
|
||||
`Upper`: 4-byte wide unsigned integer (UNIX time in seconds; newest requested envelope's creation time)
|
||||
`Bloom`: 64-byte wide array of Whisper topics encoded in a bloom filter to filter envelopes
|
||||
`Limit`: 4-byte wide unsigned integer limiting the number of returned envelopes
|
||||
`Cursor`: an array of a cursor returned from the previous request (optional)
|
||||
|
||||
The `Cursor` field SHOULD be filled in
|
||||
if a number of envelopes between `Lower` and `Upper` is greater than `Limit`
|
||||
so that the requester can send another request using the obtained `Cursor` value.
|
||||
What exactly is in the `Cursor` is up to the implementation.
|
||||
The requester SHOULD NOT use a `Cursor` obtained from one `Mailserver` in a request to another `Mailserver`
|
||||
because the format or the result MAY be different.
|
||||
|
||||
The envelope MUST be encrypted with a symmetric key agreed between the requester and `Mailserver`.
|
||||
|
||||
### Receiving historic messages
|
||||
|
||||
Historic messages MUST be sent to a peer as a packet with a P2P Message code (`0x7f`)
|
||||
followed by an array of Whisper envelopes.
|
||||
It is incompatible with the original Whisper spec (EIP-627) because it allows only a single envelope,
|
||||
however, an array of envelopes is much more performant.
|
||||
In order to stay compatible with EIP-627, a peer receiving historic message MUST handle both cases.
|
||||
|
||||
In order to receive historic messages from a `Mailserver`, a node MUST trust the selected `Mailserver`,
|
||||
that is allowed to send packets with the P2P Message code. By default, the node discards such packets.
|
||||
|
||||
Received envelopes MUST be passed through the Whisper envelope pipelines
|
||||
so that they are picked up by registered filters and passed to subscribers.
|
||||
|
||||
For a requester, to know that all messages have been sent by `Mailserver`,
|
||||
it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by the following parameters:
|
||||
|
||||
```golang
|
||||
[ RequestID, LastEnvelopeHash, Cursor ]
|
||||
```
|
||||
|
||||
`RequestID`: 32-byte wide array with a Keccak-256 hash of the envelope containing the original request
|
||||
`LastEnvelopeHash`: 32-byte wide array with a Keccak-256 hash of the last sent envelope for the request
|
||||
`Cursor`: an array of a cursor returned from the previous request (optional)
|
||||
|
||||
If `Cursor` is not empty, it means that not all messages were sent due to the set `Limit` in the request.
|
||||
One or more consecutive requests MAY be sent with `Cursor` field filled in order to receive the rest of the messages.
|
||||
|
||||
## Security considerations
|
||||
|
||||
### Confidentiality
|
||||
|
||||
The node encrypts all Whisper envelopes. A `Mailserver` node can not inspect their contents.
|
||||
|
||||
### Altruistic and centralized operator risk
|
||||
|
||||
In order to be useful, a `Mailserver` SHOULD be online most of the time. That means
|
||||
users either have to be a bit tech-savvy to run their own node, or rely on someone
|
||||
else to run it for them.
|
||||
|
||||
Currently, one of Status's legal entities provides `Mailservers` in an altruistic manner, but this is
|
||||
suboptimal from a decentralization, continuance and risk point of view. Coming
|
||||
up with a better system for this is ongoing research.
|
||||
|
||||
A Status client SHOULD allow the `Mailserver` selection to be customizable.
|
||||
|
||||
### Privacy concerns
|
||||
|
||||
In order to use a `Mailserver`, a given node needs to connect to it directly,
|
||||
i.e. add the `Mailserver` as its peer and mark it as trusted.
|
||||
This means that the `Mailserver` is able to send direct p2p messages to the node instead of broadcasting them.
|
||||
Effectively, it will have access to the bloom filter of topics
|
||||
that the user is interested in,
|
||||
when it is online as well as many metadata like IP address.
|
||||
|
||||
### Denial-of-service
|
||||
|
||||
Since a `Mailserver` is delivering expired envelopes and has a direct TCP connection with the recipient,
|
||||
the recipient is vulnerable to DoS attacks from a malicious `Mailserver` node.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
- Change to keep `Mailserver` term consistent
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Whisper](https://eips.ethereum.org/EIPS/eip-627)
|
||||
- [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md)
|
||||
- [SECURE-TRANSPORT](/status/deprecated/secure-transport.md)
|
||||
- [`shh_generateSymKeyFromPassword`](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API#shh_generatesymkeyfrompassword)
|
||||
- [Whisper v6](https://eips.ethereum.org/EIPS/eip-627)
|
||||
- [Waku V0](/waku/deprecated/5/waku0.md)
|
||||
- [Waku V1](/waku/standards/legacy/6/waku1.md)
|
||||
- [May 22, 2020 change commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
401
status/deprecated/whisper-usage.md
Normal file
401
status/deprecated/whisper-usage.md
Normal file
@@ -0,0 +1,401 @@
|
||||
---
|
||||
title: WHISPER-USAGE
|
||||
name: Whisper Usage
|
||||
status: deprecated
|
||||
description: Status uses Whisper to provide privacy-preserving routing and messaging on top of devP2P.
|
||||
editor: Filip Dimitrijevic <filip@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Andrea Piana <andreap@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
Status uses [Whisper](https://eips.ethereum.org/EIPS/eip-627) to provide
|
||||
privacy-preserving routing and messaging on top of devP2P.
|
||||
Whisper uses topics to partition its messages,
|
||||
and these are leveraged for all chat capabilities.
|
||||
In the case of public chats, the channel name maps directly to its Whisper topic.
|
||||
This allows anyone to listen on a single channel.
|
||||
|
||||
Additionally, since anyone can receive Whisper envelopes,
|
||||
it relies on the ability to decrypt messages to decide who is the correct recipient.
|
||||
Status nodes do not rely upon this property,
|
||||
and implement another secure transport layer on top of Whisper.
|
||||
|
||||
Finally, using an extension of Whisper provides the ability to do offline messaging.
|
||||
|
||||
## Reason
|
||||
|
||||
Provide routing, metadata protection, topic-based multicasting and basic
|
||||
encryption properties to support asynchronous chat.
|
||||
|
||||
## Terminology
|
||||
|
||||
* *Whisper node*: an Ethereum node with Whisper V6 enabled (in the case of go-ethereum, it's `--shh` option)
|
||||
* *Whisper network*: a group of Whisper nodes connected together through the internet connection and forming a graph
|
||||
* *Message*: a decrypted Whisper message
|
||||
* *Offline message*: an archived envelope
|
||||
* *Envelope*: an encrypted message with metadata like topic and Time-To-Live
|
||||
|
||||
## Whisper packets
|
||||
|
||||
| Packet Name | Code | EIP-627 | References |
|
||||
| --- | --: | --- | --- |
|
||||
| Status | 0 | ✔ | [Handshake](#handshake) |
|
||||
| Messages | 1 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
|
||||
| PoW Requirement | 2 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
|
||||
| Bloom Filter | 3 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
|
||||
| Batch Ack | 11 | 𝘅 | Undocumented |
|
||||
| Message Response | 12 | 𝘅 | Undocumented |
|
||||
| P2P Sync Request | 123 | 𝘅 | Undocumented |
|
||||
| P2P Sync Response | 124 | 𝘅 | Undocumented |
|
||||
| P2P Request Complete | 125 | 𝘅 | [4/WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md) |
|
||||
| P2P Request | 126 | ✔ | [4/WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md) |
|
||||
| P2P Messages | 127 | ✔/𝘅 (EIP-627 supports only single envelope in a packet) | [4/WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md) |
|
||||
|
||||
## Whisper node configuration
|
||||
|
||||
A Whisper node must be properly configured to receive messages from Status clients.
|
||||
|
||||
Nodes use Whisper's Proof Of Work algorithm to deter denial of service
|
||||
and various spam/flood attacks against the Whisper network.
|
||||
The sender of a message must perform some work which in this case means processing time.
|
||||
Because Status' main client is a mobile client, this easily leads to battery draining and poor performance of the app itself.
|
||||
Hence, all clients MUST use the following Whisper node settings:
|
||||
|
||||
* proof-of-work requirement not larger than `0.002`
|
||||
* time-to-live not lower than `10` (in seconds)
|
||||
|
||||
## Handshake
|
||||
|
||||
Handshake is a RLP-encoded packet sent to a newly connected peer. It MUST start with a Status Code (`0x00`) and follow up with items:
|
||||
|
||||
```golang
|
||||
[ protocolVersion, PoW, bloom, isLightNode, confirmationsEnabled, rateLimits ]
|
||||
```
|
||||
|
||||
`protocolVersion`: version of the Whisper protocol
|
||||
`PoW`: minimum PoW accepted by the peer
|
||||
`bloom`: bloom filter of Whisper topic accepted by the peer
|
||||
`isLightNode`: when true, the peer won't forward messages
|
||||
`confirmationsEnabled`: when true, the peer will send message confirmations
|
||||
`rateLimits`: is `[ RateLimitIP, RateLimitPeerID, RateLimitTopic ]` where each values is an integer with a number of accepted packets per second per IP, Peer ID, and Topic respectively
|
||||
|
||||
`bloom, isLightNode, confirmationsEnabled, and rateLimits` are all optional arguments in the handshake. However, if an optional field is specified, all optional fields preceding it MUST also be specified in order to be unambiguous.
|
||||
|
||||
## Rate limiting
|
||||
|
||||
In order to provide an optional very basic Denial-of-Service attack protection, each node SHOULD define its own rate limits.
|
||||
The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
|
||||
|
||||
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
|
||||
|
||||
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
|
||||
|
||||
Each node SHOULD broadcast its rate limits to its peers using rate limits packet code (`0x14`). The rate limits is RLP-encoded information:
|
||||
|
||||
```golang
|
||||
[ IP limits, PeerID limits, Topic limits ]
|
||||
```
|
||||
|
||||
`IP limits`: 4-byte wide unsigned integer
|
||||
`PeerID limits`: 4-byte wide unsigned integer
|
||||
`Topic limits`: 4-byte wide unsigned integer
|
||||
|
||||
The rate limits MAY also be sent as an optional parameter in the handshake.
|
||||
|
||||
Each node SHOULD respect rate limits advertised by its peers.
|
||||
The number of packets SHOULD be throttled in order not to exceed peer's rate limits.
|
||||
If the limit gets exceeded, the connection MAY be dropped by the peer.
|
||||
|
||||
## Keys management
|
||||
|
||||
The protocol requires a key (symmetric or asymmetric) for the following actions:
|
||||
|
||||
* signing & verifying messages (asymmetric key)
|
||||
* encrypting & decrypting messages (asymmetric or symmetric key).
|
||||
|
||||
As nodes require asymmetric keys and symmetric keys to process incoming messages,
|
||||
they must be available all the time and are stored in memory.
|
||||
|
||||
Keys management for PFS is described in [5/SECURE-TRANSPORT](/status/deprecated/whisper-mailserver.md).
|
||||
|
||||
The Status protocols uses a few particular Whisper topics to achieve its goals.
|
||||
|
||||
### Contact code topic
|
||||
|
||||
Nodes use the contact code topic to facilitate the discovery of X3DH bundles so that the first message can be PFS-encrypted.
|
||||
|
||||
Each user publishes periodically to this topic.
|
||||
If user A wants to contact user B, she SHOULD look for their bundle on this contact code topic.
|
||||
|
||||
Contact code topic MUST be created following the algorithm below:
|
||||
|
||||
```golang
|
||||
contactCode := "0x" + hexEncode(activePublicKey) + "-contact-code"
|
||||
|
||||
var hash []byte = keccak256(contactCode)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
### Partitioned topic
|
||||
|
||||
Whisper is broadcast-based protocol.
|
||||
In theory, everyone could communicate using a single topic but that would be extremely inefficient.
|
||||
Opposite would be using a unique topic for each conversation,
|
||||
however, this brings privacy concerns because it would be much easier to detect whether
|
||||
and when two parties have an active conversation.
|
||||
|
||||
Nodes use partitioned topics to broadcast private messages efficiently.
|
||||
By selecting a number of topic, it is possible to balance efficiency and privacy.
|
||||
|
||||
Currently, nodes set the number of partitioned topics to `5000`.
|
||||
They MUST be generated following the algorithm below:
|
||||
|
||||
```golang
|
||||
var partitionsNum *big.Int = big.NewInt(5000)
|
||||
var partition *big.Int = big.NewInt(0).Mod(publicKey.X, partitionsNum)
|
||||
|
||||
partitionTopic := "contact-discovery-" + strconv.FormatInt(partition.Int64(), 10)
|
||||
|
||||
var hash []byte = keccak256(partitionTopic)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
### Public chats
|
||||
|
||||
A public chat MUST use a topic derived from a public chat name following the algorithm below:
|
||||
|
||||
```golang
|
||||
var hash []byte
|
||||
hash = keccak256(name)
|
||||
|
||||
topicLen = 4
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
<!-- NOTE: commented out as it is currently not used. In code for potential future use. - C.P. Oct 8, 2019
|
||||
### Personal discovery topic
|
||||
|
||||
Personal discovery topic is used to ???
|
||||
|
||||
A client MUST implement it following the algorithm below:
|
||||
```golang
|
||||
personalDiscoveryTopic := "contact-discovery-" + hexEncode(publicKey)
|
||||
|
||||
var hash []byte = keccak256(personalDiscoveryTopic)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
Each Status Client SHOULD listen to this topic in order to receive ??? -->
|
||||
|
||||
<!-- NOTE: commented out as it is no longer valid as of V1. - C.P. Oct 8, 2019
|
||||
### Generic discovery topic
|
||||
|
||||
Generic discovery topic is a legacy topic used to handle all one-to-one chats. The newer implementation should rely on [Partitioned Topic](#partitioned-topic) and [Personal discovery topic](#personal-discovery-topic).
|
||||
|
||||
Generic discovery topic MUST be created following [Public chats](#public-chats) topic algorithm using string `contact-discovery` as a name. -->
|
||||
|
||||
### Group chat topic
|
||||
|
||||
Group chats does not have a dedicated topic.
|
||||
All group chat messages (including membership updates) are sent as one-to-one messages to multiple recipients.
|
||||
|
||||
### Negotiated topic
|
||||
|
||||
When a client sends a one to one message to another client, it MUST listen to their negotiated topic.
|
||||
This is computed by generating a diffie-hellman key exchange between two members
|
||||
and taking the first four bytes of the `SHA3-256` of the key generated.
|
||||
|
||||
```golang
|
||||
|
||||
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
|
||||
ecies.ImportECDSAPublic(theirPublicKey),
|
||||
16,
|
||||
16,
|
||||
)
|
||||
|
||||
|
||||
hexEncodedKey := hex.EncodeToString(sharedKey)
|
||||
|
||||
var hash []byte = keccak256(hexEncodedKey)
|
||||
var topicLen int = 4
|
||||
|
||||
if len(hash) < topicLen {
|
||||
topicLen = len(hash)
|
||||
}
|
||||
|
||||
var topic [4]byte
|
||||
for i = 0; i < topicLen; i++ {
|
||||
topic[i] = hash[i]
|
||||
}
|
||||
```
|
||||
|
||||
A client SHOULD send to the negotiated topic only if it has received a message from all the devices included in the conversation.
|
||||
|
||||
### Flow
|
||||
|
||||
To exchange messages with client `B`, a client `A` SHOULD:
|
||||
|
||||
* Listen to client's `B` Contact Code Topic to retrieve their bundle information, including a list of active devices
|
||||
* Send a message on client's `B` partitioned topic
|
||||
* Listen to the Negotiated Topic between `A` & `B`
|
||||
* Once client `A` receives a message from `B`, the Negotiated Topic SHOULD be used
|
||||
|
||||
## Message encryption
|
||||
|
||||
Even though, the protocol specifies an encryption layer that encrypts messages before passing them to the transport layer,
|
||||
Whisper protocol requires each Whisper message to be encrypted anyway.
|
||||
|
||||
The node encrypts public and group messages using symmetric encryption, and creates the key from a channel name string.
|
||||
The implementation is available in [`shh_generateSymKeyFromPassword`](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API#shh_generatesymkeyfrompassword) JSON-RPC method of go-ethereum Whisper implementation.
|
||||
|
||||
The node encrypts one-to-one messages using asymmetric encryption.
|
||||
|
||||
## Message confirmations
|
||||
|
||||
Sending a message is a complex process where many things can go wrong.
|
||||
Message confirmations tell a node that a message originating from it has been seen by its direct peers.
|
||||
|
||||
A node MAY send a message confirmation for any batch of messages received in a packet Messages Code (`0x01`).
|
||||
|
||||
A node sends a message confirmation using Batch Acknowledge packet (`0x0b`) or Message Response packet (`0x0c`).
|
||||
|
||||
The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data (raw bytes).
|
||||
|
||||
The Message Response packet is more complex and is followed by a Versioned Message Response:
|
||||
|
||||
```golang
|
||||
[ Version, Response]
|
||||
```
|
||||
|
||||
`Version`: a version of the Message Response, equal to `1`,
|
||||
`Response`: `[ Hash, Errors ]` where `Hash` is a keccak256 hash of the envelopes batch data (raw bytes)
|
||||
for which the confirmation is sent and `Errors` is a list of envelope errors when processing the batch.
|
||||
A single error contains `[ Hash, Code, Description ]` where `Hash` is a hash of the processed envelope,
|
||||
`Code` is an error code and `Description` is a descriptive error message.
|
||||
|
||||
The supported codes:
|
||||
`1`: means time sync error which happens when an envelope is too old
|
||||
or created in the future (the root cause is no time sync between nodes).
|
||||
|
||||
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message,
|
||||
one or more peers broadcast a corresponding confirmation.
|
||||
To limit that, both Batch Acknowledge packet (`0x0b`) and Message Response packet (`0x0c`) are not broadcast to peers of the peers,
|
||||
i.e. they do not follow epidemic spread.
|
||||
|
||||
In the current Status network setup, only `Mailservers` support message confirmations.
|
||||
A client posting a message to the network and after receiving a confirmation can be sure that the message got processed by the `Mailserver`.
|
||||
If additionally, sending a message is limited to non-`Mailserver` peers,
|
||||
it also guarantees that the message got broadcast through the network and it reached the selected `Mailserver`.
|
||||
|
||||
## Whisper / Waku bridging
|
||||
|
||||
In order to maintain compatibility between Whisper and Waku nodes,
|
||||
a Status network that implements both Whisper and Waku messaging protocols
|
||||
MUST have at least one node that is capable of discovering peers and implements
|
||||
[Whisper v6](https://eips.ethereum.org/EIPS/eip-627),
|
||||
[Waku V0](/waku/deprecated/5/waku0.md) and
|
||||
[Waku V1](/waku/standards/legacy/6/waku1.md) specifications.
|
||||
|
||||
Additionally, any Status network that implements both Whisper and Waku messaging protocols
|
||||
MUST implement bridging capabilities as detailed in
|
||||
[Waku V1#Bridging](/waku/standards/legacy/6/waku1.md#waku-whisper-bridging).
|
||||
|
||||
## Whisper V6 extensions
|
||||
|
||||
### Request historic messages
|
||||
|
||||
Sends a request for historic messages to a `Mailserver`.
|
||||
The `Mailserver` node MUST be a direct peer and MUST be marked as trusted (using `shh_markTrustedPeer`).
|
||||
|
||||
The request does not wait for the response.
|
||||
It merely sends a peer-to-peer message to the `Mailserver`
|
||||
and it's up to `Mailserver` to process it and start sending historic messages.
|
||||
|
||||
The drawback of this approach is that it is impossible to tell
|
||||
which historic messages are the result of which request.
|
||||
|
||||
It's recommended to return messages from newest to oldest.
|
||||
To move further back in time, use `cursor` and `limit`.
|
||||
|
||||
#### shhext_requestMessages
|
||||
|
||||
**Parameters**:
|
||||
|
||||
1. Object - The message request object:
|
||||
* `mailServerPeer` - `String`: `Mailserver`'s enode address.
|
||||
* `from` - `Number` (optional): Lower bound of time range as unix timestamp, default is 24 hours back from now.
|
||||
* `to` - `Number` (optional): Upper bound of time range as unix timestamp, default is now.
|
||||
* `limit` - `Number` (optional): Limit the number of messages sent back, default is no limit.
|
||||
* `cursor` - `String` (optional): Used for paginated requests.
|
||||
* `topics` - `Array`: hex-encoded message topics.
|
||||
* `symKeyID` - `String`: an ID of a symmetric key used to authenticate with the `Mailserver`, derived from Mailserver password.
|
||||
|
||||
**Returns**:
|
||||
`Boolean` - returns `true` if the request was sent.
|
||||
|
||||
The above `topics` is then converted into a bloom filter and then and sent to the `Mailserver`.
|
||||
|
||||
<!-- TODO: Clarify actual request with bloom filter to mailserver -->
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [May 22, 2020](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
|
||||
* Added Whisper / Waku Bridging section
|
||||
* Change to keep `Mailserver` term consistent
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
* [Whisper](https://eips.ethereum.org/EIPS/eip-627)
|
||||
* [WHISPER-MAILSERVER](/status/deprecated/whisper-mailserver.md)
|
||||
* [SECURE-TRANSPORT](/status/deprecated/secure-transport.md)
|
||||
* [`shh_generateSymKeyFromPassword`](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API#shh_generatesymkeyfrompassword)
|
||||
* [Whisper v6](https://eips.ethereum.org/EIPS/eip-627)
|
||||
* [Waku V0](/waku/deprecated/5/waku0.md)
|
||||
* [Waku V1](/waku/standards/legacy/6/waku1.md)
|
||||
* [May 22, 2020 change commit](https://github.com/status-im/specs/commit/664dd1c9df6ad409e4c007fefc8c8945b8d324e8)
|
||||
@@ -74,10 +74,11 @@ However, since all users of the Status app MUST be able to participate in 1:1 ch
|
||||
the functional scope of messages enabling 1:1 chats MUST be a global scope.
|
||||
Similarly, since private group chats can be set up between any subset of Status app users,
|
||||
the functional scope for messages related to private group chats MUST be global.
|
||||
As a counter-example, messages that originate within a community (and are _addressed_ to members of that community)
|
||||
are only of interest to participants that are also members of that community.
|
||||
Such messages MUST have a community-wide functional scope.
|
||||
A third group of messages are addressed only to the participant that generated those messages itself.
|
||||
Along the same principle, messages that originate within communities are of global interest
|
||||
for all users who have an interest in the Status Communities feature.
|
||||
Such messages MUST have a global functional scope,
|
||||
that can be accessed by any app users interested in communities.
|
||||
A different group of messages are addressed only to the participant that generated those messages itself.
|
||||
These _self-addressed_ messages MUST have a local functional scope.
|
||||
|
||||
If we further make a distinction between "control" and "content" messages,
|
||||
@@ -85,15 +86,19 @@ we can distinguish five distinct functional scopes.
|
||||
|
||||
All Status messages MUST have one of these functional scopes:
|
||||
|
||||
#### Global scope
|
||||
#### Global general scope
|
||||
|
||||
1. _Global control_: messages enabling the basic functioning of the app to control features that all app users should be able to participate in. Examples include Contact Requests, Community Invites, global Status Updates, Group Chat Invites, etc.
|
||||
1. _Global control_: messages enabling the basic functioning of the app to control general features that all app users should be able to participate in. Examples include Contact Requests, global Status Updates, Group Chat Invites, etc.
|
||||
2. _Global content_: messages carrying user-generated content for global functions. Examples include 1:1 chat messages, images shared over private group chats, etc.
|
||||
|
||||
#### Community scope
|
||||
#### Global community scope
|
||||
|
||||
1._Community control_: messages enabling the basic functioning of the app to control features _only relevant to members of a specific community_. Examples include Community Membership Updates, community Status Updates, etc.
|
||||
2. _Community content_: messages carrying user-generated content _only for members of a specific community_.
|
||||
1. _Global community control_: messages enabling the basic functioning of the app to control features related to communities. Examples include Community Invites, Community Membership Updates, community Status Updates, etc.
|
||||
2. _Global community content_: messages carrying user-generated content for members of any community.
|
||||
|
||||
> **Note:** a previous iteration of the Status Communities feature defined separate community-wide scopes for each community.
|
||||
However, this model was deprecated and all communities now operate on a global, shared scope.
|
||||
This implies that different communities will share shards on the routing layer.
|
||||
|
||||
#### Local scope
|
||||
|
||||
@@ -104,7 +109,7 @@ It SHOULD however inform the underlying [transport layer sharding](#pubsub-topic
|
||||
In general a Status client SHOULD subscribe to participate in:
|
||||
|
||||
- all global functions
|
||||
- (only) the community functions for communities of which it is a member, and
|
||||
- global community functions if it is interested in this feature, and
|
||||
- its own local functions.
|
||||
|
||||
### Content topics
|
||||
@@ -205,9 +210,9 @@ It is RECOMMENDED that separate pubsub topics be used for global control message
|
||||
|
||||
#### Community messages
|
||||
|
||||
The application SHOULD define at least one separate pubsub topic for each separate community's control and content messages.
|
||||
The application MAY define a set of more than one pubsub topic per community to allow traffic sharding for scalability.
|
||||
It is RECOMMENDED that separate pubsub topics be used for community control messages and community content messages.
|
||||
The application SHOULD define at least one distinct pubsub topic for global community control messages and global community content messages.
|
||||
The application MAY define a set of more than one pubsub topic for global community messages to allow traffic sharding for scalability.
|
||||
It is RECOMMENDED that separate pubsub topics be used for global community control messages and global community content messages.
|
||||
|
||||
#### Large messages
|
||||
|
||||
@@ -270,8 +275,8 @@ Full clients SHOULD use relay protocol as preferred method to subscribe to pubsu
|
||||
|
||||
1. Global control
|
||||
2. Global content
|
||||
3. Community control, for each community of which the app user is a member
|
||||
4. Community content, for each community of which the app user is a member
|
||||
3. Global community control, if the client has activated the Status Communities feature
|
||||
4. Global community content, if the client has activated the Status Communities feature
|
||||
|
||||
Light clients SHOULD use filter protocol to subscribe only to the content topics relevant to the user.
|
||||
|
||||
@@ -297,8 +302,8 @@ Full clients SHOULD use relay protocol to publish to pubsub topics matching the
|
||||
|
||||
1. Global control
|
||||
2. Global content
|
||||
3. Community control, for each community of which the app user is a member
|
||||
4. Community content, for each community of which the app user is a member
|
||||
3. Global community control, if the client has activated the Status Communities feature
|
||||
4. Global community content, if the client has activated the Status Communities feature
|
||||
|
||||
Light clients SHOULD use lightpush protocol to publish control and content messages.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: VAC-DECENTRALIZED-MESSAGING-ETHEREUM
|
||||
title: ETH-DCGKA
|
||||
name: Decentralized Key and Session Setup for Secure Messaging over Ethereum
|
||||
status: raw
|
||||
category: informational
|
||||
|
||||
@@ -1,13 +1,19 @@
|
||||
---
|
||||
title: ETH-SECPM
|
||||
name: Secure channel setup using Ethereum accounts
|
||||
status: raw
|
||||
status: deleted
|
||||
category: Standards Track
|
||||
tags:
|
||||
editor: Ramses Fernandez <ramses@status.im>
|
||||
contributors:
|
||||
---
|
||||
|
||||
## NOTE
|
||||
|
||||
The content of this specification has been split between
|
||||
[ETH-DEMLS](vac/raw/eth-demls.md) and [NOISE-X3DH-RATCHET](vac/raw/noise-x3dh-ratchet.md)
|
||||
RFCs.
|
||||
|
||||
## Motivation
|
||||
|
||||
The need for secure communications has become paramount.
|
||||
@@ -416,7 +422,7 @@ Credentials MUST follow the specifications of section 5.3 of
|
||||
|
||||
Below follows the flow diagram for the generation of credentials.
|
||||
Users MUST generate key pairs by themselves.
|
||||

|
||||

|
||||
|
||||
### Message framing
|
||||
|
||||
@@ -759,10 +765,10 @@ CredentialType credential_types<V>;
|
||||
|
||||
The flow diagram shows the procedure to fetch key material from other
|
||||
users:
|
||||

|
||||

|
||||
|
||||
Below follows the flow diagram for the creation of a group:
|
||||

|
||||

|
||||
|
||||
### Group evolution
|
||||
|
||||
@@ -837,15 +843,15 @@ The client MUST apply the proposals in the list in the order described
|
||||
in Section 12.3 of [RFC9420](https://datatracker.ietf.org/docrfc9420/).
|
||||
|
||||
Below follows the flow diagram for the addition of a member to a group:
|
||||

|
||||

|
||||
|
||||
The diagram below shows the procedure to remove a group member:
|
||||
|
||||

|
||||

|
||||
|
||||
The flow diagram below shows an update procedure:
|
||||
|
||||

|
||||

|
||||
|
||||
### Commit messages
|
||||
|
||||
@@ -1287,7 +1293,7 @@ and checks that it corresponds to an address contained in the ACL.
|
||||
7. Off-chain - Alice sends a welcome message to Bob.
|
||||
8. Off-chain - Alice SHOULD broadcasts a message announcing the
|
||||
addition of Bob to other users of the group.
|
||||

|
||||

|
||||
|
||||
#### Alice does not know Bob’s Ethereum address
|
||||
|
||||
@@ -1310,7 +1316,7 @@ contract.
|
||||
8. Off-chain - Alice SHOULD broadcasts a message announcing the
|
||||
addition of Bob to other users of the group.
|
||||
|
||||

|
||||

|
||||
|
||||
### Considerations regarding smart contracts
|
||||
|
||||
@@ -1330,7 +1336,7 @@ off-chain message.
|
||||
- The creator of the contract MUST update the ACL, and send
|
||||
messages to the group for key update.
|
||||
|
||||

|
||||

|
||||
|
||||
> It is important to note that both
|
||||
user removal and updates of any kind
|
||||
241
vac/raw/eth-mls-offchain.md
Normal file
241
vac/raw/eth-mls-offchain.md
Normal file
@@ -0,0 +1,241 @@
|
||||
---
|
||||
title: ETH-MLS-OFFCHAIN
|
||||
name: Secure channel setup using decentralized MLS and Ethereum accounts
|
||||
status: raw
|
||||
category: Standards Track
|
||||
tags:
|
||||
editor: Ugur Sen [ugur@status.im](mailto:ugur@status.im)
|
||||
contributors: seemenkina [ekaterina@status.im](mailto:ekaterina@status.im)
|
||||
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
The following document specifies Ethereum authenticated scalable
|
||||
and decentralized secure group messaging application by
|
||||
integrating Message Layer Security (MLS) backend.
|
||||
Decentralization refers each user is a node in P2P network and
|
||||
each user has voice for any changes in group.
|
||||
This is achieved by integrating a consensus mechanism.
|
||||
Lastly, this RFC can also be referred to as de-MLS,
|
||||
decentralized MLS, to emphasize its deviation
|
||||
from the centralized trust assumptions of traditional MLS deployments.
|
||||
|
||||
## Motivation
|
||||
|
||||
Group messaging is a fundamental part of digital communication,
|
||||
yet most existing systems depend on centralized servers,
|
||||
which introduce risks around privacy, censorship, and unilateral control.
|
||||
In restrictive settings, servers can be blocked or surveilled;
|
||||
in more open environments, users still face opaque moderation policies,
|
||||
data collection, and exclusion from decision-making processes.
|
||||
To address this, we propose a decentralized, scalable peer-to-peer
|
||||
group messaging system where each participant runs a node, contributes
|
||||
to message propagation, and takes part in governance autonomously.
|
||||
Group membership changes are decided collectively through a lightweight
|
||||
partially synchronous, fault-tolerant consensus protocol without a centralized identity.
|
||||
This design enables truly democratic group communication and is well-suited
|
||||
for use cases like activist collectives, research collaborations, DAOs, support groups,
|
||||
and decentralized social platforms.
|
||||
|
||||
## Format Specification
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document
|
||||
are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
### Assumptions
|
||||
|
||||
- The nodes in the P2P network can discover other nodes or will connect to other nodes when subscribing to same topic in a gossipsub.
|
||||
- We MAY have non-reliable (silent) nodes.
|
||||
- We MUST have a consensus that is lightweight, scalable and finalized in a specific time.
|
||||
|
||||
## Roles
|
||||
|
||||
The three roles used in de-MLS is as follows:
|
||||
|
||||
- `node`: Nodes are members of network without being in any secure group messaging.
|
||||
- `member`: Members are special nodes in the secure group messaging who
|
||||
obtains current group key of secure group messaging.
|
||||
- `steward`: Stewards are special and transparent members in secure group
|
||||
messaging who organizes the changes upon the voted-proposals.
|
||||
|
||||
## MLS Background
|
||||
|
||||
The de-MLS consists of MLS backend, so the MLS services and other MLS components
|
||||
are taken from the original [MLS specification](https://datatracker.ietf.org/doc/rfc9420/), with or without modifications.
|
||||
|
||||
### MLS Services
|
||||
|
||||
MLS is operated in two services authentication service (AS) and delivery service (DS).
|
||||
Authentication service enables group members to authenticate the credentials presented by other group members.
|
||||
The delivery service routes MLS messages among the nodes or
|
||||
members in the protocol in the correct order and
|
||||
manage the `keyPackage` of the users where the `keyPackage` is the objects
|
||||
that provide some public information about a user.
|
||||
|
||||
### MLS Objects
|
||||
|
||||
Following section presents the MLS objects and components that used in this RFC:
|
||||
|
||||
`Epoch`: Fixed time intervals that changes the state that is defined by members,
|
||||
section 3.4 in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/).
|
||||
|
||||
`MLS proposal message:` Members MUST receive the proposal message prior to the
|
||||
corresponding commit message that initiates a new epoch with key changes,
|
||||
in order to ensure the intended security properties, section 12.1 in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/).
|
||||
Here, the add and remove proposals are used.
|
||||
|
||||
`Application message`: This message type used in arbitrary encrypted communication between group members.
|
||||
This is restricted by [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/) as if there is pending proposal,
|
||||
the application message should be cut.
|
||||
Note that: Since the MLS is based on servers, this delay between proposal and commit messages are very small.
|
||||
`Commit message:` After members receive the proposals regarding group changes,
|
||||
the committer, who may be any member of the group, as specified in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/),
|
||||
generates the necessary key material for the next epoch, including the appropriate welcome messages
|
||||
for new joiners and new entropy for removed members. In this RFC, the committers only MUST be stewards.
|
||||
|
||||
### de-MLS Objects
|
||||
|
||||
`Voting proposal` Similar to MLS proposals, but processed only if approved through a voting process.
|
||||
They function as application messages in the MLS group,
|
||||
allowing the steward to collect them without halting the protocol.
|
||||
|
||||
## Flow
|
||||
|
||||
General flow is as follows:
|
||||
|
||||
- A steward initializes a group just once, and then sends out Group Announcements (GA) periodically.
|
||||
|
||||
- Meanwhile, each`node`creates and sends their`credential` includes `keyPackage`.
|
||||
- Each `member`creates `voting proposals` sends them to from MLS group during epoch E.
|
||||
- Meanwhile, the `steward` collects finalized `voting proposals` from MLS group and converts them into
|
||||
`MLS proposals` then sends them with correspondng `commit messages`
|
||||
- Evantually, with the commit messages, all members starts the next epoch E+1.
|
||||
|
||||
## Creating Voting Proposal
|
||||
|
||||
A `member` MAY initializes the voting with the proposal payload
|
||||
which is implemented using [protocol buffers v3](https://protobuf.dev/) as follows:
|
||||
|
||||
```protobuf
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
message Proposal {
|
||||
string name = 10; // Proposal name
|
||||
string payload = 11; // Describes the what is voting fore
|
||||
int32 proposal_id = 12; // Unique identifier of the proposal
|
||||
bytes proposal_owner = 13; // Public key of the creator
|
||||
repeated Vote votes = 14; // Vote list in the proposal
|
||||
int32 expected_voters_count = 15; // Maximum number of distinct voters
|
||||
int32 round = 16; // Number of Votes
|
||||
int64 timestamp = 17; // Creation time of proposal
|
||||
int64 expiration_time = 18; // Time interval that the proposal is active
|
||||
bool liveness_criteria_yes = 19; // Shows how managing the silent peers vote
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
message Vote {
|
||||
int32 vote_id = 20; // Unique identifier of the vote
|
||||
bytes vote_owner = 21; // Voter's public key
|
||||
int64 timestamp = 22; // Time when the vote was cast
|
||||
bool vote = 23; // Vote bool value (true/false)
|
||||
bytes parent_hash = 24; // Hash of previous owner's Vote
|
||||
bytes received_hash = 25; // Hash of previous received Vote
|
||||
bytes vote_hash = 26; // Hash of all previously defined fields in Vote
|
||||
bytes signature = 27; // Signature of vote_hash
|
||||
}
|
||||
```
|
||||
|
||||
The voting proposal MAY include adding a `node` or removing a `member`.
|
||||
After the `member` creates the voting proposal,
|
||||
it is emitted to the network via the MLS `Application message` with a lightweight,
|
||||
epoch based voting such as [hashgraphlike consensus.](https://github.com/vacp2p/rfc-index/blob/consensus-hashgraph-like/vac/raw/consensus-hashgraphlike.md)
|
||||
This consensus result MUST be finalized within the epoch as YES or NO.
|
||||
|
||||
If the voting result is YES, this points out the voting proposal will be converted into
|
||||
the MLS proposal by the `steward` and following commit message that starts the new epoch.
|
||||
|
||||
## Creating welcome message
|
||||
|
||||
When a MLS `MLS proposal message` is created by the `steward`,
|
||||
a `commit message` SHOULD follow,
|
||||
as in section 12.04 [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/) to the members.
|
||||
In order for the new `member` joining the group to synchronize with the current members
|
||||
who received the `commit message`,
|
||||
the `steward` sends a welcome message to the node as the new `member`,
|
||||
as in section 12.4.3.1. [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/).
|
||||
|
||||
## Single steward
|
||||
|
||||
To naive way to create a decentralized secure group messaging is having a single transparent `steward`
|
||||
who only applies the changes regarding the result of the voting.
|
||||
|
||||
This is mostly similar with the general flow and specified in voting proposal and welcome message creation sections.
|
||||
|
||||
1. Each time a single `steward` initializes a group with group parameters with parameters
|
||||
as in section 8.1. Group Context in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/).
|
||||
2. `steward` creates a group anouncement (GA) according to the previous step and
|
||||
broadcast it to the all network periodically. GA message is visible in network to all `nodes`.
|
||||
3. The each `node` who wants to be a member needs to obtain this anouncement and create `credential`
|
||||
includes `keyPackage` that is specified in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/) section 10.
|
||||
4. The `steward` aggregates all `KeyPackages` utilizes them to provision group additions for new members,
|
||||
based on the outcome of the voting process.
|
||||
5. Any `member` start to create `voting proposals` for adding or removing users,
|
||||
and present them to the voting in the MLS group as an application message.
|
||||
|
||||
However, unlimited use of `voting proposals` within the group may be misused by
|
||||
malicious or overly active members.
|
||||
Therefore, an application-level constraint can be introduced to limit the number
|
||||
or frequency of proposals initiated by each member to prevent spam or abuse.
|
||||
6. Meanwhile, the `steward` collects finalized `voting proposals` with in epoch `E`,
|
||||
that have received affirmative votes from members via application messages.
|
||||
Otherwise, the `steward` discards proposals that did not receive a majority of "YES" votes.
|
||||
Since voting proposals are transmitted as application messages, omitting them does not affect
|
||||
the protocol’s correctness or consistency.
|
||||
7. The `steward` converts all approved `voting proposals` into
|
||||
corresponding `MLS proposals` and `commit message`, and
|
||||
transmits both in a single operation as in [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/) section 12.4,
|
||||
including welcome messages for the new members. Therefore, the `commit message` ends the previous epoch and create new ones.
|
||||
|
||||
## Multi stewards
|
||||
|
||||
Decentralization has already been achieved in the previous section.
|
||||
However, to improve availability and ensure censorship resistance,
|
||||
the single-steward protocol is extended to a multi-steward architecture.
|
||||
In this design, each epoch is coordinated by a designated steward,
|
||||
operating under the same protocol as the single-steward model.
|
||||
Thus, the multi-steward approach primarily defines how steward roles
|
||||
rotate across epochs while preserving the underlying structure and logic of the original protocol.
|
||||
Two variants of the multi-steward design are introduced to address different system requirements.
|
||||
|
||||
### Multi steward with single consensus
|
||||
|
||||
In this model, all group modifications, such as adding or removing members,
|
||||
must be approved through consensus by all participants,
|
||||
including the steward assigned for epoch `E`.
|
||||
A configuration with multiple stewards operating under a shared consensus protocol offers
|
||||
increased decentralization and stronger protection against censorship.
|
||||
However, this benefit comes with reduced operational efficiency.
|
||||
The model is therefore best suited for small groups that value
|
||||
decentralization and censorship resistance more than performance.
|
||||
|
||||
### Multi steward with two consensuses
|
||||
|
||||
The two-consensus model offers improved efficiency with a trade-off in decentralization.
|
||||
In this design, group changes require consensus only among the stewards, rather than all members.
|
||||
Regular members participate by periodically selecting the stewards but do not take part in each decision.
|
||||
This structure enables faster coordination since consensus is achieved within a smaller group of stewards.
|
||||
It is particularly suitable for large user groups, where involving every member in each decision would be impractical.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/)
|
||||
|
||||
### References
|
||||
|
||||
- [MLS RFC 9420](https://datatracker.ietf.org/doc/rfc9420/)
|
||||
- [Hashgraphlike Consensus](https://github.com/vacp2p/rfc-index/blob/consensus-hashgraph-like/vac/raw/consensus-hashgraphlike.md)
|
||||
- [vacp2p/de-mls](https://github.com/vacp2p/de-mls)
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: ETH-DEMLS
|
||||
title: ETH-MLS-ONCHAIN
|
||||
name: Secure channel setup using decentralized MLS and Ethereum accounts
|
||||
status: raw
|
||||
category: Standards Track
|
||||
1501
vac/raw/mix.md
1501
vac/raw/mix.md
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: ETH-SECURE-CHANNEL
|
||||
title: NOISE-X3DH-DOUBLE-RATCHET
|
||||
name: Secure 1-to-1 channel setup using X3DH and the double ratchet
|
||||
status: raw
|
||||
category: Standards Track
|
||||
@@ -55,6 +55,9 @@ but improves scalability by reducing direct interactions between participants.
|
||||
Each message has a globally unique, immutable ID (or hash).
|
||||
Messages can be requested from the high-availability caches or
|
||||
other participants using the corresponding message ID.
|
||||
* **Participant ID:**
|
||||
Each participant has a globally unique, immutable ID
|
||||
visible to other participants in the communication.
|
||||
|
||||
## Wire protocol
|
||||
|
||||
@@ -69,18 +72,24 @@ Messages MUST adhere to the following meta structure:
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
message HistoryEntry {
|
||||
string message_id = 1; // Unique identifier of the SDS message, as defined in `Message`
|
||||
optional bytes retrieval_hint = 2; // Optional information to help remote parties retrieve this SDS message; For example, A Waku deterministic message hash or routing payload hash
|
||||
}
|
||||
|
||||
message Message {
|
||||
// 1 Reserved for sender/participant id
|
||||
string sender_id = 1; // Participant ID of the message sender
|
||||
string message_id = 2; // Unique identifier of the message
|
||||
string channel_id = 3; // Identifier of the channel to which the message belongs
|
||||
optional int32 lamport_timestamp = 10; // Logical timestamp for causal ordering in channel
|
||||
optional repeated string causal_history = 11; // List of preceding message IDs that this message causally depends on. Generally 2 or 3 message IDs are included.
|
||||
repeated HistoryEntry causal_history = 11; // List of preceding message IDs that this message causally depends on. Generally 2 or 3 message IDs are included.
|
||||
optional bytes bloom_filter = 12; // Bloom filter representing received message IDs in channel
|
||||
optional bytes content = 20; // Actual content of the message
|
||||
}
|
||||
```
|
||||
|
||||
Each message MUST include its globally unique identifier in the `message_id` field,
|
||||
The sending participant MUST include its own globally unique identifier in the `sender_id` field.
|
||||
In addition, it MUST include a globally unique identifier for the message in the `message_id` field,
|
||||
likely based on a message hash.
|
||||
The `channel_id` field MUST be set to the identifier of the channel of group communication
|
||||
that is being synchronized.
|
||||
@@ -93,6 +102,10 @@ These fields MAY be left unset in the case of [ephemeral messages](#ephemeral-me
|
||||
The message `content` MAY be left empty for [periodic sync messages](#periodic-sync-message),
|
||||
otherwise it MUST contain the application-level content
|
||||
|
||||
> **_Note:_** Close readers may notice that, outside of filtering messages originating from the sender itself,
|
||||
the `sender_id` field is not used for much.
|
||||
Its importance is expected to increase once a p2p retrieval mechanism is added to SDS, as is planned for the protocol.
|
||||
|
||||
### Participant state
|
||||
|
||||
Each participant MUST maintain:
|
||||
@@ -137,6 +150,10 @@ include this in the `lamport_timestamp` field.
|
||||
and include these in an ordered list in the `causal_history` field.
|
||||
The number of message IDs to include in the `causal_history` depends on the application.
|
||||
We recommend a causal history of two message IDs.
|
||||
* the participant MAY include a `retrieval_hint` in the `HistoryEntry`
|
||||
for each message ID in the `causal_history` field.
|
||||
This is an application-specific field to facilitate retrieval of messages,
|
||||
e.g. from high-availability caches.
|
||||
* the participant MUST include the current `bloom_filter`
|
||||
state in the broadcast message.
|
||||
|
||||
@@ -148,16 +165,19 @@ of unacknowledged outgoing messages.
|
||||
|
||||
Upon receiving a message,
|
||||
|
||||
* the participant SHOULD ignore the message if it has a `sender_id` matching its own.
|
||||
* the participant MAY deduplicate the message by comparing its `message_id` to previously received message IDs.
|
||||
* the participant MUST [review the ACK status](#review-ack-status) of messages
|
||||
in its unacknowledged outgoing buffer
|
||||
using the received message's causal history and bloom filter.
|
||||
* the participant MUST include the received message ID in its local bloom filter.
|
||||
* if the message has a populated `content` field,
|
||||
the participant MUST include the received message ID in its local bloom filter.
|
||||
* the participant MUST verify that all causal dependencies are met
|
||||
for the received message.
|
||||
Dependencies are met if the message IDs in the `causal_history` of the received message
|
||||
appear in the local history of the receiving participant.
|
||||
|
||||
If all dependencies are met,
|
||||
If all dependencies are met and the message has a populated `content` field,
|
||||
the participant MUST [deliver the message](#deliver-message).
|
||||
If dependencies are unmet,
|
||||
the participant MUST add the message to the incoming buffer of messages
|
||||
@@ -207,6 +227,7 @@ For each message in the incoming buffer:
|
||||
|
||||
* the participant MAY attempt to retrieve missing dependencies from the Store node
|
||||
(high-availability cache) or other peers.
|
||||
It MAY use the application-specific `retrieval_hint` in the `HistoryEntry` to facilitate retrieval.
|
||||
* if all dependencies of a message are met,
|
||||
the participant MUST proceed to [deliver the message](#deliver-message).
|
||||
|
||||
@@ -225,14 +246,26 @@ prioritizing **unacknowledged** messages.
|
||||
#### Periodic Sync Message
|
||||
|
||||
For each channel of communication,
|
||||
participants SHOULD periodically send an empty-content message to maintain sync state,
|
||||
without incrementing the Lamport timestamp.
|
||||
participants SHOULD periodically send sync messages to maintain state.
|
||||
These sync messages:
|
||||
|
||||
* MUST be sent with empty content
|
||||
* MUST include an incremented Lamport timestamp
|
||||
* MUST include causal history and bloom filter according to regular message rules
|
||||
* MUST NOT be added to the unacknowledged outgoing buffer
|
||||
* MUST NOT be included in causal histories of subsequent messages
|
||||
* MUST NOT be included in bloom filters
|
||||
* MUST NOT be added to the local log
|
||||
|
||||
Since sync messages are not persisted,
|
||||
they MAY have non-unique message IDs without impacting the protocol.
|
||||
To avoid network activity bursts in large groups,
|
||||
a participant MAY choose to only send periodic sync messages
|
||||
if no other messages have been broadcast in the channel after a random backoff period.
|
||||
|
||||
Participants MUST process these sync messages
|
||||
following the same steps as regular messages.
|
||||
Participants MUST process the causal history and bloom filter of these sync messages
|
||||
following the same steps as regular messages,
|
||||
but MUST NOT persist the sync messages themselves.
|
||||
|
||||
#### Ephemeral Messages
|
||||
|
||||
|
||||
395
vac/raw/staking-streamer.md
Normal file
395
vac/raw/staking-streamer.md
Normal file
@@ -0,0 +1,395 @@
|
||||
---
|
||||
title: SNT-STAKING-STREAMER
|
||||
name: Status Network Staking Streamer Protocol
|
||||
status: raw
|
||||
category: Standards Track
|
||||
tags: status-network
|
||||
editor:
|
||||
contributors:
|
||||
- Jimmy Debe <jimmy@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes the components used for the Status Network Staking Streamer protocol.
|
||||
The protocol is a set of smart contract protocols currently used in [Status Network](https://status.network/).
|
||||
|
||||
## Background/Motivation
|
||||
|
||||
Traditional layer 2 blockchains generate revenue from gass fees required to make transactions on the network.
|
||||
This can introduce a barrier of entry for users wanting to build and
|
||||
interact with that blockchain.
|
||||
The Status Network is a layer 2 roll up where applications and
|
||||
users do not blockchain with gasless transactions.
|
||||
Revenue is derived from a few sources including native yield commission.
|
||||
Assests bridged to the Status Network are rehypothecated into yield-bearing equivalents and
|
||||
users earn an [ERC-20 token](https://eips.ethereum.org/EIPS/eip-20) called Karma.
|
||||
Karma is an goverance token in the Status Network that gives the user access to a certain throughput of free transactions,
|
||||
votings and other network benefits.
|
||||
The SNT token can also be staked with the staking streamer protocol to earn Karma.
|
||||
|
||||
The goal is to have a staking mechanism that is fair to all participants based on the stake amount and time staked.
|
||||
Participants will have significant increases in voting power after committing their stake for a longer period,
|
||||
even if their stake is not the largest among all participants.
|
||||
|
||||
## Specification
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”,
|
||||
“SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
The staking streamer protocol is a set of smart contracts that implements the staking system.
|
||||
The protocol consists of the following components:
|
||||
|
||||
- accounts
|
||||
- staking operator
|
||||
- multipler points
|
||||
- reward mechanism
|
||||
- staking vaults
|
||||
|
||||
The staking system MUST be supported by an `OPERATOR` who will introduce reward amounts,
|
||||
reward periods, the token defined by its contract address REQUIRED for use in the system and
|
||||
making updates to the staking smart contracts.
|
||||
It is the responsibility of the `OPERATOR` to keep accounts informed about the requirements
|
||||
and intended changes to the staking system.
|
||||
|
||||
### Accounts
|
||||
|
||||
Accounts are users who contribute a token, ERC-20 standard, to the staking protocol.
|
||||
Accounts SHOULD interact with the protocol through a layer-1 blockchain smart contract.
|
||||
To participate,
|
||||
users MAY use an external owned account (EOA) and/or a smart account to interact with a `stakeVault` contract.
|
||||
A `stakeVault` is a smart contract that records and
|
||||
maintains the current amount of tokens transferred by accounts to the protocol.
|
||||
Each `stakeVault` MUST have one registered as the owner.
|
||||
Once registered, the user is considered as an account.
|
||||
Accounts MAY be the owner of one or more `stakeVault`s.
|
||||
|
||||
#### Account Staking Flow
|
||||
|
||||
When joining a staking system,
|
||||
accounts MUST use a `stakeVault` to transfer a token amount to the `stakingManager` contract,
|
||||
which is described [below](#operator-role).
|
||||
The account MUST use the same token, identitied by the token address,
|
||||
as registered in the `stakeVault` by the `OPERATOR`.
|
||||
Accounts MAY set a predefined duration to lock those tokens.
|
||||
Tokens will not be retrievable by accounts until the set duration expires.
|
||||
This is designed to improve the rewards accrued by accounts deciding to lock funds.
|
||||
|
||||
After tokens have successfully been transferred to the `stakeVault`,
|
||||
the account responsible will start to accumulate multiper points.
|
||||
The rewards and multipler points are described in the [Rewards section](#rewards) below.
|
||||
If staked tokens are not locked, the account MAY withdraw the full amount of tokens at any time.
|
||||
|
||||
### Operator Role
|
||||
|
||||
The `OPERATOR` is the owner of the staking system by registering an address on the `stakeManager` contract.
|
||||
The `OPERATOR` MUST deploy a `stakeManager` smart contract and
|
||||
facilitate the requirements of the system.
|
||||
Each account MUST use a `stakeVault` to interact with the `stakeManager`.
|
||||
This includes transferring to and from the staking system.
|
||||
|
||||
The operator MAY set a preferred minimum lock duration for all tokens entering the system.
|
||||
Accounts transaction MUST fail if the minimum lock duration is not set.
|
||||
After a minimum lock duration is set,
|
||||
the operator SHOULD NOT be able to update this amount until the set period is complete.
|
||||
|
||||
The `stakeManager` SHOULD be upgradable only by the registered operator.
|
||||
Contract upgrades will occur when an operator wants to add and/or
|
||||
change the requirements of the system.
|
||||
When accounts are aware of changes, either by the operator informing accounts or
|
||||
accounts reading the blockchain state,
|
||||
accounts MAY leave the system if they do not agree with the changes.
|
||||
Including when the operator enables `emergencyMode`.
|
||||
|
||||
- The operator decides the amount of rewards that should be awarded to stakers.
|
||||
- The operator MUST specify the amount of time for rewards to be distributed.
|
||||
|
||||
#### Emergency Mode
|
||||
|
||||
In some cases,
|
||||
an operator may choose to stop the functionality of the staking system.
|
||||
When `emergencyMode` is enabled, the following SHOULD apply:
|
||||
|
||||
- new vaults can not register,
|
||||
- current registered vaults can not stake or stake additional tokens with a lock period,
|
||||
- operators can not update the global state
|
||||
- accounts can not claim accrued rewards.
|
||||
|
||||
### Rewards
|
||||
|
||||
Accounts participate in the staking protocol to earn rewards.
|
||||
The staking system implements a rewards distribution mechanism,
|
||||
that works in conjunction with a internal accouting system called multiplier points.
|
||||
|
||||
#### Multiplier Points
|
||||
|
||||
When accounts have staked for a certain amount of time,
|
||||
multipler points (MP) are accrued.
|
||||
The accrued MP is counted toward the account's weight for reward distribution, as described [below](#account-weight).
|
||||
This ensures that participants are rewarded fairly based on the stake amount
|
||||
and time staked.
|
||||
The MP is accumulated over time and MUST not be transferable.
|
||||
The calculation is based on two factors, Initial MP and Accured MP.
|
||||
|
||||
##### Initial MP
|
||||
|
||||
Initial MP SHOULD be issued immediately when tokens are staked with or
|
||||
without a lock-up period.
|
||||
It is based on the stake amount and lock-up duration.
|
||||
The formula for Initial MP is derived as follows:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Initial} = \text{Stake} \times \left( 1 + \frac{\text{APY} \times T_ \text{lock}}{100 \times T_ \text{year}} \right)
|
||||
$$
|
||||
|
||||
Where:
|
||||
|
||||
- $Stake$ = The amount of tokens staked.
|
||||
- $APY$ = Annual Percentage Yield, set at 100%.
|
||||
- $T_{lock}$ = Lock-up duration in seconds.
|
||||
- $T_{year}$ = Total seconds in a year.
|
||||
|
||||
This formula calculates the Initial MP issued immediately when tokens are staked with a lock-up period.
|
||||
The longer the lock-up period, the more MP are issued.
|
||||
Accounts locking stake MAY earn bonus MP, which MAY be added to the initial MP amount.
|
||||
For example, Alice stakes 100 tokens with no lock-up time:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Initial} = 100 \times \left( 1 + \frac{100 \times 0}{100 \times 365} \right)
|
||||
$$
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Initial} = 100 \times \left( 1 + 0\right) = 100
|
||||
$$
|
||||
|
||||
Alice receives 100 MP.
|
||||
|
||||
Another example, Alice stakes 100 tokens with a 30 day lock-up period:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Initial} = 100 \times \left( 1 + \frac{100 \times 30}{100 \times 365} \right)
|
||||
$$
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Initial} = 100 \times \left( 1 + 0.082 \right) = 108.2
|
||||
$$
|
||||
|
||||
Alice receives 108.2 MP.
|
||||
By locking up the stake for 30 days,
|
||||
Alice receives an additional 8.2 MP.
|
||||
Alice cannot access the tokens until the lock-up period has passed.
|
||||
|
||||
##### Accrued MP
|
||||
|
||||
The Accrued MP SHOULD be accumulated over time as a function of the stake amount,
|
||||
elapsed time, and annual percentage yield (APY).
|
||||
The Accrued MP formula is derived as follows:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Accrued} = \text{Stake} \times \frac{\text{APY} \times T_ \text{elapsed}}{100 \times T_ \text{year}}
|
||||
$$
|
||||
|
||||
Where:
|
||||
|
||||
- $T_{elapsed}$: Time elapsed since staking began, measured in seconds.
|
||||
|
||||
This formula adds MP as a function of time,
|
||||
rewarding accounts with locked stake.
|
||||
The Accrued MP is calculated based on the stake amount.
|
||||
Already accrued MP SHOULD not affect the calculation of new accrued MP.
|
||||
For example, Alice stakes 100 tokens for 15 days:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Accrued} = 100 \times \frac{100 \times 15}{100 \times 365} = 4.1
|
||||
$$
|
||||
|
||||
Alice receives 4.1 MP for the 15 days of staked tokens.
|
||||
This is exactly half of the MP that would have been received if tokens were
|
||||
locked for 30 days.
|
||||
Another example, Alice stakes 100 tokens for 30 days:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Accrued} = 100 \times \frac{100 \times 30}{100 \times 365} = 8.2
|
||||
$$
|
||||
|
||||
Alice receives 8.2 MP for the 30 days she has staked.
|
||||
|
||||
The `stakingManager` contract SHOULD account for all MP accrued by all accounts in the system.
|
||||
Accounts are REQUIRED to make an on-chain transaction to claim MP.
|
||||
At any time, accounts MUST be able to retrieve the total MP accrued by the staking system by reading the contract state.
|
||||
Total MP combines both accrued MP and pending MP.
|
||||
The accrued MP contains the initial MP and the MP accrued over time.
|
||||
Pending MP are MP that have yet to be "claimed" by the accounts:
|
||||
|
||||
$$
|
||||
\text{MP}_ \text{Total} = \text{MP}_ \text{Accrued} + \text{MP}_ \text{Pending}
|
||||
$$
|
||||
|
||||
Accounts SHOULD be limited to a total maximum amount of MP that can accrue.
|
||||
This will prevent accounts from accumulating large amounts of MP over time compared to other accounts.
|
||||
The maximum amount of MP an account can accrue is capped at:
|
||||
|
||||
$$
|
||||
\text{MP}_\text{Maximum} = \text{MP}_ \text{Initial} + \text{MP}_ \text{Potential}
|
||||
$$
|
||||
|
||||
- $\text{MP}_ \text{Initial}$: The initial MP an account receives when staking, \*_including the bonus MP_.
|
||||
- $\text{MP}_ \text{Potential}$: The initial MP amount multiplied by a $MAX\_MULTIPLIER$ factor.
|
||||
- $MAX\_MULTIPLIER$: A constant that determines the multiplier for the maximum amount of MP in the system.
|
||||
|
||||
For example, assuming a $MAX\_MULTIPLIER$ of `4`, an account that stakes 100 tokens would have a maximum of:
|
||||
|
||||
$$
|
||||
\mathrm{MP}_{\text{Maximum}} = 100 + (100 \times 4) = 500
|
||||
$$
|
||||
|
||||
This means that the account can never have more than 500 MP,
|
||||
no matter how long they stake.
|
||||
|
||||
#### Reward Distribution
|
||||
|
||||
The system distributes rewards based on a few main factors:
|
||||
|
||||
- The amount of tokens staked
|
||||
- The account's Multiplier Points (MP)
|
||||
- the account's system weight
|
||||
- The account's reward index
|
||||
|
||||
#### Account Weight
|
||||
|
||||
Every account in the system has a weight associated with it.
|
||||
This weight is calculated as the sum of the staked tokens
|
||||
and the MP:
|
||||
|
||||
$$
|
||||
\text{Account Weight} = \text{Staked Balance} + \text{MP Balance}
|
||||
$$
|
||||
|
||||
In addition, the system MUST track the total weight of all accounts in the system:
|
||||
|
||||
$$
|
||||
\text{Total System Weight} = \sum_{i=1}^{n} \text{Account Weight}_i
|
||||
$$
|
||||
|
||||
The account's MP grows linearly with time,
|
||||
which means, account weights increase with time, which in turn means,
|
||||
the total weight of the system increases with time as well.
|
||||
With the weights mechanism implemented,
|
||||
the system can now calculate the rewards for each account.
|
||||
For an individual account,
|
||||
divide the account's weight by the total system weight and
|
||||
multiply it by the total reward amount:
|
||||
|
||||
$$
|
||||
\text{Reward Amount} = \text{Account Weight} \times \frac{\text{Total Reward Amount}}{\text{Total System Weight}}
|
||||
$$
|
||||
|
||||
The rewards are calculated in real-time,
|
||||
they don't actually need to be claimed by any account.
|
||||
However, whenever an account performs blockchain state change,
|
||||
such as staking or unstaking funds,
|
||||
the rewards that have been accrued up to this point are updated in the account's storage.
|
||||
|
||||
Any real-time rewards are considered as pending rewards,
|
||||
until the account interacts with the system.
|
||||
Pending rewards are calculated as:
|
||||
|
||||
$$
|
||||
\text{Pending Rewards} = \text{Account Weight} \times \left( \text{Current Reward Index} - \text{Account's Last Reward Index} \right)
|
||||
$$
|
||||
|
||||
Rewards are accrued by accounts over time and
|
||||
SHOULD be available for withdrawal at the end of each reward period.
|
||||
Based on the reward duration and amount of rewards,
|
||||
stakers will accrue rewards in real-time based on the rate of rewards which is calculated:
|
||||
|
||||
$$
|
||||
\text{Reward Rate} = \frac{\text{Total Reward Amount}}{\text{Reward Duration}}
|
||||
$$
|
||||
|
||||
- When the `rewardEndTime` period has occurred,
|
||||
the operator MUST set the new reward ratio.
|
||||
- account weights increase with
|
||||
time, which in turn means,
|
||||
the total weight of the system increases with time as well.
|
||||
|
||||
#### Reward Indices
|
||||
|
||||
The reward system uses an accounting mechanism based on indices to track and
|
||||
distribute rewards accurately.
|
||||
This approach ensures fair distribution even as accounts enter and
|
||||
exit the system or change their stakes over time.
|
||||
|
||||
##### Global Reward Index
|
||||
|
||||
The global reward index represents the cumulative rewards per unit of weight since the system's inception.
|
||||
It increases whenever new rewards are added to the system:
|
||||
|
||||
$$
|
||||
\text{New Index} = \text{Current Index} + \frac{\text{New Rewards} \times \text{Scale Factor}}{\text{Total System Weight}}
|
||||
$$
|
||||
|
||||
Where:
|
||||
|
||||
- `Scale Factor` is 1e18 (used for precision)
|
||||
- `Total System Weight` is the sum of all staked tokens and MP in the system
|
||||
|
||||
##### Account Reward Indices
|
||||
|
||||
Each account maintains its own reward index,
|
||||
which represents the point at which they last "claimed" rewards.
|
||||
The difference between the global index and
|
||||
an account's index is multiplied by the account's
|
||||
weight determines their unclaimed rewards:
|
||||
|
||||
$$
|
||||
\text{Unclaimed Rewards} = \text{Account Weight} \times (\text{Global Index} - \text{Account Index})
|
||||
$$
|
||||
|
||||
#### Reward Index Updates
|
||||
|
||||
The system MUST update indices in the following situations:
|
||||
|
||||
- When new rewards are added by the operator
|
||||
- When accounts stake or unstake tokens
|
||||
|
||||
This mechanism ensures that historical rewards are preserved accurately and
|
||||
accounts receive their fair share based on their weight over time.
|
||||
Also, accounts MUST NOT receive the rewards that were accounted for in the system
|
||||
at the time the account entered the system.
|
||||
|
||||
##### Index Adjustment Example
|
||||
|
||||
For example, the system has:
|
||||
|
||||
- Total Weight: 1000 units
|
||||
- Current Index: 0.5
|
||||
- New Rewards: 100 tokens
|
||||
|
||||
The index would increase by:
|
||||
|
||||
```text
|
||||
Increase = (100 × 1e18) / 1000 = 0.1e18
|
||||
New Index = 0.5e18 + 0.1e18 = 0.6e18
|
||||
```
|
||||
|
||||
If an account has:
|
||||
|
||||
- Weight: 200 units
|
||||
- Last Index: 0.5e18
|
||||
|
||||
Their unclaimed rewards would be:
|
||||
|
||||
```text
|
||||
Unclaimed = 200 × (0.6e18 - 0.5e18) / 1e18 = 20 tokens
|
||||
```
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [Status Network](https://status.network/)
|
||||
- [ERC-20 token](https://eips.ethereum.org/EIPS/eip-20)
|
||||
@@ -8,17 +8,18 @@ editor: Oskar Thoren <oskarth@titanproxy.com>
|
||||
contributors:
|
||||
- Hanno Cornelius <hanno@status.im>
|
||||
- Daniel Kaiser <danielkaiser@status.im>
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
This document outlines recommended usage of topic names in Waku v2.
|
||||
In [10/WAKU2 spec](../../standards/core/10/waku2.md) there are two types of topics:
|
||||
In [10/WAKU2 spec](/waku/standards/core/10/waku2.md) there are two types of topics:
|
||||
|
||||
- pubsub topics, used for routing
|
||||
- Pubsub topics, used for routing
|
||||
- Content topics, used for content-based filtering
|
||||
|
||||
## Pubsub Topics
|
||||
|
||||
Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](../../standards/core/11/relay.md)),
|
||||
Pubsub topics are used for routing of messages (see [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)),
|
||||
and can be named implicitly by Waku sharding (see [RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)).
|
||||
This document comprises recommendations for explicitly naming pubsub topics
|
||||
(e.g. when choosing *named sharding* as specified in [RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)).
|
||||
@@ -32,7 +33,7 @@ Pubsub topics SHOULD follow the following structure:
|
||||
This namespaced structure makes compatibility, discoverability,
|
||||
and automatic handling of new topics easier.
|
||||
|
||||
The first two parts indicate
|
||||
The first two parts indicate:
|
||||
|
||||
1) it relates to the Waku protocol domain, and
|
||||
2) the version is 2.
|
||||
@@ -42,15 +43,15 @@ in a hierarchical way as well.
|
||||
|
||||
> *Note*: In previous versions of this document, the structure was `/waku/2/{topic-name}/{encoding}`.
|
||||
The now deprecated `/{encoding}` was always set to `/proto`,
|
||||
which indicated that the [data field](../../standards/core/11/RELAY.md/#protobuf-definition)
|
||||
which indicated that the [data field](/waku/standards/core/11/relay.md#protobuf-definition)
|
||||
in pubsub is serialized/encoded as protobuf.
|
||||
The inspiration for this format was taken from
|
||||
[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
|
||||
However, because the payload of messages transmitted over [11/WAKU2-RELAY](../../standards/core/11/relay.md)
|
||||
must be a [14/WAKU2-MESSAGE](../../standards/core/14/message.md),
|
||||
However, because the payload of messages transmitted over [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
must be a [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md),
|
||||
which specifies the wire format as protobuf,`/proto` is the only valid encoding.
|
||||
This makes the `/proto` indication obsolete.
|
||||
The encoding of the `payload` field of a Waku Message
|
||||
The encoding of the `payload` field of a WakuMessage
|
||||
is indicated by the `/{encoding}` part of the content topic name.
|
||||
Specifying an encoding is only significant for the actual payload/data field.
|
||||
Waku preserves this option by allowing to specify an encoding
|
||||
@@ -94,18 +95,19 @@ This indicates explicitly that the network traffic has been partitioned into 10
|
||||
|
||||
The other type of topic that exists in Waku v2 is a content topic.
|
||||
This is used for content based filtering.
|
||||
See [14/WAKU2-MESSAGE spec](../../standards/core/14/message.md)
|
||||
See [14/WAKU2-MESSAGE spec](/waku/standards/core/14/message.md)
|
||||
for where this is specified.
|
||||
Note that this doesn't impact routing of messages between relaying nodes,
|
||||
but it does impact how request/reply protocols such as
|
||||
[12/WAKU2-FILTER](../../standards/core/12/filter.md) and
|
||||
[13/WAKU2-STORE](../../standards/core/13/store.md) are used.
|
||||
but it does impact using request/reply protocols such as
|
||||
[12/WAKU2-FILTER](/waku/standards/core/12/filter.md) and
|
||||
[13/WAKU2-STORE](/waku/standards/core/13/store.md).
|
||||
|
||||
This is especially useful for nodes that have limited bandwidth,
|
||||
and only want to pull down messages that match this given content topic.
|
||||
|
||||
Since all messages are relayed using the relay protocol regardless of content topic,
|
||||
you MAY use any content topic you wish without impacting how messages are relayed.
|
||||
you MAY use any content topic you wish
|
||||
without impacting how messages are relayed.
|
||||
|
||||
### Content Topic Format
|
||||
|
||||
@@ -121,28 +123,38 @@ As an example, here's the content topic used for an upcoming testnet:
|
||||
### Content Topic Naming Recommendations
|
||||
|
||||
Application names SHOULD be unique to avoid conflicting issues with other protocols.
|
||||
Applications SHOULD specify their version (if applicable) in the version field.
|
||||
Application version (if applicable) SHOULD be specified in the version field.
|
||||
The `{content-topic-name}` portion of the content topic is up to the application,
|
||||
and depends on the problem domain.
|
||||
It can be hierarchical, for instance to separate content, or
|
||||
to indicate different bandwidth and privacy guarantees.
|
||||
The encoding field indicates the serialization/encoding scheme
|
||||
for the [WakuMessage payload](../../standards/core/14/message.md/#payloads) field.
|
||||
for the [WakuMessage payload](/waku/standards/core/14/message.md#payloads) field.
|
||||
|
||||
### Content Topic usage guidelines
|
||||
|
||||
Applications SHOULD be mindful while designing/using content topics so that a bloat of content-topics does not happen.
|
||||
A content-topic bloat causes performance degradation in Store and Filter protocols while trying to retrieve messages.
|
||||
Applications SHOULD be mindful while designing/using content topics
|
||||
so that a bloat of content-topics does not happen.
|
||||
A content-topic bloat causes performance degradation in Store
|
||||
and Filter protocols while trying to retrieve messages.
|
||||
|
||||
Store queries have been noticed to be considerably slow (e.g doubling of response-time when content-topic count is increased from 10 to 100) when a lot of content-topics are involved in a single query.
|
||||
Similarly number of filter subscriptions increase, which increases complexity on client side to maintain and manage these subscriptions.
|
||||
Store queries have been noticed to be considerably slow
|
||||
(e.g doubling of response-time when content-topic count is increased from 10 to 100)
|
||||
when a lot of content-topics are involved in a single query.
|
||||
Similarly, a number of filter subscriptions increase,
|
||||
which increases complexity on client side to maintain
|
||||
and manage these subscriptions.
|
||||
|
||||
Applications SHOULD analyze the query/filter criteria for fetching messages from the network and select/design content topics to match such filter criteria.
|
||||
e.g: even though applications may want to segregate messages into different sets based on some application logic, if those sets of messages are always fetched/queried together from the network, then all those messages SHOULD use a single content-topic.
|
||||
Applications SHOULD analyze the query/filter criteria for fetching messages from the network
|
||||
and select/design content topics to match such filter criteria.
|
||||
e.g: even though applications may want to segregate messages into different sets
|
||||
based on some application logic,
|
||||
if those sets of messages are always fetched/queried together from the network,
|
||||
then all those messages SHOULD use a single content-topic.
|
||||
|
||||
## Differences with Waku v1
|
||||
|
||||
In [5/WAKU1](../../deprecated/5/waku0.md) there is no actual routing.
|
||||
In [5/WAKU1](/waku/deprecated/5/waku0.md) there is no actual routing.
|
||||
All messages are sent to all other nodes.
|
||||
This means that we are implicitly using the same pubsub topic
|
||||
that would be something like:
|
||||
@@ -155,7 +167,7 @@ Topics in Waku v1 correspond to Content Topics in Waku v2.
|
||||
|
||||
### Bridging Waku v1 and Waku v2
|
||||
|
||||
To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](../../standards/core/15/bridge.md).
|
||||
To bridge Waku v1 and Waku v2 we have a [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md).
|
||||
For mapping Waku v1 topics to Waku v2 content topics,
|
||||
the following structure for the content topic SHOULD be used:
|
||||
|
||||
@@ -166,9 +178,9 @@ the following structure for the content topic SHOULD be used:
|
||||
The `<4bytes-waku-v1-topic>` SHOULD be the lowercase hex representation
|
||||
of the 4-byte Waku v1 topic.
|
||||
A `0x` prefix SHOULD be used.
|
||||
`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](../../standards/application/26/payload.md).
|
||||
See [15/WAKU-BRIDGE](../../standards/core/15/bridge.md) for a description
|
||||
of the bridged fields.
|
||||
`/rfc26` indicates that the bridged content is encoded according to RFC [26/WAKU2-PAYLOAD](/waku/standards/application/26/payload.md).
|
||||
See [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md)
|
||||
for a description of the bridged fields.
|
||||
|
||||
This creates a direct mapping between the two protocols.
|
||||
For example:
|
||||
@@ -184,13 +196,13 @@ Copyright and related rights waived via
|
||||
|
||||
## References
|
||||
|
||||
- [10/WAKU2 spec](../../standards/core/10/waku2.md)
|
||||
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
|
||||
- [10/WAKU2 spec](/waku/standards/core/10/waku2.md)
|
||||
- [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
- [RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)
|
||||
- [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages)
|
||||
- [14/WAKU2-MESSAGE](../../standards/core/14/message.md)
|
||||
- [12/WAKU2-FILTER](../../standards/core/12/filter.md)
|
||||
- [13/WAKU2-STORE](../../standards/core/13/store.md)
|
||||
- [6/WAKU1](../../deprecated/5/waku0.md)
|
||||
- [15/WAKU-BRIDGE](../../standards/core/15/bridge.md)
|
||||
- [26/WAKU-PAYLOAD](../../standards/application/26/payload.md)
|
||||
- [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
- [12/WAKU2-FILTER](/waku/standards/core/12/filter.md)
|
||||
- [13/WAKU2-STORE](/waku/standards/core/13/store.md)
|
||||
- [6/WAKU1](/waku/deprecated/5/waku0.md)
|
||||
- [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md)
|
||||
- [26/WAKU-PAYLOAD](/waku/standards/application/26/payload.md)
|
||||
|
||||
@@ -5,6 +5,7 @@ name: Waku v2 Client Peer Management Recommendations
|
||||
status: draft
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
`27/WAKU2-PEERS` describes a recommended minimal set of peer storage and
|
||||
@@ -18,7 +19,8 @@ or the ability to store peer data on disk to resume state after a client restart
|
||||
Peer _management_ is a closely related concept and
|
||||
refers to the set of actions a client MAY choose to perform
|
||||
based on its knowledge of its connected peers,
|
||||
e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
|
||||
e.g. triggering reconnects/disconnects,
|
||||
keeping certain connections alive, etc.
|
||||
|
||||
## Peer store
|
||||
|
||||
@@ -55,7 +57,7 @@ disconnected gracefully.
|
||||
- **`Connected`**: The client is actively connected to this peer.
|
||||
|
||||
This list does not preclude clients from tracking more advanced connectivity metadata,
|
||||
such as a peer's blacklist status (see [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)).
|
||||
such as a peer's blacklist status (see [`18/WAKU2-SWAP`](/waku/deprecated/18/swap.md)).
|
||||
|
||||
### Persistence
|
||||
|
||||
@@ -91,8 +93,8 @@ This requires keeping track of the [last time each peer was disconnected](#track
|
||||
|
||||
A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers.
|
||||
If a client chooses to implement keep-alive on a connection,
|
||||
it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
|
||||
as per `10/WAKU2` [client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients).
|
||||
it SHOULD do so by sending periodic [libp2p pings](https://docs.libp2p.io/concepts/fundamentals/protocols/#ping)
|
||||
as per `10/WAKU2` [client recommendations](/waku/standards/core/10/waku2.md#recommendations-for-clients).
|
||||
The recommended period between pings SHOULD be _at most_ 50%
|
||||
of the shortest idle connection timeout for the specific client and transport.
|
||||
For example, idle TCP connections often times out after 10 to 15 minutes.
|
||||
@@ -111,9 +113,9 @@ Copyright and related rights waived via
|
||||
- [`Peer ID`](https://docs.libp2p.io/concepts/peer-id/)
|
||||
- [`multiaddrs`](https://docs.libp2p.io/concepts/addressing/)
|
||||
- [`protocol IDs`](https://docs.libp2p.io/concepts/protocols/#protocol-ids)
|
||||
- [`11/WAKU2-RELAY`](../../standards/core/11/relay.md)
|
||||
- [`13/WAKU2-STORE`](../../standards/core/13/store.md)
|
||||
- [`18/WAKU2-SWAP`](../../standards/application/18/swap.md)
|
||||
- [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
|
||||
- [libp2p pings](https://docs.libp2p.io/concepts/protocols/#ping)
|
||||
- [`10/WAKU2` client recommendations](../../standards/core/10/waku2.md/#recommendations-for-clients)
|
||||
- [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md)
|
||||
- [`13/WAKU2-STORE`](/waku/standards/core/13/store.md)
|
||||
- [`18/WAKU2-SWAP`](/waku/deprecated/18/swap.md)
|
||||
- [backing off period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md/#prune-backoff-and-peer-exchange)
|
||||
- [libp2p pings](https://docs.libp2p.io/concepts/fundamentals/protocols/#ping)
|
||||
- [`10/WAKU2` client recommendations](/waku/standards/core/10/waku2.md#recommendations-for-clients)
|
||||
|
||||
@@ -5,6 +5,7 @@ name: Waku v2 Client Parameter Configuration Recommendations
|
||||
status: draft
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
`29/WAKU2-CONFIG` describes the RECOMMENDED values
|
||||
@@ -12,7 +13,7 @@ to assign to configurable parameters for Waku v2 clients.
|
||||
Since Waku v2 is built on [libp2p](https://github.com/libp2p/specs),
|
||||
most of the parameters and reasonable defaults are derived from there.
|
||||
|
||||
Waku v2 relay messaging is specified in [`11/WAKU2-RELAY`](../../standards/core/11/relay.md),
|
||||
Waku v2 relay messaging is specified in [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md),
|
||||
a minor extension of the [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
|
||||
GossipSub behaviour is controlled by a series of adjustable parameters.
|
||||
Waku v2 clients SHOULD configure these parameters to the recommended values below.
|
||||
@@ -26,7 +27,7 @@ We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementation
|
||||
|----------------------|-------------------------------------------------------|-------------------|
|
||||
| `D` | The desired outbound degree of the network | 6 |
|
||||
| `D_low` | Lower bound for outbound degree | 4 |
|
||||
| `D_high` | Upper bound for outbound degree | 8 |
|
||||
| `D_high` | Upper bound for outbound degree | 8 |
|
||||
| `D_lazy` | (Optional) the outbound degree for gossip emission | `D` |
|
||||
| `heartbeat_interval` | Time between heartbeats | 1 second |
|
||||
| `fanout_ttl` | Time-to-live for each topic's fanout state | 60 seconds |
|
||||
@@ -36,9 +37,10 @@ We repeat them here with RECOMMMENDED values for `11/WAKU2-RELAY` implementation
|
||||
|
||||
## GossipSub v1.1 parameters
|
||||
|
||||
GossipSub v1.1 extended GossipSub v1.0 and introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
|
||||
We repeat the global parameters here with RECOMMMENDED values
|
||||
for `11/WAKU2-RELAY` implementations.
|
||||
GossipSub v1.1 extended GossipSub v1.0 and
|
||||
introduced [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters).
|
||||
We repeat the global parameters here
|
||||
with RECOMMMENDED values for `11/WAKU2-RELAY` implementations.
|
||||
|
||||
| Parameter | Description | RECOMMENDED value |
|
||||
|----------------|------------------------------------------------------------------------|-------------------|
|
||||
@@ -73,7 +75,7 @@ Copyright and related rights waived via
|
||||
## References
|
||||
|
||||
- [libp2p](https://github.com/libp2p/specs)
|
||||
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
|
||||
- [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
- [libp2p GossipSub protocol](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
|
||||
- [corresponding libp2p specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters)
|
||||
- [several new parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#overview-of-new-parameters)
|
||||
|
||||
@@ -5,6 +5,7 @@ name: Adaptive nodes
|
||||
status: draft
|
||||
editor: Oskar Thorén <oskarth@titanproxy.com>
|
||||
contributors:
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
This is an informational spec that show cases the concept of adaptive nodes.
|
||||
@@ -12,7 +13,8 @@ This is an informational spec that show cases the concept of adaptive nodes.
|
||||
## Node types - a continuum
|
||||
|
||||
We can look at node types as a continuum,
|
||||
from more restricted to less restricted, fewer resources to more resources.
|
||||
from more restricted to less restricted,
|
||||
fewer resources to more resources.
|
||||
|
||||

|
||||
|
||||
@@ -58,8 +60,8 @@ Each node can choose which protocols to support, depending on its resources and
|
||||
|
||||

|
||||
|
||||
In the case of protocols like [11/WAKU2-RELAY](../../standards/core/11/relay.md)
|
||||
etc (12, 13, 19, 21) these correspond to Libp2p protocols.
|
||||
Protocols like [11/WAKU2-RELAY](/waku/standards/core/11/relay.md),
|
||||
as well as [12], [13], [19], and [21], correspond to libp2p protocols.
|
||||
|
||||
However, other protocols like 16/WAKU2-RPC
|
||||
(local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY,
|
||||
@@ -77,13 +79,15 @@ We can better visualize the network with some illustrative examples.
|
||||
|
||||
### Topology and topics
|
||||
|
||||
The first one shows an example topology with different PubSub topics
|
||||
This illustration shows an example topology with different PubSub topics
|
||||
for the relay protocol.
|
||||
|
||||

|
||||
|
||||
### Legend
|
||||
|
||||
This illustration shows an example of content topics a node is interested in.
|
||||
|
||||

|
||||
|
||||
The dotted box shows what content topics (application-specific)
|
||||
@@ -99,7 +103,8 @@ a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
|
||||
This is a separate component with its own topology.
|
||||
|
||||
Behavior and interaction with other protocols specified in Vac RFCs,
|
||||
e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
|
||||
e.g. [25/LIBP2P-DNS-DISCOVERY](/vac/25/libp2p-dns-discovery.md)
|
||||
and [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md).
|
||||
|
||||
### Node Cross Section
|
||||
|
||||
@@ -110,8 +115,11 @@ shows how the connections look different for different protocols.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
- [11/WAKU2-RELAY](../../standards/core/11/relay.md)
|
||||
- [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
- [25/LIBP2P-DNS-DISCOVERY](/vac/25/libp2p-dns-discovery.md)
|
||||
- [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md)
|
||||
|
||||
@@ -10,29 +10,38 @@ contributors:
|
||||
|
||||
**Content Topics**:
|
||||
|
||||
- Public Key Broadcast: `/eth-pm/1/public-key/proto`,
|
||||
- Private Message: `/eth-pm/1/private-message/proto`.
|
||||
- Public Key Broadcast: `/eth-pm/1/public-key/proto`
|
||||
- Private Message: `/eth-pm/1/private-message/proto`
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification explains the Toy Ethereum Private Message protocol
|
||||
which enables a peer to send an encrypted message to another peer
|
||||
using the Waku v2 network, and the peer's Ethereum address.
|
||||
|
||||
The main purpose of this specification
|
||||
is to demonstrate how Waku v2 can be used for encrypted messaging purposes,
|
||||
using Ethereum accounts for identity.
|
||||
This protocol caters for Web3 wallets restrictions,
|
||||
allowing it to be implemented only using standard Web3 API.
|
||||
In the current state,
|
||||
the protocol has privacy and features [limitations](#limitations),
|
||||
has not been audited and hence is not fit for production usage.
|
||||
We hope this can be an inspiration for developers
|
||||
wishing to build on top of Waku v2.
|
||||
over the Waku network using the peer's Ethereum address.
|
||||
|
||||
## Goal
|
||||
|
||||
Alice wants to send an encrypted message to Bob, where only Bob can decrypt the message.
|
||||
Alice wants to send an encrypted message to Bob,
|
||||
where only Bob can decrypt the message.
|
||||
Alice only knows Bob's Ethereum Address.
|
||||
|
||||
The goal of this specification
|
||||
is to demonstrate how Waku can be used for encrypted messaging purposes,
|
||||
using Ethereum accounts for identity.
|
||||
This protocol caters to Web3 wallet restrictions,
|
||||
allowing it to be implemented using standard Web3 API.
|
||||
In the current state,
|
||||
Toy Ethereum Private Message, ETH-PM, has privacy and features [limitations](#limitations),
|
||||
has not been audited and hence is not fit for production usage.
|
||||
We hope this can be an inspiration for developers
|
||||
wishing to build on top of Waku.
|
||||
|
||||
## Design Requirements
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”,
|
||||
“SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
## Variables
|
||||
|
||||
Here are the variables used in the protocol and their definition:
|
||||
@@ -42,19 +51,17 @@ Here are the variables used in the protocol and their definition:
|
||||
- `B'` is Bob's Encryption Public Key, for which `b'` is the private key.
|
||||
- `M` is the private message that Alice sends to Bob.
|
||||
|
||||
## Design Requirements
|
||||
|
||||
The proposed protocol MUST adhere to the following design requirements:
|
||||
|
||||
1. Alice knows Bob's Ethereum address,
|
||||
2. Bob is willing to participate to Eth-PM, and publishes `B'`,
|
||||
3. Bob's ownership of `B'` MUST be verifiable,
|
||||
4. Alice wants to send message `M` to Bob,
|
||||
5. Bob SHOULD be able to get `M` using [10/WAKU2 spec](../../core/10/waku2.md),
|
||||
6. Participants only have access to their Ethereum Wallet via the Web3 API,
|
||||
7. Carole MUST NOT be able to read `M`'s content
|
||||
even if she is storing it or relaying it,
|
||||
8. [Waku Message Version 1](../26/payload.md) Asymmetric Encryption
|
||||
1. Alice knows Bob's Ethereum address
|
||||
2. Bob is willing to participate to Eth-PM, and publishes `B'`
|
||||
3. Bob's ownership of `B'` MUST be verifiable
|
||||
4. Alice wants to send message `M` to Bob
|
||||
5. Bob SHOULD be able to get `M` using [10/WAKU2](waku/standards/core/10/waku2.md)
|
||||
6. Participants only have access to their Ethereum Wallet via the Web3 API
|
||||
7. Carole MUST NOT be able to read `M`'s content,
|
||||
even if she is storing it or relaying it
|
||||
8. [Waku Message Version 1](waku/standards/application/26/payload.md) Asymmetric Encryption
|
||||
is used for encryption purposes.
|
||||
|
||||
## Limitations
|
||||
@@ -72,14 +79,14 @@ If Bob's private key is compromised, past and future messages could be decrypted
|
||||
A solution combining regular [X3DH](https://www.signal.org/docs/specifications/x3dh/)
|
||||
bundle broadcast with [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
|
||||
encryption would remove these limitations;
|
||||
See the [Status secure transport spec](https://specs.status.im/spec/5)
|
||||
See the [Status secure transport specification](status/deprecated/secure-transport.md)
|
||||
for an example of a protocol that achieves this in a peer-to-peer setting.
|
||||
|
||||
Bob MUST decide to participate in the protocol before Alice can send him a message.
|
||||
This is discussed in more in details in
|
||||
This is discussed in more detail in
|
||||
[Consideration for a non-interactive/uncoordinated protocol](#consideration-for-a-non-interactiveuncoordinated-protocol)
|
||||
|
||||
## The protocol
|
||||
## The Protocol
|
||||
|
||||
### Generate Encryption KeyPair
|
||||
|
||||
@@ -100,7 +107,7 @@ Bob MUST sign `B'` using `B`.
|
||||
|
||||
To prove ownership of the Encryption Public Key,
|
||||
Bob must sign it using [EIP-712](https://eips.ethereum.org/EIPS/eip-712) v3,
|
||||
meaning calling `eth_signTypedData_v3` on his Wallet's API.
|
||||
meaning calling `eth_signTypedData_v3` on his wallet's API.
|
||||
|
||||
Note: While v4 also exists, it is not available on all wallets and
|
||||
the features brought by v4 is not needed for the current use case.
|
||||
@@ -156,9 +163,9 @@ message PublicKeyMessage {
|
||||
}
|
||||
```
|
||||
|
||||
This MUST be wrapped in a Waku Message version 0,
|
||||
This MUST be wrapped in a [14/WAKU-MESSAGE](/waku/standards/core/14/message.md) version 0,
|
||||
with the Public Key Broadcast content topic.
|
||||
Finally, Bob SHOULD publish the message on Waku v2.
|
||||
Finally, Bob SHOULD publish the message on Waku.
|
||||
|
||||
## Consideration for a non-interactive/uncoordinated protocol
|
||||
|
||||
@@ -169,7 +176,7 @@ it is not enough in itself to deduce Bob's Public Key.
|
||||
This is why the protocol dictates that Bob MUST send his Public Key first,
|
||||
and Alice MUST receive it before she can send him a message.
|
||||
|
||||
Moreover, nim-waku, the reference implementation of [13/WAKU2-STORE](../../core/13/store.md),
|
||||
Moreover, nwaku, the reference implementation of [13/WAKU2-STORE](/waku/standards/core/13/store.md),
|
||||
stores messages for a maximum period of 30 days.
|
||||
This means that Bob would need to broadcast his public key
|
||||
at least every 30 days to be reachable.
|
||||
@@ -201,7 +208,7 @@ Note that these would resolve a UX issue
|
||||
only if a sender wants to proceed with _air drops_.
|
||||
|
||||
Indeed, if Bob does not publish his Public Key in the first place
|
||||
then it can be an indication that he simply does not participate in this protocol
|
||||
then it MAY be an indication that he does not participate in this protocol
|
||||
and hence will not receive messages.
|
||||
|
||||
However, these solutions would be helpful
|
||||
@@ -221,7 +228,7 @@ Another improvement would be for Bob not having to re-publish his public key
|
||||
every 30 days or less.
|
||||
Similarly to above,
|
||||
if Bob stops publishing his public key
|
||||
then it may be an indication that he does not participate in the protocol anymore.
|
||||
then it MAY be an indication that he does not participate in the protocol anymore.
|
||||
|
||||
In any case,
|
||||
the protocol could be modified to store the Public Key in a more permanent storage,
|
||||
@@ -229,19 +236,19 @@ such as a dedicated smart contract on the blockchain.
|
||||
|
||||
## Send Private Message
|
||||
|
||||
Alice MAY monitor the Waku v2 to collect Ethereum Address and
|
||||
Alice MAY monitor the Waku network to collect Ethereum Address and
|
||||
Encryption Public Key tuples.
|
||||
Alice SHOULD verify that the `signature`s of `PublicKeyMessage`s she receives
|
||||
are valid as per EIP-712.
|
||||
She SHOULD drop any message without a signature or with an invalid signature.
|
||||
|
||||
Using Bob's Encryption Public Key,
|
||||
retrieved via [10/WAKU2 spec](../../core/10/waku2.md),
|
||||
retrieved via [10/WAKU2](/waku/standards/core/10/waku2.md),
|
||||
Alice MAY now send an encrypted message to Bob.
|
||||
|
||||
If she wishes to do so,
|
||||
Alice MUST encrypt her message `M` using Bob's Encryption Public Key `B'`,
|
||||
as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](../26/payload.md/#asymmetric).
|
||||
as per [26/WAKU-PAYLOAD Asymmetric Encryption specs](waku/standards/application/26/payload.md/#asymmetric).
|
||||
|
||||
Alice SHOULD now publish this message on the Private Message content topic.
|
||||
|
||||
@@ -251,11 +258,11 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
## References
|
||||
|
||||
- [10/WAKU2 spec](../../core/10/waku2.md)
|
||||
- [Waku Message Version 1](../26/payload.md)
|
||||
- [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
- [Waku Message Version 1](waku/standards/application/26/payload.md)
|
||||
- [X3DH](https://www.signal.org/docs/specifications/x3dh/)
|
||||
- [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
|
||||
- [Status secure transport spec](https://specs.status.im/spec/5)
|
||||
- [Status secure transport specification](status/deprecated/secure-transport.md)
|
||||
- [EIP-712](https://eips.ethereum.org/EIPS/eip-712)
|
||||
- [13/WAKU2-STORE](../../core/13/store.md)
|
||||
- [13/WAKU2-STORE](/waku/standards/core/13/store.md)
|
||||
- [The Graph](https://thegraph.com/)
|
||||
|
||||
@@ -5,39 +5,65 @@ name: Waku Message Payload Encryption
|
||||
status: draft
|
||||
editor: Oskar Thoren <oskarth@titanproxy.com>
|
||||
contributors:
|
||||
- Oskar Thoren <oskarth@titanproxy.com>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes how Waku provides confidentiality, authenticity, and
|
||||
integrity, as well as some form of unlinkability.
|
||||
Specifically, it describes how encryption, decryption and
|
||||
signing works in [6/WAKU1](../../legacy/6/waku1.md) and
|
||||
in [10/WAKU2 spec](../../core/10/waku2.md) with [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1).
|
||||
signing works in [6/WAKU1](waku/standards/legacy/6/waku1.md) and
|
||||
in [10/WAKU2](waku/standards/core/10/waku2.md) with [14/WAKU-MESSAGE](waku/standards/core/14/message.md/#version1).
|
||||
|
||||
This specification effectively replaces [7/WAKU-DATA](../../legacy/7/data.md)
|
||||
as well as [6/WAKU1 Payload encryption](../../legacy/6/waku1.md/#payload-encryption)
|
||||
but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
|
||||
This specification effectively replaces [7/WAKU-DATA](waku/standards/legacy/7/data.md)
|
||||
as well as [6/WAKU1 Payload encryption](waku/standards/legacy/6/waku1.md/#payload-encryption)
|
||||
but written in a way that is agnostic and self-contained for [6/WAKU1](waku/standards/legacy/6/waku1.md) and [10/WAKU2](waku/standards/core/10/waku2.md).
|
||||
|
||||
Large sections of the specification originate from
|
||||
[EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627) as well from
|
||||
[RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
|
||||
with some modifications.
|
||||
|
||||
## Specification
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
For [6/WAKU1](waku/standards/legacy/6/waku1.md),
|
||||
the `data` field is used in the [waku envelope](waku/standards/legacy/6/waku1.md#abnf-specification)
|
||||
and the field MAY contain the encrypted payload.
|
||||
|
||||
For [10/WAKU2](waku/standards/core/10/waku2.md),
|
||||
the `payload` field is used in `WakuMessage`
|
||||
and MAY contain the encrypted payload.
|
||||
|
||||
The fields that are concatenated and
|
||||
encrypted as part of the `data` (Waku legacy) or
|
||||
`payload` (Waku) field are:
|
||||
|
||||
- `flags`
|
||||
- `payload-length`
|
||||
- `payload`
|
||||
- `padding`
|
||||
- `signature`
|
||||
|
||||
## Design requirements
|
||||
|
||||
- *Confidentiality*:
|
||||
The adversary should not be able to learn what data is being sent from one Waku node
|
||||
The adversary SHOULD NOT be able to learn what data is being sent from one Waku node
|
||||
to one or several other Waku nodes.
|
||||
- *Authenticity*:
|
||||
The adversary should not be able to cause Waku endpoint
|
||||
The adversary SHOULD NOT be able to cause Waku endpoint
|
||||
to accept data from any third party as though it came from the other endpoint.
|
||||
- *Integrity*:
|
||||
The adversary should not be able to cause a Waku endpoint to
|
||||
The adversary SHOULD NOT be able to cause a Waku endpoint to
|
||||
accept data that has been tampered with.
|
||||
|
||||
Notable, *forward secrecy* is not provided for at this layer.
|
||||
If this property is desired,
|
||||
a more fully featured secure communication protocol can be used on top,
|
||||
such as [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
|
||||
a more fully featured secure communication protocol can be used on top.
|
||||
|
||||
It also provides some form of *unlinkability* since:
|
||||
|
||||
@@ -58,25 +84,6 @@ ECIES is using the following cryptosystem:
|
||||
- MAC: HMAC with SHA-256
|
||||
- AES: AES-128-CTR
|
||||
|
||||
## Specification
|
||||
|
||||
For [6/WAKU1](../../legacy/6/waku1.md),
|
||||
the `data` field is used in the `waku envelope`,
|
||||
and the field MAY contain the encrypted payload.
|
||||
|
||||
For [10/WAKU2 spec](../../core/10/waku2.md),
|
||||
the `payload` field is used in `WakuMessage` and
|
||||
MAY contain the encrypted payload.
|
||||
|
||||
The fields that are concatenated and
|
||||
encrypted as part of the `data` (Waku v1) / `payload` (Waku v2) field are:
|
||||
|
||||
- flags
|
||||
- payload-length
|
||||
- payload
|
||||
- padding
|
||||
- signature
|
||||
|
||||
### ABNF
|
||||
|
||||
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
|
||||
@@ -101,7 +108,7 @@ signature = 65OCTET
|
||||
|
||||
data = flags payload-length payload padding [signature]
|
||||
|
||||
; This field is called payload in Waku v2
|
||||
; This field is called payload in Waku
|
||||
payload = data
|
||||
```
|
||||
|
||||
@@ -109,7 +116,7 @@ payload = data
|
||||
|
||||
Those unable to decrypt the payload/data are also unable to access the signature.
|
||||
The signature, if provided,
|
||||
is the ECDSA signature of the Keccak-256 hash of the unencrypted data
|
||||
SHOULD be the ECDSA signature of the Keccak-256 hash of the unencrypted data
|
||||
using the secret key of the originator identity.
|
||||
The signature is serialized as the concatenation of the `r`, `s` and `v` parameters
|
||||
of the SECP-256k1 ECDSA signature, in that order.
|
||||
@@ -129,7 +136,7 @@ Symmetric encryption uses AES-256-GCM for
|
||||
The output of encryption is of the form (`ciphertext`, `tag`, `iv`)
|
||||
where `ciphertext` is the encrypted message,
|
||||
`tag` is a 16 byte message authentication tag and
|
||||
`iv` is a 12 byte initialization vector (nonce).
|
||||
`iv` is a 12 byte initialization vector (nonce).
|
||||
The message authentication `tag` and
|
||||
initialization vector `iv` field MUST be appended to the resulting `ciphertext`,
|
||||
in that order.
|
||||
@@ -144,7 +151,7 @@ Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Sch
|
||||
#### ECIES
|
||||
|
||||
This section originates from the [RLPx Transport Protocol spec](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
|
||||
spec with minor modifications.
|
||||
specification with minor modifications.
|
||||
|
||||
The cryptosystem used is:
|
||||
|
||||
@@ -177,11 +184,11 @@ then obtains the plaintext as `m = AES(kE, iv || c)`.
|
||||
|
||||
### Padding
|
||||
|
||||
The padding field is used to align data size,
|
||||
The `padding` field is used to align data size,
|
||||
since data size alone might reveal important metainformation.
|
||||
Padding can be arbitrary size.
|
||||
However, it is recommended that the size of Data Field
|
||||
(excluding the IV and tag) before encryption (i.e. plain text)
|
||||
However, it is recommended that the size of `data` field
|
||||
(excluding the `iv` and `tag`) before encryption (i.e. plain text)
|
||||
SHOULD be a multiple of 256 bytes.
|
||||
|
||||
### Decoding a message
|
||||
@@ -196,13 +203,13 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
## References
|
||||
|
||||
1. [6/WAKU1](../../legacy/6/waku1.md)
|
||||
2. [10/WAKU2 spec](../../core/10/waku2.md)
|
||||
3. [14/WAKU-MESSAGE version 1](../../core/14/message.md/#version1)
|
||||
4. [7/WAKU-DATA](../../legacy/7/data.md)
|
||||
1. [6/WAKU1](waku/standards/legacy/6/waku1.md)
|
||||
2. [10/WAKU2 spec](waku/standards/core/10/waku2.md)
|
||||
3. [14/WAKU-MESSAGE version 1](waku/standards/core/14/message.md/#version1)
|
||||
4. [7/WAKU-DATA](waku/standards/legacy/7/data.md)
|
||||
5. [EIP-627: Whisper spec](https://eips.ethereum.org/EIPS/eip-627)
|
||||
6. [RLPx Transport Protocol spec (ECIES encryption)](https://github.com/ethereum/devp2p/blob/master/rlpx.md#ecies-encryption)
|
||||
7. [Status 5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
|
||||
7. [Status 5/SECURE-TRANSPORT](status/deprecated/secure-transport.md)
|
||||
8. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
|
||||
9. [Ethereum "Yellow paper": Appendix F Signing transactions](https://ethereum.github.io/yellowpaper/paper.pdf)
|
||||
10. [authenticated encryption](https://en.wikipedia.org/wiki/Authenticated_encryption)
|
||||
|
||||
@@ -7,11 +7,12 @@ category: Standards Track
|
||||
tags: waku-application
|
||||
editor: Aaryamann Challani <p1ge0nh8er@proton.me>
|
||||
contributors:
|
||||
- Andrea Piana <andreap@status.im>
|
||||
- Pedro Pombeiro <pedro@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Andrea Piana <andreap@status.im>
|
||||
- Pedro Pombeiro <pedro@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
@@ -38,7 +39,7 @@ without other nodes network being able to read their messages.
|
||||
which provide assurances that session keys will not be compromised
|
||||
even if the private keys of the participants are compromised.
|
||||
Specifically, past messages cannot be decrypted by a third-party
|
||||
who manages to get a hold of a private key.
|
||||
who manages to obtain those private key.
|
||||
|
||||
- **Secret channel** describes a communication channel
|
||||
where a Double Ratchet algorithm is in use.
|
||||
@@ -73,7 +74,7 @@ The main cryptographic protocol is a Double Ratchet protocol,
|
||||
which is derived from the
|
||||
[Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html),
|
||||
using a different ratchet.
|
||||
[The Waku v2 protocol](../../core/10/waku2.md)
|
||||
[The Waku v2 protocol](/waku/standards/core/10/waku2.md)
|
||||
subsequently encrypts the message payload, using symmetric key encryption.
|
||||
Furthermore, the concept of prekeys
|
||||
(through the use of [X3DH](https://signal.org/docs/specifications/x3dh/))
|
||||
@@ -234,38 +235,41 @@ Where:
|
||||
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L47))
|
||||
|
||||
```protobuf
|
||||
message X3DHHeader {
|
||||
// Alice's ephemeral key `EK_A`
|
||||
bytes key = 1;
|
||||
// Bob's bundle signed prekey
|
||||
bytes id = 4;
|
||||
}
|
||||
message X3DHHeader {
|
||||
// Alice's ephemeral key `EK_A`
|
||||
bytes key = 1;
|
||||
// Bob's bundle signed prekey
|
||||
bytes id = 4;
|
||||
}
|
||||
```
|
||||
|
||||
- `DR_header`: Double ratchet header ([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L31)).
|
||||
Used when Bob's public bundle is available:
|
||||
|
||||
``` protobuf
|
||||
message DRHeader {
|
||||
// Alice's current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet))
|
||||
bytes key = 1;
|
||||
// number of the message in the sending chain
|
||||
uint32 n = 2;
|
||||
// length of the previous sending chain
|
||||
uint32 pn = 3;
|
||||
// Bob's bundle ID
|
||||
bytes id = 4;
|
||||
}
|
||||
message DRHeader {
|
||||
// Alice's current ratchet public key
|
||||
bytes key = 1;
|
||||
// number of the message in the sending chain
|
||||
uint32 n = 2;
|
||||
// length of the previous sending chain
|
||||
uint32 pn = 3;
|
||||
// Bob's bundle ID
|
||||
bytes id = 4;
|
||||
}
|
||||
```
|
||||
|
||||
Alice's current ratchet public key (above) is mentioned in
|
||||
[DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)
|
||||
|
||||
- `DH_header`: Diffie-Hellman header (used when Bob's bundle is not available):
|
||||
([reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L42))
|
||||
|
||||
``` protobuf
|
||||
message DHHeader {
|
||||
// Alice's compressed ephemeral public key.
|
||||
bytes key = 1;
|
||||
}
|
||||
message DHHeader {
|
||||
// Alice's compressed ephemeral public key.
|
||||
bytes key = 1;
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Chain key update
|
||||
@@ -286,7 +290,7 @@ The message key MUST be used to encrypt the next message to be sent.
|
||||
1. Inherits the security considerations of [X3DH](https://signal.org/docs/specifications/x3dh/#security-considerations)
|
||||
and [Double Ratchet](https://signal.org/docs/specifications/doubleratchet/#security-considerations).
|
||||
|
||||
2. Inherits the security considerations of the [Waku v2 protocol](../../core/10/waku2.md).
|
||||
2. Inherits the security considerations of the [Waku v2 protocol](/waku/standards/core/10/waku2.md).
|
||||
|
||||
3. The protocol is designed to be used in a decentralized manner, however,
|
||||
it is possible to use a centralized server to serve prekey bundles.
|
||||
@@ -299,7 +303,8 @@ It is possible to link messages signed by the same keypair.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
@@ -308,7 +313,7 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
- [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
|
||||
- [Protobuf](https://developers.google.com/protocol-buffers/)
|
||||
- [Off-the-Record protocol](https://otr.cypherpunks.ca/Protocol-v3-4.1.1.html)
|
||||
- [The Waku v2 protocol](../../core/10/waku2.md)
|
||||
- [The Waku v2 protocol](/waku/standards/core/10/waku2.md)
|
||||
- [HKDF](https://www.rfc-editor.org/rfc/rfc5869)
|
||||
- [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles)
|
||||
- [reference wire format](https://github.com/status-im/status-go/blob/a904d9325e76f18f54d59efc099b63293d3dcad3/services/shhext/chat/encryption.proto#L12)
|
||||
|
||||
@@ -7,11 +7,12 @@ category: Standards Track
|
||||
tags: waku-application
|
||||
editor: Aaryamann Challani <p1ge0nh8er@proton.me>
|
||||
contributors:
|
||||
- Andrea Piana <andreap@status.im>
|
||||
- Pedro Pombeiro <pedro@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Andrea Piana <andreap@status.im>
|
||||
- Pedro Pombeiro <pedro@status.im>
|
||||
- Corey Petty <corey@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
@@ -20,7 +21,7 @@ This document specifies how to manage sessions based on an X3DH key exchange.
|
||||
This includes how to establish new sessions,
|
||||
how to re-establish them, how to maintain them, and how to close them.
|
||||
|
||||
[53/WAKU2-X3DH](../53/x3dh.md) specifies the Waku `X3DH` protocol
|
||||
[53/WAKU2-X3DH](/waku/standards/application/53/x3dh.md) specifies the Waku `X3DH` protocol
|
||||
for end-to-end encryption.
|
||||
Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
|
||||
|
||||
@@ -80,12 +81,11 @@ of the shared secret encoded in hexadecimal format.
|
||||
|
||||
```js
|
||||
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
|
||||
ecies.ImportECDSAPublic(theirPublicKey),
|
||||
16,
|
||||
16,
|
||||
ecies.ImportECDSAPublic(theirPublicKey),
|
||||
16,
|
||||
16,
|
||||
)
|
||||
|
||||
|
||||
hexEncodedKey := hex.EncodeToString(sharedKey)
|
||||
|
||||
var hash []byte = keccak256(hexEncodedKey)
|
||||
@@ -183,7 +183,7 @@ which will notify the receiving end not to include the device in any further com
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. Inherits all security considerations from [53/WAKU2-X3DH](../53/x3dh.md).
|
||||
1. Inherits all security considerations from [53/WAKU2-X3DH](/waku/standards/application/53/x3dh.md).
|
||||
|
||||
### Recommendations
|
||||
|
||||
@@ -194,9 +194,10 @@ which will notify the receiving end not to include the device in any further com
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
1. [53/WAKU2-X3DH](../53/x3dh.md)
|
||||
2. [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)
|
||||
- [53/WAKU2-X3DH](/waku/standards/application/53/x3dh.md)
|
||||
- [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/)
|
||||
|
||||
@@ -14,9 +14,9 @@ contributors:
|
||||
|
||||
## Abstract
|
||||
|
||||
Waku v2 is family of modular peer-to-peer protocols for secure communication.
|
||||
Waku is a family of modular peer-to-peer protocols for secure communication.
|
||||
The protocols are designed to be secure, privacy-preserving, censorship-resistant
|
||||
and being able to run in resource restricted environments.
|
||||
and being able to run in resource-restricted environments.
|
||||
At a high level, it implements Pub/Sub over [libp2p](https://github.com/libp2p/specs)
|
||||
and adds a set of capabilities to it.
|
||||
These capabilities are things such as:
|
||||
@@ -24,27 +24,27 @@ These capabilities are things such as:
|
||||
(ii) adaptive nodes, allowing for heterogeneous nodes to contribute to the network
|
||||
(iii) preserving bandwidth usage for resource-restriced devices
|
||||
|
||||
This makes Waku ideal for running a p2p protocol on mobile and
|
||||
in similarly restricted environments.
|
||||
This makes Waku ideal for running a p2p protocol on mobile devices and
|
||||
other similar restricted environments.
|
||||
|
||||
Historically, it has its roots in [6/WAKU1](../../legacy/6/waku1.md),
|
||||
Historically, it has its roots in [6/WAKU1](/waku/standards/legacy/6/waku1.md),
|
||||
which stems from [Whisper](https://eips.ethereum.org/EIPS/eip-627),
|
||||
originally part of the Ethereum stack.
|
||||
However, Waku v2 acts more as a thin wrapper for PubSub and has a different API.
|
||||
However, Waku acts more as a thin wrapper for Pub/Sub and has a different API.
|
||||
It is implemented in an iterative manner where initial focus
|
||||
is on porting essential functionality to libp2p.
|
||||
See [rough road map (2020)](https://vac.dev/waku-v2-plan) for more historical context.
|
||||
|
||||
## Motivation and goals
|
||||
## Motivation and Goals
|
||||
|
||||
Waku as a family of protocols is designed to have a set of properties
|
||||
Waku, as a family of protocols, is designed to have a set of properties
|
||||
that are useful for many applications:
|
||||
|
||||
1.**Useful for generalized messaging.**
|
||||
|
||||
Many applications require some form of messaging protocol to communicate
|
||||
between different subsystems or different nodes.
|
||||
This messaging can be human-to-human or machine-to-machine or a mix.
|
||||
This messaging can be human-to-human, machine-to-machine or a mix.
|
||||
Waku is designed to work for all these scenarios.
|
||||
|
||||
2.**Peer-to-peer.**
|
||||
@@ -62,7 +62,7 @@ Applications often run in restricted environments,
|
||||
where resources or the environment is restricted in some fashion.
|
||||
For example:
|
||||
|
||||
- Limited bandwidth, CPU, memory, disk, battery, etc
|
||||
- Limited bandwidth, CPU, memory, disk, battery, etc.
|
||||
- Not being publicly connectable
|
||||
- Only being intermittently connected; mostly-offline
|
||||
|
||||
@@ -72,7 +72,7 @@ For example:
|
||||
|
||||
- Pseudonymity and not being tied to any personally identifiable information (PII)
|
||||
- Metadata protection in transit
|
||||
- Various forms of unlinkability, etc
|
||||
- Various forms of unlinkability, etc.
|
||||
|
||||
5.**Modular design.**
|
||||
|
||||
@@ -89,25 +89,31 @@ For example:
|
||||
- Stronger guarantees for spam protection vs economic registration cost
|
||||
|
||||
For more on the concept of adaptive nodes and what this means in practice,
|
||||
please see the [30/ADAPTIVE-NODES](../../../informational/30/adaptive-nodes.md) spec.
|
||||
please see the [30/ADAPTIVE-NODES](/waku/informational/30/adaptive-nodes.md) spec.
|
||||
|
||||
## Network interaction domains
|
||||
## Specification
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
### Network Interaction Domains
|
||||
|
||||
While Waku is best thought of as a single cohesive thing,
|
||||
there are three network interaction domains:
|
||||
|
||||
(a) gossip domain
|
||||
(b) discovery domain
|
||||
(c) req/resp domain
|
||||
(c) request/response domain
|
||||
|
||||
### Protocols and identifiers
|
||||
#### Protocols and Identifiers
|
||||
|
||||
Since Waku v2 is built on top of libp2p, many protocols have a libp2p protocol identifier.
|
||||
Since Waku is built on top of libp2p, many protocols have a libp2p protocol identifier.
|
||||
The current main [protocol identifiers](https://docs.libp2p.io/concepts/protocols/)
|
||||
are:
|
||||
|
||||
1. `/vac/waku/relay/2.0.0`
|
||||
2. `/vac/waku/store/2.0.0-beta4`
|
||||
2. `/vac/waku/store-query/3.0.0`
|
||||
3. `/vac/waku/filter/2.0.0-beta1`
|
||||
4. `/vac/waku/lightpush/2.0.0-beta1`
|
||||
|
||||
@@ -117,19 +123,19 @@ Since these aren't negotiated libp2p protocols,
|
||||
they are referred to by their RFC ID.
|
||||
For example:
|
||||
|
||||
- [14/WAKU2-MESSAGE](../14/message.md) and
|
||||
[26/WAKU-PAYLOAD](../../application/26/payload.md) for message payloads
|
||||
- [23/WAKU2-TOPICS](../../../informational/23/topics.md) and
|
||||
[27/WAKU2-PEERS](../../../informational/27/peers.md) for recommendations around usage
|
||||
- [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) and
|
||||
[26/WAKU-PAYLOAD](/waku/standards/application/26/payload.md) for message payloads
|
||||
- [23/WAKU2-TOPICS](/waku/informational/23/topics.md) and
|
||||
[27/WAKU2-PEERS](/waku/informational/27/peers.md) for recommendations around usage
|
||||
|
||||
There are also more experimental libp2p protocols such as:
|
||||
|
||||
1. `/vac/waku/swap/2.0.0-beta1`
|
||||
2. `/vac/waku/waku-rln-relay/2.0.0-alpha1`
|
||||
1. `/vac/waku/waku-rln-relay/2.0.0-alpha1`
|
||||
2. `/vac/waku/peer-exchange/2.0.0-alpha1`
|
||||
|
||||
These protocols and their semantics are elaborated on in their own specs.
|
||||
The semantics of these protocols are referred to by RFC ID [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md) and [34/WAKU2-PEER-EXCHANGE](/waku/standards/core/34/peer-exchange.md).
|
||||
|
||||
### Use of libp2p and protobuf
|
||||
#### Use of libp2p and Protobuf
|
||||
|
||||
Unless otherwise specified,
|
||||
all protocols are implemented over libp2p and use Protobuf by default.
|
||||
@@ -139,23 +145,24 @@ libp2p protocols prefix binary message payloads with
|
||||
the length of the message in bytes.
|
||||
This length integer is encoded as a [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints).
|
||||
|
||||
### Gossip domain
|
||||
#### Gossip Domain
|
||||
|
||||
Waku is using gossiping to disseminate messages throughout the network.
|
||||
|
||||
**Protocol identifier**: `/vac/waku/relay/2.0.0`
|
||||
|
||||
See [11/WAKU2-RELAY](../11/relay.md) spec for more details.
|
||||
See [11/WAKU2-RELAY](/waku/standards/core/11/relay.md) specification for more details.
|
||||
|
||||
For an experimental privacy-preserving economic spam protection mechanism, see [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
|
||||
For an experimental privacy-preserving economic spam protection mechanism,
|
||||
see [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md).
|
||||
|
||||
See [23/WAKU2-TOPICS](../../../informational/23/topics.md)
|
||||
for more information about recommended topic usage.
|
||||
See [23/WAKU2-TOPICS](/waku/informational/23/topics.md)
|
||||
for more information about the recommended topic usage.
|
||||
|
||||
### Direct use of libp2p protocols
|
||||
#### Direct use of libp2p protocols
|
||||
|
||||
In addition to `/vac/waku/*` protocols,
|
||||
Waku v2 MAY directly use the following libp2p protocols:
|
||||
Waku MAY directly use the following libp2p protocols:
|
||||
|
||||
- [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping)
|
||||
with protocol id
|
||||
@@ -164,14 +171,14 @@ with protocol id
|
||||
/ipfs/ping/1.0.0
|
||||
```
|
||||
|
||||
for liveness checks between peers, or to keep peer-to-peer connections alive.
|
||||
for liveness checks between peers, or
|
||||
to keep peer-to-peer connections alive.
|
||||
|
||||
- [libp2p identity and identity/push](https://docs.libp2p.io/concepts/protocols/#identify)
|
||||
with protocol IDs
|
||||
|
||||
```text
|
||||
/ipfs/id/1.0.0
|
||||
|
||||
```
|
||||
|
||||
and
|
||||
@@ -182,167 +189,166 @@ and
|
||||
|
||||
respectively, as basic means for capability discovery.
|
||||
These protocols are anyway used by the libp2p connection
|
||||
establishment layer Waku v2 is built on.
|
||||
establishment layer Waku is built on.
|
||||
We plan to introduce a new Vac capability discovery protocol
|
||||
with better anonymity properties and more functionality.
|
||||
|
||||
#### Transports
|
||||
|
||||
Waku v2 is built in top of libp2p, and like libp2p it strives to be transport agnostic.
|
||||
Waku is built in top of libp2p, and like libp2p it strives to be transport agnostic.
|
||||
We define a set of recommended transports in order to achieve a baseline of
|
||||
interoperability between clients.
|
||||
|
||||
This section describes these recommended transports.
|
||||
|
||||
Waku client implementations SHOULD support the TCP transport.
|
||||
|
||||
Where TCP is supported it MUST be enabled for both dialing and listening,
|
||||
even if other transports are available.
|
||||
|
||||
Waku v2 nodes where the environment do not allow to use TCP directly,
|
||||
Waku nodes running in environments that do not allow the use of TCP directly,
|
||||
MAY use other transports.
|
||||
|
||||
A Waku v2 node SHOULD support secure websockets for bidirectional communication streams,
|
||||
A Waku node SHOULD support secure websockets for bidirectional communication streams,
|
||||
for example in a web browser context.
|
||||
|
||||
A node MAY support unsecure websockets if required by the application or
|
||||
running environment.
|
||||
|
||||
### Discovery domain
|
||||
### Discovery Domain
|
||||
|
||||
#### Discovery methods
|
||||
#### Discovery Methods
|
||||
|
||||
Waku v2 can retrieve a list of nodes to connect to using DNS-based discovery
|
||||
Waku can retrieve a list of nodes to connect to using DNS-based discovery
|
||||
as per [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459).
|
||||
While this is a useful way of bootstrapping connection to a set of peers,
|
||||
it MAY be used in conjunction with an [ambient peer discovery](https://docs.libp2p.io/concepts/publish-subscribe/#discovery)
|
||||
procedure to find still other nodes to connect to,
|
||||
procedure to find other nodes to connect to,
|
||||
such as [Node Discovery v5](https://github.com/ethereum/devp2p/blob/8fd5f7e1c1ec496a9d8dc1640a8548b8a8b5986b/discv5/discv5.md).
|
||||
More ambient peer discovery methods are being tested for Waku v2,
|
||||
and will be specified for wider adoption.
|
||||
It is possible to bypass the discovery domain by specifying static nodes.
|
||||
|
||||
#### Use of ENR
|
||||
|
||||
[WAKU2-ENR](https://github.com/waku-org/specs/blob/master/standards/core/enr.md)
|
||||
describes the usage of [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778)
|
||||
for Waku v2 discovery purposes.
|
||||
for Waku discovery purposes.
|
||||
It introduces two new ENR fields, `multiaddrs` and
|
||||
`waku2`, that a Waku v2 node MAY use for discovery purposes.
|
||||
These fields MUST be used under certain conditions, as set out in the spec.
|
||||
Both EIP-1459 DNS-based discovery and Node Discovery v5 operates on ENR,
|
||||
and it's reasonable to expect even wider utility for ENR in Waku v2 networks in future.
|
||||
`waku2`, that a Waku node MAY use for discovery purposes.
|
||||
These fields MUST be used under certain conditions, as set out in the specification.
|
||||
Both EIP-1459 DNS-based discovery and Node Discovery v5 operate on ENR,
|
||||
and it's reasonable to expect even wider utility for ENR in Waku networks in the future.
|
||||
|
||||
### Request/Reply domain
|
||||
### Request/Response Domain
|
||||
|
||||
In addition to the Gossip domain,
|
||||
Waku provides a set of Request/Reply protocols.
|
||||
Waku provides a set of request/response protocols.
|
||||
They are primarily used in order to get Waku to run in resource restricted environments,
|
||||
such as low bandwidth or being mostly offline.
|
||||
|
||||
#### Historical message support
|
||||
#### Historical Message Support
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
|
||||
**Protocol identifier***: `/vac/waku/store-query/3.0.0`
|
||||
|
||||
This is used to fetch historical messages for mostly offline devices.
|
||||
See [13/WAKU2-STORE spec](../13/store.md) spec for more details.
|
||||
See [13/WAKU2-STORE spec](/waku/standards/core/13/store.md) specification for more details.
|
||||
|
||||
There is also an experimental fault-tolerant addition to the store protocol
|
||||
that relaxes the high availability requirement.
|
||||
See [21/WAKU2-FT-STORE](../../application/21/fault-tolerant-store.md)
|
||||
See [21/WAKU2-FAULT-TOLERANT-STORE](/waku/standards/application/21/fault-tolerant-store.md)
|
||||
|
||||
#### Content filtering
|
||||
#### Content Filtering
|
||||
|
||||
**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
|
||||
|
||||
This is used to make fetching of a subset of messages more bandwidth preserving.
|
||||
See [12/WAKU2-FILTER](../12/filter.md) spec for more details.
|
||||
This is used to preserve more bandwidth when fetching a subset of messages.
|
||||
See [12/WAKU2-FILTER](/waku/standards/core/12/filter.md) specification for more details.
|
||||
|
||||
#### Light push
|
||||
#### LightPush
|
||||
|
||||
**Protocol identifier***: `/vac/waku/lightpush/2.0.0-beta1`
|
||||
|
||||
This is used for nodes with short connection windows and
|
||||
limited bandwidth to publish messages into the Waku network.
|
||||
See [19/WAKU2-LIGHTPUSH](../19/lightpush.md) spec for more details.
|
||||
See [19/WAKU2-LIGHTPUSH](/waku/standards/core/19/lightpush.md) specification for more details.
|
||||
|
||||
#### Other protocols
|
||||
#### Other Protocols
|
||||
|
||||
The above is a non-exhaustive list,
|
||||
and due to the modular design of Waku,
|
||||
there may be other protocols here that provide a useful service to the Waku network.
|
||||
|
||||
### Overview of protocol interaction
|
||||
### Overview of Protocol Interaction
|
||||
|
||||
See the sequence diagram below for an overview of how different protocols interact.
|
||||
|
||||

|
||||

|
||||
|
||||
0. We have six nodes, A-F.
|
||||
The protocols initially mounted are indicated as such.
|
||||
The PubSub topics `pubtopic1` and
|
||||
`pubtopic2` is used for routing and
|
||||
indicates that it is subscribed to messages on that topic for relay,
|
||||
see [11/WAKU2-RELAY](../11/relay.md) for details.
|
||||
Ditto for [13/WAKU2-STORE](../13/store.md)
|
||||
see [11/WAKU2-RELAY](/waku/standards/core/11/relay.md) for details.
|
||||
Ditto for [13/WAKU2-STORE](/waku/standards/core/13/store.md)
|
||||
where it indicates that these messages are persisted on that node.
|
||||
|
||||
1. Node A creates a WakuMessage `msg1` with a ContentTopic `contentTopic1`.
|
||||
See [14/WAKU2-MESSAGE](../14/message.md) for more details.
|
||||
See [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) for more details.
|
||||
If WakuMessage version is set to 1,
|
||||
we use the [6/WAKU1](../../legacy/6/waku1.md) compatible `data` field with encryption.
|
||||
See [7/WAKU-DATA](../../legacy/7/data.md) for more details.
|
||||
we use the [6/WAKU1](/waku/standards/legacy/6/waku1.md) compatible `data` field with encryption.
|
||||
See [7/WAKU-DATA](/waku/standards/legacy/7/data.md) for more details.
|
||||
|
||||
2. Node F requests to get messages filtered by PubSub topic `pubtopic1` and
|
||||
ContentTopic `contentTopic1`.
|
||||
Node D subscribes F to this filter and
|
||||
will in the future forward messages that match that filter.
|
||||
See [12/WAKU2-FILTER](../12/filter.md) for more details.
|
||||
See [12/WAKU2-FILTER](/waku/standards/core/12/filter.md) for more details.
|
||||
|
||||
3. Node A publishes `msg1` on `pubtopic1` and
|
||||
subscribes to that relay topic pick it up.
|
||||
subscribes to that relay topic.
|
||||
It then gets relayed further from B to D, but
|
||||
not C since it doesn't subscribe to that topic.
|
||||
See [11/WAKU2-RELAY](../11/relay.md).
|
||||
See [11/WAKU2-RELAY](/waku/standards/core/11/relay.md).
|
||||
|
||||
4. Node D saves `msg1` for possible later retrieval by other nodes.
|
||||
See [13/WAKU2-STORE](../13/store.md).
|
||||
See [13/WAKU2-STORE](/waku/standards/core/13/store.md).
|
||||
|
||||
5. Node D also pushes `msg1` to F, as it has previously subscribed F to this filter.
|
||||
See [12/WAKU2-FILTER](../12/filter.md).
|
||||
5. Node D also pushes `msg1` to F,
|
||||
as it has previously subscribed F to this filter.
|
||||
See [12/WAKU2-FILTER](/waku/standards/core/12/filter.md).
|
||||
|
||||
6. At a later time, Node E comes online.
|
||||
It then requests messages matching `pubtopic1` and `contentTopic1` from Node D.
|
||||
Node D responds with messages meeting this (and possibly other) criteria. See [13/WAKU2-STORE](../13/store.md).
|
||||
It then requests messages matching `pubtopic1` and
|
||||
`contentTopic1` from Node D.
|
||||
Node D responds with messages meeting this (and possibly other) criteria.
|
||||
See [13/WAKU2-STORE](/waku/standards/core/13/store.md).
|
||||
|
||||
## Appendix A: Upgradability and Compatibility
|
||||
|
||||
### Compatibility with Waku v1
|
||||
### Compatibility with Waku Legacy
|
||||
|
||||
Waku v1 and Waku v2 are different protocols all together.
|
||||
[6/WAKU1](/waku/standards/legacy/6/waku1.md) and Waku are different protocols all together.
|
||||
They use a different transport protocol underneath;
|
||||
Waku v1 is devp2p RLPx based while Waku v2 uses libp2p.
|
||||
[6/WAKU1](/waku/standards/legacy/6/waku1.md) is devp2p RLPx based while Waku uses libp2p.
|
||||
The protocols themselves also differ as does their data format.
|
||||
Compatibility can be achieved only by using a bridge
|
||||
that not only talks both devp2p RLPx and libp2p,
|
||||
but that also transfers (partially) the content of a packet from one version
|
||||
to the other.
|
||||
|
||||
See [15/WAKU-BRIDGE](../15/bridge.md) for details on a bidirectional bridge mode.
|
||||
See [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md) for details on a bidirectional bridge mode.
|
||||
|
||||
## Appendix B: Security
|
||||
|
||||
Each protocol layer of Waku v2 provides a distinct service and
|
||||
Each protocol layer of Waku provides a distinct service and
|
||||
is associated with a separate set of security features and concerns.
|
||||
Therefore, the overall security of Waku v2
|
||||
Therefore, the overall security of Waku
|
||||
depends on how the different layers are utilized.
|
||||
In this section,
|
||||
we overview the security properties of Waku v2 protocols
|
||||
we overview the security properties of Waku protocols
|
||||
against a static adversarial model which is described below.
|
||||
Note that a more detailed security analysis of each Waku protocol
|
||||
is supplied in its respective specification as well.
|
||||
|
||||
## Primary Adversarial Model
|
||||
### Primary Adversarial Model
|
||||
|
||||
In the primary adversarial model,
|
||||
we consider adversary as a passive entity that attempts to collect information
|
||||
@@ -357,11 +363,11 @@ between arbitrary pairs of peers
|
||||
(unless the adversary is one end of the communication).
|
||||
Specifically, the communication channels are assumed to be secure.
|
||||
|
||||
## Security Features
|
||||
### Security Features
|
||||
|
||||
### Pseudonymity
|
||||
#### Pseudonymity
|
||||
|
||||
Waku v2 by default guarantees pseudonymity for all of the protocol layers
|
||||
Waku by default guarantees pseudonymity for all of the protocol layers
|
||||
since parties do not have to disclose their true identity
|
||||
and instead they utilize libp2p `PeerID` as their identifiers.
|
||||
While pseudonymity is an appealing security feature,
|
||||
@@ -369,7 +375,7 @@ it does not guarantee full anonymity since the actions taken under the same pseu
|
||||
i.e., `PeerID` can be linked together and
|
||||
potentially result in the re-identification of the true actor.
|
||||
|
||||
### Anonymity / Unlinkability
|
||||
#### Anonymity / Unlinkability
|
||||
|
||||
At a high level,
|
||||
anonymity is the inability of an adversary in linking an actor
|
||||
@@ -384,19 +390,19 @@ hence count as PII.
|
||||
Notice that users' actions can be traced through their PIIs
|
||||
(e.g., signatures) and hence result in their re-identification risk.
|
||||
As such, we seek anonymity by avoiding linkability between actions and
|
||||
the actors / actors' PII. Concerning anonymity, Waku v2 provides the following features:
|
||||
the actors / actors' PII. Concerning anonymity, Waku provides the following features:
|
||||
|
||||
**Publisher-Message Unlinkability**:
|
||||
This feature signifies the unlinkability of a publisher
|
||||
to its published messages in the 11/WAKU2-RELAY protocol.
|
||||
The [Publisher-Message Unlinkability](../11/relay.md/#security-analysis)
|
||||
The [Publisher-Message Unlinkability](/waku/standards/core/11/relay.md/#security-analysis)
|
||||
is enforced through the `StrictNoSign` policy due to which the data fields
|
||||
of pubsub messages that count as PII for the publisher must be left unspecified.
|
||||
|
||||
**Subscriber-Topic Unlinkability**:
|
||||
This feature stands for the unlinkability of the subscriber
|
||||
to its subscribed topics in the 11/WAKU2-RELAY protocol.
|
||||
The [Subscriber-Topic Unlinkability](../11/relay.md/#security-analysis)
|
||||
The [Subscriber-Topic Unlinkability](/waku/standards/core/11/relay.md/#security-analysis)
|
||||
is achieved through the utilization of a single PubSub topic.
|
||||
As such, subscribers are not re-identifiable from their subscribed topic IDs
|
||||
as the entire network is linked to the same topic ID.
|
||||
@@ -405,7 +411,7 @@ where k is proportional to the system size (number of subscribers).
|
||||
Note that there is no hard limit on the number of the pubsub topics, however,
|
||||
the use of one topic is recommended for the sake of anonymity.
|
||||
|
||||
### Spam protection
|
||||
#### Spam protection
|
||||
|
||||
This property indicates that no adversary can flood the system
|
||||
(i.e., publishing a large number of messages in a short amount of time),
|
||||
@@ -413,19 +419,19 @@ either accidentally or deliberately, with any kind of message
|
||||
i.e. even if the message content is valid or useful.
|
||||
Spam protection is partly provided in `11/WAKU2-RELAY`
|
||||
through the [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures)
|
||||
rovided for by GossipSub v1.1.
|
||||
provided for by GossipSub v1.1.
|
||||
At a high level,
|
||||
peers utilize a scoring function to locally score the behavior
|
||||
of their connections and remove peers with a low score.
|
||||
|
||||
### Data confidentiality, Integrity, and Authenticity
|
||||
#### Data confidentiality, Integrity, and Authenticity
|
||||
|
||||
Confidentiality can be addressed through data encryption whereas integrity and
|
||||
authenticity are achievable through digital signatures.
|
||||
These features are provided for in [14/WAKU2-MESSAGE (version 1)](../14/message.md/#version-1)`
|
||||
These features are provided for in [14/WAKU2-MESSAGE (version 1)](/waku/standards/core/14/message.md/#version-1)`
|
||||
through payload encryption as well as encrypted signatures.
|
||||
|
||||
## Security Considerations
|
||||
### Security Considerations
|
||||
|
||||
Lack of anonymity/unlinkability in the protocols involving direct connections
|
||||
including `13/WAKU2-STORE` and `12/WAKU2-FILTER` protocols:
|
||||
@@ -454,51 +460,51 @@ the security features of each layer-->
|
||||
|
||||
### Implementation Matrix
|
||||
|
||||
There are multiple implementations of Waku v2 and its protocols:
|
||||
There are multiple implementations of Waku and its protocols:
|
||||
|
||||
- [nim-waku (Nim)](https://github.com/status-im/nim-waku/)
|
||||
- [go-waku (Go)](https://github.com/status-im/go-waku/)
|
||||
- [js-waku (NodeJS and Browser)](https://github.com/status-im/js-waku/)
|
||||
|
||||
Below you can find an overview of the specs that they implement
|
||||
as they relate to Waku v2.
|
||||
This includes Waku v1 specs, as they are used for bridging between the two networks.
|
||||
Below you can find an overview of the specifications that they implement
|
||||
as they relate to Waku.
|
||||
This includes Waku legacy specifications, as they are used for bridging between the two networks.
|
||||
|
||||
| Spec | nim-waku (Nim) | go-waku (Go) | js-waku (Node JS) | js-waku (Browser JS) |
|
||||
| ---- | -------------- | ------------ | ----------------- | -------------------- |
|
||||
|[6/WAKU1](../../legacy/6/waku1.md)|✔||||
|
||||
|[7/WAKU-DATA](../../legacy/7/data.md)|✔|✔|||
|
||||
|[8/WAKU-MAIL](../../legacy/8/mail.md)|✔||||
|
||||
|[9/WAKU-RPC](../../legacy/9/rpc.md)|✔||||
|
||||
|[10/WAKU2](../10/waku2.md)|✔|🚧|🚧|🚧|
|
||||
|[11/WAKU2-RELAY](../11/relay.md)|✔|✔|✔|✔|
|
||||
|[12/WAKU2-FILTER](../12/filter.md)|✔|✔|||
|
||||
|[13/WAKU2-STORE](../13/store.md)|✔|✔|✔\*|✔\*|
|
||||
|[14/WAKU2-MESSAGE](../14/message.md))|✔|✔|✔|✔|
|
||||
|[15/WAKU2-BRIDGE](../15/bridge.md)|✔||||
|
||||
|[16/WAKU2-RPC](../16/rpc.md)|✔||||
|
||||
|[17/WAKU2-RLN-RELAY](../17/rln-relay.md)|🚧||||
|
||||
|[18/WAKU2-SWAP](../../application/18/swap.md)|🚧||||
|
||||
|[19/WAKU2-LIGHTPUSH](../19/lightpush.md)|✔|✔|✔\**|✔\**|
|
||||
|[21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)|✔|✔|||
|
||||
|[6/WAKU1](/waku/standards/legacy/6/waku1.md)|✔||||
|
||||
|[7/WAKU-DATA](/waku/standards/legacy/7/data.md)|✔|✔|||
|
||||
|[8/WAKU-MAIL](/waku/standards/legacy/8/mail.md)|✔||||
|
||||
|[9/WAKU-RPC](/waku/standards/legacy/9/rpc.md)|✔||||
|
||||
|[10/WAKU2](/waku/standards/core/10/waku2.md)|✔|🚧|🚧|✔|
|
||||
|[11/WAKU2-RELAY](/waku/standards/core/11/relay.md)|✔|✔|✔|✔|
|
||||
|[12/WAKU2-FILTER](/waku/standards/core/12/filter.md)|✔|✔|||
|
||||
|[13/WAKU2-STORE](/waku/standards/core/13/store.md)|✔|✔|✔\*|✔\*|
|
||||
|[14/WAKU2-MESSAGE](/waku/standards/core/14/message.md))|✔|✔|✔|✔|
|
||||
|[15/WAKU2-BRIDGE](/waku/standards/core/15/bridge.md)|✔||||
|
||||
|[16/WAKU2-RPC](/waku/deprecated/16/rpc.md)|✔||||
|
||||
|[17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md)|🚧||||
|
||||
|[18/WAKU2-SWAP](/waku/standards/application/18/swap.md)|🚧||||
|
||||
|[19/WAKU2-LIGHTPUSH](/waku/standards/core/19/lightpush.md)|✔|✔|✔\**|✔\**|
|
||||
|[21/WAKU2-FAULT-TOLERANT-STORE](/waku/standards/application/21/fault-tolerant-store.md)|✔|✔|||
|
||||
|
||||
*js-waku implements [13/WAKU2-STORE](../13/store.md) as a querying node only.
|
||||
**js-waku only implements [19/WAKU2-LIGHTPUSH](../19/lightpush.md) requests.
|
||||
*js-waku implements [13/WAKU2-STORE](/waku/standards/core/13/store.md) as a querying node only.
|
||||
**js-waku only implements [19/WAKU2-LIGHTPUSH](/waku/standards/core/19/lightpush.md) requests.
|
||||
|
||||
### Recommendations for clients
|
||||
### Recommendations for Clients
|
||||
|
||||
To implement a minimal Waku v2 client,
|
||||
To implement a minimal Waku client,
|
||||
we recommend implementing the following subset in the following order:
|
||||
|
||||
- [10/WAKU2](../10/waku2.md) - this spec
|
||||
- [11/WAKU2-RELAY](../11/relay.md) - for basic operation
|
||||
- [14/WAKU2-MESSAGE](../14/message.md) - version 0 (unencrypted)
|
||||
- [13/WAKU2-STORE](../13/store.md) - for historical messaging (query mode only)
|
||||
- [10/WAKU2](/waku/standards/core/10/waku2.md) - this specification
|
||||
- [11/WAKU2-RELAY](/waku/standards/core/11/relay.md) - for basic operation
|
||||
- [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) - version 0 (unencrypted)
|
||||
- [13/WAKU2-STORE](/waku/standards/core/13/store.md) - for historical messaging (query mode only)
|
||||
|
||||
To get compatibility with Waku v1:
|
||||
To get compatibility with Waku Legacy:
|
||||
|
||||
- [7/WAKU-DATA](../../legacy/7/data.md)
|
||||
- [14/WAKU2-MESSAGE](../14/message.md) - version 1 (encrypted with `7/WAKU-DATA`)
|
||||
- [7/WAKU-DATA](/waku/standards/legacy/7/data.md)
|
||||
- [14/WAKU2-MESSAGE](/waku/standards/14/message.md) - version 1 (encrypted with `7/WAKU-DATA`)
|
||||
|
||||
For an interoperable keep-alive mechanism:
|
||||
|
||||
@@ -507,13 +513,14 @@ with periodic pings to connected peers
|
||||
|
||||
## Appendix D: Future work
|
||||
|
||||
The following features are currently experimental and under research and
|
||||
initial implementation:
|
||||
The following features are currently experimental,
|
||||
under research and initial implementations:
|
||||
|
||||
**Economic Spam Resistance**:
|
||||
|
||||
**Economic Spam resistance**:
|
||||
We aim to enable an incentivized spam protection technique
|
||||
to enhance `11/WAKU2-RELAY` by using rate limiting nullifiers.
|
||||
More details on this can be found in [17/WAKU2-RLN-RELAY](../17/rln-relay.md).
|
||||
More details on this can be found in [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md).
|
||||
In this advanced method,
|
||||
peers are limited to a certain rate of messaging per epoch and
|
||||
an immediate financial penalty is enforced for spammers who break this rate.
|
||||
@@ -522,7 +529,7 @@ an immediate financial penalty is enforced for spammers who break this rate.
|
||||
Denial of service signifies the case where an adversarial node
|
||||
exhausts another node's service capacity (e.g., by making a large number of requests)
|
||||
and makes it unavailable to the rest of the system.
|
||||
DoS attack is to be mitigated through the accounting model as described in [18/WAKU2-SWAP](../../application/18/swap.md).
|
||||
DoS attack is to be mitigated through the accounting model as described in [18/WAKU2-SWAP](/waku/deprecated/18/swap.md).
|
||||
In a nutshell, peers have to pay for the service they obtain from each other.
|
||||
In addition to incentivizing the service provider,
|
||||
accounting also makes DoS attacks costly for malicious peers.
|
||||
@@ -531,7 +538,7 @@ The accounting model can be used in `13/WAKU2-STORE` and
|
||||
|
||||
Additionally, this gives node operators who provide a useful service to the network
|
||||
an incentive to perform that service.
|
||||
See [18/WAKU2-SWAP](../../application/18/swap.md)
|
||||
See [18/WAKU2-SWAP](/waku/deprecated/18/swap.md)
|
||||
for more details on this piece of work.
|
||||
|
||||
## Copyright
|
||||
@@ -542,31 +549,31 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
1. [libp2p specs](https://github.com/libp2p/specs)
|
||||
|
||||
2. [6/WAKU1](../../legacy/6/waku1.md)
|
||||
2. [6/WAKU1](/waku/standards/legacy/6/waku1.md)
|
||||
|
||||
3. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)
|
||||
|
||||
4. [Waku v2 plan](https://vac.dev/waku-v2-plan)
|
||||
|
||||
5. [30/ADAPTIVE-NODES](../../../informational/30/adaptive-nodes.md)
|
||||
5. [30/ADAPTIVE-NODES](/waku/informational/30/adaptive-nodes.md)
|
||||
|
||||
6. [Protocol Identifiers](https://docs.libp2p.io/concepts/protocols/)
|
||||
|
||||
7. [14/WAKU2-MESSAGE](../14/message.md)
|
||||
7. [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
|
||||
8. [26/WAKU-PAYLOAD](../../application/26/payload.md)
|
||||
8. [26/WAKU-PAYLOAD](/waku/standards/application/26/payload.md)
|
||||
|
||||
9. [23/WAKU2-TOPICS](../../../informational/23/topics.md)
|
||||
9. [23/WAKU2-TOPICS](/waku/informational/23/topics.md)
|
||||
|
||||
10. [27/WAKU2-PEERS](../../../informational/27/peers.md)
|
||||
10. [27/WAKU2-PEERS](/waku/informational/27/peers.md)
|
||||
|
||||
11. [bi-directional binary stream](https://docs.libp2p.io/concepts/protocols/)
|
||||
|
||||
12. [Protobuf varint encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints)
|
||||
|
||||
13. [11/WAKU2-RELAY spec](../11/relay.md)
|
||||
13. [11/WAKU2-RELAY spec](/waku/standards/core/11/relay.md)
|
||||
|
||||
14. [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
|
||||
14. [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md)
|
||||
|
||||
15. [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
|
||||
|
||||
@@ -578,17 +585,17 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
19. [EIP-778 ENR (Ethereum Node Records)](https://eips.ethereum.org/EIPS/eip-778)
|
||||
|
||||
20. [13/WAKU2-STORE spec](../13/store.md)
|
||||
20. [13/WAKU2-STORE spec](/waku/standards/core/13/store.md)
|
||||
|
||||
21. [21/WAKU2-FT-STORE](../../application/21/ft-store.md)
|
||||
21. [21/WAKU2-FT-STORE](/waku/standards/application/21/ft-store.md)
|
||||
|
||||
22. [12/WAKU2-FILTER](../12/filter.md)
|
||||
22. [12/WAKU2-FILTER](/waku/standards/core/12/filter.md)
|
||||
|
||||
23. [19/WAKU2-LIGHTPUSH](../19/lightpush.md)
|
||||
23. [19/WAKU2-LIGHTPUSH](/waku/standards/core/19/lightpush.md)
|
||||
|
||||
24. [7/WAKU-DATA](../../legacy/7/data.md)
|
||||
24. [7/WAKU-DATA](/waku/standards/legacy/7/data.md)
|
||||
|
||||
25. [15/WAKU-BRIDGE](../15/bridge.md)
|
||||
25. [15/WAKU-BRIDGE](/waku/standards/core/15/bridge.md)
|
||||
|
||||
26. [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/)
|
||||
|
||||
@@ -600,12 +607,12 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
30. [js-waku (NodeJS and Browser)](https://github.com/status-im/js-waku/)
|
||||
|
||||
31. [8/WAKU-MAIL](../../legacy/8/mail.md)
|
||||
31. [8/WAKU-MAIL](/waku/standards/legacy/8/mail.md)
|
||||
|
||||
32. [9/WAKU-RPC](../../legacy/9/rpc.md)
|
||||
32. [9/WAKU-RPC](/waku/standards/legacy/9/rpc.md)
|
||||
|
||||
33. [16/WAKU2-RPC](../16/rpc.md)
|
||||
33. [16/WAKU2-RPC](waku/deprecated/16/rpc.md)
|
||||
|
||||
34. [18/WAKU2-SWAP spec](../../application/18/swap.md)
|
||||
34. [18/WAKU2-SWAP spec](waku/deprecated/18/swap.md)
|
||||
|
||||
35. [21/WAKU2-FAULT-TOLERANT-STORE](../../application/21/fault-tolerant-store.md)
|
||||
|
||||
@@ -13,61 +13,54 @@ contributors:
|
||||
- Ebube Ud <ebube@status.im>
|
||||
---
|
||||
|
||||
previous versions: [00](./previous-versions00)
|
||||
|
||||
---
|
||||
|
||||
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives.
|
||||
This is a more lightweight version of `WakuRelay`
|
||||
specifically designed for bandwidth restricted devices.
|
||||
This is due to the fact that light nodes subscribe to full-nodes and
|
||||
only receive the messages they desire.
|
||||
|
||||
## Content filtering
|
||||
previous versions: [00](/waku/standards/core/12/previous-versions/00/filter.md)
|
||||
|
||||
**Protocol identifiers**:
|
||||
|
||||
- _filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
|
||||
- _filter-push_: `/vac/waku/filter-push/2.0.0-beta1`
|
||||
|
||||
Content filtering is a way to do [message-based
|
||||
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
|
||||
Currently the only content filter being applied is on `contentTopic`. This
|
||||
corresponds to topics in Waku v1.
|
||||
---
|
||||
|
||||
## Rationale
|
||||
## Abstract
|
||||
|
||||
Unlike the `store` protocol for historical messages, this protocol allows for
|
||||
native lower latency scenarios such as instant messaging. It is thus
|
||||
complementary to it.
|
||||
This specification describes the `12/WAKU2-FILTER` protocol,
|
||||
which enables a client to subscribe to a subset of real-time messages from a Waku peer.
|
||||
This is a more lightweight version of [11/WAKU2-RELAY](/waku/standards/core/11/relay.md),
|
||||
useful for bandwidth restricted devices.
|
||||
This is often used by nodes with lower resource limits to subscribe to full Relay nodes and
|
||||
only receive the subset of messages they desire,
|
||||
based on content topic interest.
|
||||
|
||||
Strictly speaking, it is not just doing basic request response, but performs
|
||||
sender push based on receiver intent. While this can be seen as a form of light
|
||||
pub/sub, it is only used between two nodes in a direct fashion. Unlike the
|
||||
Gossip domain, this is meant for light nodes which put a premium on bandwidth.
|
||||
## Motivation
|
||||
|
||||
Unlike the [13/WAKU2-STORE](/waku/standards/core/13/store.md) protocol
|
||||
for historical messages, this protocol allows for native lower latency scenarios,
|
||||
such as instant messaging.
|
||||
It is thus complementary to it.
|
||||
|
||||
Strictly speaking, it is not just doing basic request-response, but
|
||||
performs sender push based on receiver intent.
|
||||
While this can be seen as a form of light publish/subscribe,
|
||||
it is only used between two nodes in a direct fashion. Unlike the
|
||||
Gossip domain, this is suitable for light nodes which put a premium on bandwidth.
|
||||
No gossiping takes place.
|
||||
|
||||
It is worth noting that a light node could get by with only using the `store`
|
||||
protocol to query for a recent time window, provided it is acceptable to do
|
||||
frequent polling.
|
||||
It is worth noting that a light node could get by with only using the
|
||||
[13/WAKU2-STORE](/waku/standards/core/13/store.md) protocol to
|
||||
query for a recent time window, provided it is acceptable to do frequent polling.
|
||||
|
||||
## Design Requirements
|
||||
## Semantics
|
||||
|
||||
The effectiveness and reliability of the content filtering service enabled by
|
||||
`WakuFilter` protocol rely on the _high availability_ of the full nodes
|
||||
as the service providers.
|
||||
To this end, full nodes must feature _high uptime_
|
||||
(to persistently listen and capture the network messages)
|
||||
as well as _high Bandwidth_ (to provide timely message delivery to the light nodes).
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”,
|
||||
“SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
## Security Consideration
|
||||
### Content filtering
|
||||
|
||||
Note that while using `WakuFilter` allows light nodes to save bandwidth,
|
||||
it comes with a privacy cost in the sense that they need to
|
||||
disclose their liking topics to the full nodes to retrieve the relevant messages.
|
||||
Currently, anonymous subscription is not supported by the `WakuFilter`, however,
|
||||
potential solutions in this regard are sketched
|
||||
below in [Future Work](#future-work) section.
|
||||
Content filtering is a way to do
|
||||
[message-based filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
|
||||
Currently the only content filter being applied is on `contentTopic`.
|
||||
|
||||
### Terminology
|
||||
|
||||
@@ -76,33 +69,11 @@ refers to any piece of data that can be used to uniquely identify a user.
|
||||
For example, the signature verification key, and
|
||||
the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
## Adversarial Model
|
||||
|
||||
Any node running the `WakuFilter` protocol
|
||||
i.e., both the subscriber node and the queried node are considered as an adversary.
|
||||
Furthermore, we consider the adversary as a passive entity
|
||||
that attempts to collect information from other nodes to conduct an attack but
|
||||
it does so without violating protocol definitions and instructions.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node intentionally hides the messages
|
||||
matching to one's subscribed content filter
|
||||
as it is against the description of the `WakuFilter` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the nodes and their connections.
|
||||
- An adversary that can eavesdrop on communication links
|
||||
between arbitrary pairs of nodes (unless the adversary is one end of the communication).
|
||||
In specific, the communication channels are assumed to be secure.
|
||||
|
||||
### Protobuf
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
// 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/
|
||||
package waku.filter.v2;
|
||||
|
||||
// Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1
|
||||
message FilterSubscribeRequest {
|
||||
enum FilterSubscribeType {
|
||||
@@ -145,7 +116,6 @@ in its registered subscriptions.
|
||||
|
||||
Since a filter service node is consuming resources to provide this service,
|
||||
it MAY account for usage and adapt its service provision to certain clients.
|
||||
An incentive mechanism is currently planned but underspecified.
|
||||
|
||||
#### Filter Subscribe Request
|
||||
|
||||
@@ -156,8 +126,8 @@ Each request MUST include a `filter_subscribe_type`, indicating the type of requ
|
||||
|
||||
#### Filter Subscribe Response
|
||||
|
||||
In return to any `FilterSubscribeRequest`,
|
||||
a filter service node SHOULD respond with a `FilterSubscribeResponse`
|
||||
When responding to a `FilterSubscribeRequest`,
|
||||
a filter service node SHOULD send a `FilterSubscribeResponse`
|
||||
with a `requestId` matching that of the request.
|
||||
This response MUST contain a `status_code` indicating if the request was successful
|
||||
or not.
|
||||
@@ -178,41 +148,43 @@ conditional to the selected `filter_subscribe_type`.
|
||||
If the request contains filter criteria,
|
||||
it MUST contain a `pubsub_topic`
|
||||
and the `content_topics` set MUST NOT be empty.
|
||||
A `WakuMessage` matches filter criteria
|
||||
A [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) matches filter criteria
|
||||
when its `content_topic` is in the `content_topics` set
|
||||
and it was published on a matching `pubsub_topic`.
|
||||
|
||||
#### Filter Subscribe Types
|
||||
|
||||
The following filter subscribe types are defined:
|
||||
The filter-subscribe types are defined as follows:
|
||||
|
||||
##### SUBSCRIBER_PING
|
||||
|
||||
A filter client that sends a `FilterSubscribeRequest` with
|
||||
`filter_subscribe_type` set to `SUBSCRIBER_PING`
|
||||
requests that the service node SHOULD indicate if it has any active subscriptions
|
||||
`filter_subscribe_type` set to `SUBSCRIBER_PING`,
|
||||
requests that the filter service node SHOULD indicate if it has any active subscriptions
|
||||
for this client.
|
||||
The filter client SHOULD exclude any filter criteria from the request.
|
||||
The filter service node SHOULD respond with a success code
|
||||
The filter service node SHOULD respond with a success `status_code`
|
||||
if it has any active subscriptions for this client
|
||||
or an error code if not.
|
||||
or an error `status_code` if not.
|
||||
The filter service node SHOULD ignore any filter criteria in the request.
|
||||
|
||||
##### SUBSCRIBE
|
||||
|
||||
A filter client that sends a `FilterSubscribeRequest` with
|
||||
`filter_subscribe_type` set to `SUBSCRIBE`
|
||||
requests that the service node SHOULD push messages matching this filter to the client.
|
||||
requests that the filter service node SHOULD push messages
|
||||
matching this filter to the client.
|
||||
The filter client MUST include the desired filter criteria in the request.
|
||||
A client MAY use this request type to _modify_ an existing subscription
|
||||
by providing _additional_ filter criteria in a new request.
|
||||
A client MAY use this request type to _refresh_ an existing subscription
|
||||
by providing _the same_ filter criteria in a new request.
|
||||
The filter service node SHOULD respond with a success code
|
||||
The filter service node SHOULD respond with a success `status_code`
|
||||
if it successfully honored this request
|
||||
or an error code if not.
|
||||
The filter service node SHOULD respond with an error code and discard the request
|
||||
if the subscribe request does not contain valid filter criteria,
|
||||
or an error `status_code` if not.
|
||||
The filter service node SHOULD respond with an error `status_code` and
|
||||
discard the request if the `FilterSubscribeRequest`
|
||||
does not contain valid filter criteria,
|
||||
i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
|
||||
|
||||
##### UNSUBSCRIBE
|
||||
@@ -226,11 +198,11 @@ it desires to unsubscribe from in the request.
|
||||
A client MAY use this request type to _modify_ an existing subscription
|
||||
by providing _a subset of_ the original filter criteria
|
||||
to unsubscribe from in a new request.
|
||||
The filter service node SHOULD respond with a success code
|
||||
The filter service node SHOULD respond with a success `status_code`
|
||||
if it successfully honored this request
|
||||
or an error code if not.
|
||||
The filter service node SHOULD respond with an error code and discard the request
|
||||
if the unsubscribe request does not contain valid filter criteria,
|
||||
or an error `status_code` if not.
|
||||
The filter service node SHOULD respond with an error `status_code` and
|
||||
discard the request if the unsubscribe request does not contain valid filter criteria,
|
||||
i.e. both a `pubsub_topic` _and_ a non-empty `content_topics` set.
|
||||
|
||||
##### UNSUBSCRIBE_ALL
|
||||
@@ -241,8 +213,8 @@ requests that the service node SHOULD _stop_ pushing messages
|
||||
matching _any_ filter to the client.
|
||||
The filter client SHOULD exclude any filter criteria from the request.
|
||||
The filter service node SHOULD remove any existing subscriptions for this client.
|
||||
It SHOULD respond with a success code if it successfully honored this request
|
||||
or an error code if not.
|
||||
It SHOULD respond with a success `status_code` if it successfully honored this request
|
||||
or an error `status_code` if not.
|
||||
|
||||
### Filter-Push
|
||||
|
||||
@@ -253,7 +225,8 @@ matching registered subscriptions to this client.
|
||||
A filter service node SHOULD push all messages
|
||||
matching the filter criteria in a registered subscription
|
||||
to the subscribed filter client.
|
||||
These [`WakuMessage`s](../14/message.md) are likely to come from [`11/WAKU2-RELAY`](../11/relay.md),
|
||||
These [`WakuMessage`s](/waku/standards/core/14/message.md)
|
||||
are likely to come from [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md),
|
||||
but there MAY be other sources or protocols where this comes from.
|
||||
This is up to the consumer of the protocol.
|
||||
|
||||
@@ -264,7 +237,7 @@ for a period of time,
|
||||
the filter service node MAY choose to stop pushing messages to the client and
|
||||
remove its subscription.
|
||||
This period is up to the service node implementation.
|
||||
We consider `1 minute` to be a reasonable default.
|
||||
It is RECOMMENDED to set `1 minute` as a reasonable default.
|
||||
|
||||
#### Message Push
|
||||
|
||||
@@ -280,9 +253,35 @@ A filter client SHOULD verify that each `MessagePush` it receives
|
||||
originated from a service node where the client has an active subscription
|
||||
and that it matches filter criteria belonging to that subscription.
|
||||
|
||||
---
|
||||
### Adversarial Model
|
||||
|
||||
## Future Work
|
||||
Any node running the `WakuFilter` protocol
|
||||
i.e., both the subscriber node and
|
||||
the queried node are considered as an adversary.
|
||||
Furthermore, we consider the adversary as a passive entity
|
||||
that attempts to collect information from other nodes to conduct an attack but
|
||||
it does so without violating protocol definitions and instructions.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node intentionally hides the messages
|
||||
matching to one's subscribed content filter
|
||||
as it is against the description of the `WakuFilter` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the nodes and their connections.
|
||||
- An adversary that can eavesdrop on communication links
|
||||
between arbitrary pairs of nodes (unless the adversary is one end of the communication).
|
||||
In specific, the communication channels are assumed to be secure.
|
||||
|
||||
### Security Considerations
|
||||
|
||||
Note that while using `WakuFilter` allows light nodes to save bandwidth,
|
||||
it comes with a privacy cost in the sense that they need to
|
||||
disclose their liking topics to the full nodes to retrieve the relevant messages.
|
||||
Currently, anonymous subscription is not supported by the `WakuFilter`, however,
|
||||
potential solutions in this regard are discussed below.
|
||||
|
||||
#### Future Work
|
||||
<!-- Alternative title: Filter-subscriber unlinkability -->
|
||||
**Anonymous filter subscription**:
|
||||
This feature guarantees that nodes can anonymously subscribe for a message filter
|
||||
@@ -318,25 +317,6 @@ Examples of such 2PC protocols are
|
||||
[Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.)
|
||||
and one-way Private Set Intersections (PSI).
|
||||
|
||||
## Changelog
|
||||
|
||||
### Next
|
||||
|
||||
- Added initial threat model and security analysis.
|
||||
|
||||
### 2.0.0-beta2
|
||||
|
||||
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
|
||||
|
||||
- Fix: Ensure contentFilter is a repeated field, on implementation
|
||||
- Change: Add ability to unsubscribe from filters.
|
||||
Make `subscribe` an explicit boolean indication.
|
||||
Edit protobuf field order to be consistent with libp2p.
|
||||
|
||||
### 2.0.0-beta1
|
||||
|
||||
Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
@@ -344,13 +324,14 @@ Copyright and related rights waived via
|
||||
|
||||
## References
|
||||
|
||||
- [message-based
|
||||
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
|
||||
- [`WakuMessage`s](../14/message.md)
|
||||
- [`11/WAKU2-RELAY`](../11/relay.md)
|
||||
- [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
- [message-based filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
|
||||
- [13/WAKU2-STORE](/waku/standards/core/13/store.md)
|
||||
- [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
- [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains)
|
||||
- previous versions: [00](./previous-versions00)
|
||||
- 12/WAKU2-FILTER previous version: [00](waku/standards/core/12/previous-versions/00/filter.md)
|
||||
|
||||
### Informative
|
||||
|
||||
1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
|
||||
|
||||
2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)
|
||||
|
||||
@@ -4,6 +4,7 @@ title: 12/WAKU2-FILTER
|
||||
name: Waku v2 Filter
|
||||
status: draft
|
||||
tags: waku-core
|
||||
version: v00
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
@@ -11,9 +12,8 @@ contributors:
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
- Ebube Ud <ebube@status.im>
|
||||
---
|
||||
version: 00
|
||||
|
||||
---
|
||||
|
||||
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives.
|
||||
This is a more lightweight version of `WakuRelay`
|
||||
specifically designed for bandwidth restricted devices.
|
||||
359
waku/standards/core/13/previous-versions/00/store.md
Normal file
359
waku/standards/core/13/previous-versions/00/store.md
Normal file
@@ -0,0 +1,359 @@
|
||||
---
|
||||
slug: 13
|
||||
title: 13/WAKU2-STORE
|
||||
name: Waku v2 Store
|
||||
status: draft
|
||||
tags: waku-core
|
||||
version: 00
|
||||
editor: Simon-Pierre Vivier <simvivier@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Aaryamann Challani <p1ge0nh8er@proton.me>
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
- Hanno Cornelius <hanno@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification explains the `13/WAKU2-STORE` protocol
|
||||
which enables querying of messages received through the relay protocol and
|
||||
stored by other nodes.
|
||||
It also supports pagination for more efficient querying of historical messages.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
|
||||
|
||||
## Terminology
|
||||
|
||||
The term PII, Personally Identifiable Information,
|
||||
refers to any piece of data that can be used to uniquely identify a user.
|
||||
For example, the signature verification key, and
|
||||
the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
## Design Requirements
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
|
||||
SHOULD provide a complete and full view of message history.
|
||||
As such, they are required to be *highly available* and
|
||||
specifically have a *high uptime* to consistently receive and store network messages.
|
||||
The high uptime requirement makes sure that no message is missed out
|
||||
hence a complete and intact view of the message history
|
||||
is delivered to the querying nodes.
|
||||
Nevertheless, in case storage provider nodes cannot afford high availability,
|
||||
the querying nodes may retrieve the historical messages from multiple sources
|
||||
to achieve a full and intact view of the past.
|
||||
|
||||
The concept of `ephemeral` messages introduced in
|
||||
[`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
|
||||
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in
|
||||
[14/WAKU2-MESSAGE](../14/message.md).
|
||||
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages
|
||||
with the `ephemeral` flag set to `true`.
|
||||
|
||||
## Adversarial Model
|
||||
|
||||
Any peer running the `13/WAKU2-STORE` protocol, i.e.
|
||||
both the querying node and the queried node, are considered as an adversary.
|
||||
Furthermore,
|
||||
we currently consider the adversary as a passive entity
|
||||
that attempts to collect information from other peers to conduct an attack but
|
||||
it does so without violating protocol definitions and instructions.
|
||||
As we evolve the protocol,
|
||||
further adversarial models will be considered.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node hides or
|
||||
lies about the history of messages
|
||||
as it is against the description of the `13/WAKU2-STORE` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links
|
||||
between arbitrary pairs of peers (unless the adversary is one end of the communication).
|
||||
In specific, the communication channels are assumed to be secure.
|
||||
|
||||
## Wire Specification
|
||||
|
||||
Peers communicate with each other using a request / response API.
|
||||
The messages sent are Protobuf RPC messages which are implemented using
|
||||
[protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
The following are the specifications of the Protobuf messages.
|
||||
|
||||
### Payloads
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
message Index {
|
||||
bytes digest = 1;
|
||||
sint64 receiverTime = 2;
|
||||
sint64 senderTime = 3;
|
||||
string pubsubTopic = 4;
|
||||
}
|
||||
|
||||
message PagingInfo {
|
||||
uint64 pageSize = 1;
|
||||
Index cursor = 2;
|
||||
enum Direction {
|
||||
BACKWARD = 0;
|
||||
FORWARD = 1;
|
||||
}
|
||||
Direction direction = 3;
|
||||
}
|
||||
|
||||
message ContentFilter {
|
||||
string contentTopic = 1;
|
||||
}
|
||||
|
||||
message HistoryQuery {
|
||||
// the first field is reserved for future use
|
||||
string pubsubtopic = 2;
|
||||
repeated ContentFilter contentFilters = 3;
|
||||
PagingInfo pagingInfo = 4;
|
||||
}
|
||||
|
||||
message HistoryResponse {
|
||||
// the first field is reserved for future use
|
||||
repeated WakuMessage messages = 2;
|
||||
PagingInfo pagingInfo = 3;
|
||||
enum Error {
|
||||
NONE = 0;
|
||||
INVALID_CURSOR = 1;
|
||||
}
|
||||
Error error = 4;
|
||||
}
|
||||
|
||||
message HistoryRPC {
|
||||
string request_id = 1;
|
||||
HistoryQuery query = 2;
|
||||
HistoryResponse response = 3;
|
||||
}
|
||||
```
|
||||
|
||||
#### Index
|
||||
|
||||
To perform pagination,
|
||||
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol
|
||||
is associated with a unique `Index` that encapsulates the following parts.
|
||||
|
||||
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
|
||||
The hash is computed over the concatenation of `contentTopic`
|
||||
and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
|
||||
- `receiverTime`: the UNIX time in nanoseconds
|
||||
at which the `WakuMessage` is received by the receiving node.
|
||||
- `senderTime`: the UNIX time in nanoseconds
|
||||
at which the `WakuMessage` is generated by its sender.
|
||||
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
|
||||
|
||||
#### PagingInfo
|
||||
|
||||
`PagingInfo` holds the information required for pagination.
|
||||
It consists of the following components.
|
||||
|
||||
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s
|
||||
in a `HistoryQuery`
|
||||
(or retrieved `WakuMessage`s in a `HistoryResponse`).
|
||||
- `cursor`: holds the `Index` of a `WakuMessage`.
|
||||
- `direction`: indicates the direction of paging
|
||||
which can be either `FORWARD` or `BACKWARD`.
|
||||
|
||||
#### ContentFilter
|
||||
|
||||
`ContentFilter` carries the information required for filtering historical messages.
|
||||
|
||||
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
|
||||
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
|
||||
|
||||
#### HistoryQuery
|
||||
|
||||
RPC call to query historical messages.
|
||||
|
||||
- The `pubsubTopic` field MUST indicate the pubsub topic
|
||||
of the historical messages to be retrieved.
|
||||
This field denotes the pubsub topic on which `WakuMessage`s are published.
|
||||
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
|
||||
Leaving this field empty means no filter on the pubsub topic
|
||||
of message history is requested.
|
||||
This field SHOULD be left empty in order to retrieve the historical `WakuMessage`
|
||||
regardless of the pubsub topics on which they are published.
|
||||
- The `contentFilters` field MUST indicate the list of content filters
|
||||
based on which the historical messages are to be retrieved.
|
||||
Leaving this field empty means no filter on the content topic
|
||||
of message history is required.
|
||||
This field SHOULD be left empty in order
|
||||
to retrieve historical `WakuMessage` regardless of their content topics.
|
||||
- `PagingInfo` holds the information required for pagination.
|
||||
Its `pageSize` field indicates the number of `WakuMessage`s
|
||||
to be included in the corresponding `HistoryResponse`.
|
||||
It is RECOMMENDED that the queried node defines a maximum page size internally.
|
||||
If the querying node leaves the `pageSize` unspecified,
|
||||
or if the `pageSize` exceeds the maximum page size,
|
||||
the queried node SHOULD auto-paginate the `HistoryResponse`
|
||||
to no more than the configured maximum page size.
|
||||
This allows mitigation of long response time for `HistoryQuery`.
|
||||
In the forward pagination request,
|
||||
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
|
||||
the `pageSize` amount of `WakuMessage` whose `Index`
|
||||
values are larger than the given `cursor`
|
||||
(and vise versa for the backward pagination).
|
||||
Note that the `cursor` of a `HistoryQuery` MAY be empty
|
||||
(e.g., for the initial query), as such, and
|
||||
depending on whether the `direction` is `BACKWARD` or
|
||||
`FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned,
|
||||
respectively.
|
||||
|
||||
#### Sorting Messages
|
||||
|
||||
The queried node MUST sort the `WakuMessage` based on their `Index`,
|
||||
where the `senderTime` constitutes the most significant part and
|
||||
the `digest` comes next, and
|
||||
then perform pagination on the sorted result.
|
||||
As such, the retrieved page contains an ordered list of `WakuMessage`
|
||||
from the oldest messages to the most recent one.
|
||||
Alternatively, the `receiverTime` (instead of `senderTime`)
|
||||
MAY be used to sort messages during the paging process.
|
||||
However, it is RECOMMENDED the use of the `senderTime`
|
||||
for sorting as it is invariant and
|
||||
consistent across all the nodes.
|
||||
This has the benefit of `cursor` reusability i.e.,
|
||||
a `cursor` obtained from one node can be consistently used
|
||||
to query from another node.
|
||||
However, this `cursor` reusability does not hold when the `receiverTime` is utilized
|
||||
as the receiver time is affected by the network delay and
|
||||
nodes' clock asynchrony.
|
||||
|
||||
#### HistoryResponse
|
||||
|
||||
RPC call to respond to a HistoryQuery call.
|
||||
|
||||
- The `messages` field MUST contain the messages found,
|
||||
these are [14/WAKU2-MESSAGE](../14/message.md) types.
|
||||
- `PagingInfo` holds the paging information based
|
||||
on which the querying node can resume its further history queries.
|
||||
The `pageSize` indicates the number of returned Waku messages
|
||||
(i.e., the number of messages included in the `messages` field of `HistoryResponse`).
|
||||
The `direction` is the same direction as in the corresponding `HistoryQuery`.
|
||||
In the forward pagination, the `cursor` holds the `Index` of the last message
|
||||
in the `HistoryResponse` `messages` (and the first message in the backward paging).
|
||||
Regardless of the paging direction,
|
||||
the retrieved `messages` are always sorted in ascending order
|
||||
based on their timestamp as explained in the [sorting messages](#sorting-messages)section,
|
||||
that is, from the oldest to the most recent.
|
||||
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery`
|
||||
to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).
|
||||
The `cursor` obtained from one node SHOULD NOT be used in a request to another node
|
||||
because the result may be different.
|
||||
- The `error` field contains information about any error that has occurred
|
||||
while processing the corresponding `HistoryQuery`.
|
||||
`NONE` stands for no error.
|
||||
This is also the default value.
|
||||
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery`
|
||||
does not match with the `Index` of any of the `WakuMessage`
|
||||
persisted by the queried node.
|
||||
|
||||
## Security Consideration
|
||||
|
||||
The main security consideration to take into account
|
||||
while using this protocol is that a querying node
|
||||
have to reveal their content filters of interest to the queried node,
|
||||
hence potentially compromising their privacy.
|
||||
|
||||
## Future Work
|
||||
|
||||
- **Anonymous query**: This feature guarantees that nodes
|
||||
can anonymously query historical messages from other nodes i.e.,
|
||||
without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md)
|
||||
they are interested in.
|
||||
As such, no adversary in the `13/WAKU2-STORE` protocol
|
||||
would be able to learn which peer is interested in which content filters i.e.,
|
||||
content topics of [14/WAKU2-MESSAGE](../14/message.md).
|
||||
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity
|
||||
for historical queries,
|
||||
as the querying node needs to directly connect to another node
|
||||
in the `13/WAKU2-STORE` protocol and
|
||||
explicitly disclose the content filters of its interest
|
||||
to retrieve the corresponding messages.
|
||||
However, one can consider preserving anonymity through one of the following ways:
|
||||
- By hiding the source of the request i.e., anonymous communication.
|
||||
That is the querying node shall hide all its PII in its history request
|
||||
e.g., its IP address.
|
||||
This can happen by the utilization of a proxy server or by using Tor.
|
||||
Note that the current structure of historical requests
|
||||
does not embody any piece of PII, otherwise,
|
||||
such data fields must be treated carefully to achieve query anonymity.
|
||||
<!-- TODO: if nodes have to disclose their PeerIDs
|
||||
(e.g., for authentication purposes) when connecting to other nodes
|
||||
in the store protocol,
|
||||
then Tor does not preserve anonymity since it only helps in hiding the IP.
|
||||
So, the PeerId usage in switches must be investigated further.
|
||||
Depending on how PeerId is used, one may be able to link between a querying node
|
||||
and its queried topics despite hiding the IP address-->
|
||||
- By deploying secure 2-party computations in which the querying node
|
||||
obtains the historical messages of a certain topic,
|
||||
the queried node learns nothing about the query.
|
||||
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
|
||||
<!-- TODO: add a reference for PSIs? -->
|
||||
<!-- TODO: more techniques to be included -->
|
||||
<!-- TODO: Censorship resistant:
|
||||
this is about a node that hides the historical messages from other nodes.
|
||||
This attack is not included in the specs
|
||||
since it does not fit the passive adversarial model
|
||||
(the attacker needs to deviate from the store protocol).-->
|
||||
|
||||
- **Robust and verifiable timestamps**:
|
||||
Messages timestamp is a way to show that the message existed
|
||||
prior to some point in time.
|
||||
However, the lack of timestamp verifiability can create room for a range of attacks,
|
||||
including injecting messages with invalid timestamps pointing to the far future.
|
||||
To better understand the attack,
|
||||
consider a store node whose current clock shows `2021-01-01 00:00:30`
|
||||
(and assume all the other nodes have a synchronized clocks +-20seconds).
|
||||
The store node already has a list of messages,
|
||||
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
|
||||
that are sorted based on their timestamp.
|
||||
An attacker sends a message with an arbitrary large timestamp e.g.,
|
||||
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
|
||||
The store node places `m'` at the end of the list,
|
||||
|
||||
```text
|
||||
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),(m',2021-01-01 10:00:30).
|
||||
```
|
||||
|
||||
Now another message arrives with a valid timestamp e.g.,
|
||||
`(m11, 2021-01-01 00:00:45)`.
|
||||
However, since its timestamp precedes the malicious message `m'`,
|
||||
it gets placed before `m'` in the list i.e.,
|
||||
|
||||
```text
|
||||
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30).
|
||||
```
|
||||
|
||||
In fact, for the next 10 hours,
|
||||
`m'` will always be considered as the most recent message and
|
||||
served as the last message to the querying nodes irrespective
|
||||
of how many other messages arrive afterward.
|
||||
|
||||
A robust and verifiable timestamp allows the receiver of a message
|
||||
to verify that a message has been generated prior to the claimed timestamp.
|
||||
One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
|
||||
block height in Blockchain-based timestamps.
|
||||
That is, messages contain the most recent block height
|
||||
perceived by their senders at the time of message generation.
|
||||
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
|
||||
seconds (e.g., in Ethereum 2.0) from the time of origination.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
1. [14/WAKU2-MESSAGE](../14/message.md)
|
||||
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
|
||||
3. [11/WAKU2-RELAY](../11/relay.md)
|
||||
4. [Open timestamps](https://opentimestamps.org/)
|
||||
@@ -1,311 +1,403 @@
|
||||
---
|
||||
slug: 13
|
||||
title: 13/WAKU2-STORE
|
||||
name: Waku v2 Store
|
||||
name: Waku Store Query
|
||||
status: draft
|
||||
tags: waku-core
|
||||
editor: Simon-Pierre Vivier <simvivier@status.im>
|
||||
version: 01
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Aaryamann Challani <p1ge0nh8er@proton.me>
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
- Hanno Cornelius <hanno@status.im>
|
||||
---
|
||||
Previous version: [00](/waku/standards/core/13/previous-versions/00/store.md)
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification explains the `13/WAKU2-STORE` protocol
|
||||
which enables querying of messages received through the relay protocol and
|
||||
stored by other nodes.
|
||||
It also supports pagination for more efficient querying of historical messages.
|
||||
This specification explains the `WAKU2-STORE` protocol,
|
||||
which enables querying of [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)s.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4`
|
||||
**Protocol identifier***: `/vac/waku/store-query/3.0.0`
|
||||
|
||||
## Terminology
|
||||
### Terminology
|
||||
|
||||
The term PII, Personally Identifiable Information,
|
||||
refers to any piece of data that can be used to uniquely identify a user.
|
||||
For example, the signature verification key, and
|
||||
the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
## Design Requirements
|
||||
## Wire Specification
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
|
||||
SHOULD provide a complete and full view of message history.
|
||||
As such, they are required to be *highly available* and
|
||||
specifically have a *high uptime* to consistently receive and store network messages.
|
||||
The high uptime requirement makes sure that no message is missed out
|
||||
hence a complete and intact view of the message history
|
||||
is delivered to the querying nodes.
|
||||
Nevertheless, in case storage provider nodes cannot afford high availability,
|
||||
the querying nodes may retrieve the historical messages from multiple sources
|
||||
to achieve a full and intact view of the past.
|
||||
### Design Requirements
|
||||
|
||||
The concept of `ephemeral` messages introduced in
|
||||
[`14/WAKU2-MESSAGE`](../14/message.md) affects `13/WAKU2-STORE` as well.
|
||||
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in
|
||||
[14/WAKU2-MESSAGE](../14/message.md).
|
||||
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages
|
||||
with the `ephemeral` flag set to `true`.
|
||||
|
||||
## Adversarial Model
|
||||
|
||||
Any peer running the `13/WAKU2-STORE` protocol, i.e.
|
||||
both the querying node and the queried node, are considered as an adversary.
|
||||
Furthermore,
|
||||
we currently consider the adversary as a passive entity
|
||||
that attempts to collect information from other peers to conduct an attack but
|
||||
it does so without violating protocol definitions and instructions.
|
||||
As we evolve the protocol,
|
||||
further adversarial models will be considered.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node hides or
|
||||
lies about the history of messages
|
||||
as it is against the description of the `13/WAKU2-STORE` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links
|
||||
between arbitrary pairs of peers (unless the adversary is one end of the communication).
|
||||
In specific, the communication channels are assumed to be secure.
|
||||
|
||||
## Wire Specification
|
||||
|
||||
Peers communicate with each other using a request / response API.
|
||||
The messages sent are Protobuf RPC messages which are implemented using
|
||||
[protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
The following are the specifications of the Protobuf messages.
|
||||
The concept of `ephemeral` messages introduced in [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) affects `WAKU2-STORE` as well.
|
||||
Nodes running `WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md).
|
||||
Nodes running `WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
|
||||
|
||||
### Payloads
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
message Index {
|
||||
bytes digest = 1;
|
||||
sint64 receiverTime = 2;
|
||||
sint64 senderTime = 3;
|
||||
string pubsubTopic = 4;
|
||||
// Protocol identifier: /vac/waku/store-query/3.0.0
|
||||
package waku.store.v3;
|
||||
|
||||
import "waku/message/v1/message.proto";
|
||||
|
||||
message WakuMessageKeyValue {
|
||||
optional bytes message_hash = 1; // Globally unique key for a Waku Message
|
||||
|
||||
// Full message content and associated pubsub_topic as value
|
||||
optional waku.message.v1.WakuMessage message = 2;
|
||||
optional string pubsub_topic = 3;
|
||||
}
|
||||
|
||||
message PagingInfo {
|
||||
uint64 pageSize = 1;
|
||||
Index cursor = 2;
|
||||
enum Direction {
|
||||
BACKWARD = 0;
|
||||
FORWARD = 1;
|
||||
}
|
||||
Direction direction = 3;
|
||||
}
|
||||
|
||||
message ContentFilter {
|
||||
string contentTopic = 1;
|
||||
}
|
||||
|
||||
message HistoryQuery {
|
||||
// the first field is reserved for future use
|
||||
string pubsubtopic = 2;
|
||||
repeated ContentFilter contentFilters = 3;
|
||||
PagingInfo pagingInfo = 4;
|
||||
}
|
||||
|
||||
message HistoryResponse {
|
||||
// the first field is reserved for future use
|
||||
repeated WakuMessage messages = 2;
|
||||
PagingInfo pagingInfo = 3;
|
||||
enum Error {
|
||||
NONE = 0;
|
||||
INVALID_CURSOR = 1;
|
||||
}
|
||||
Error error = 4;
|
||||
}
|
||||
|
||||
message HistoryRPC {
|
||||
message StoreQueryRequest {
|
||||
string request_id = 1;
|
||||
HistoryQuery query = 2;
|
||||
HistoryResponse response = 3;
|
||||
bool include_data = 2; // Response should include full message content
|
||||
|
||||
// Filter criteria for content-filtered queries
|
||||
optional string pubsub_topic = 10;
|
||||
repeated string content_topics = 11;
|
||||
optional sint64 time_start = 12;
|
||||
optional sint64 time_end = 13;
|
||||
|
||||
// List of key criteria for lookup queries
|
||||
repeated bytes message_hashes = 20; // Message hashes (keys) to lookup
|
||||
|
||||
// Pagination info. 50 Reserved
|
||||
optional bytes pagination_cursor = 51; // Message hash (key) from where to start query (exclusive)
|
||||
bool pagination_forward = 52;
|
||||
optional uint64 pagination_limit = 53;
|
||||
}
|
||||
|
||||
message StoreQueryResponse {
|
||||
string request_id = 1;
|
||||
|
||||
optional uint32 status_code = 10;
|
||||
optional string status_desc = 11;
|
||||
|
||||
repeated WakuMessageKeyValue messages = 20;
|
||||
|
||||
optional bytes pagination_cursor = 51;
|
||||
}
|
||||
```
|
||||
|
||||
#### Index
|
||||
### General Store Query Concepts
|
||||
|
||||
To perform pagination,
|
||||
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol
|
||||
is associated with a unique `Index` that encapsulates the following parts.
|
||||
#### Waku Message Key-Value Pairs
|
||||
|
||||
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
|
||||
The hash is computed over the concatenation of `contentTopic`
|
||||
and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](../14/message.md)).
|
||||
- `receiverTime`: the UNIX time in nanoseconds
|
||||
at which the `WakuMessage` is received by the receiving node.
|
||||
- `senderTime`: the UNIX time in nanoseconds
|
||||
at which the `WakuMessage` is generated by its sender.
|
||||
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
|
||||
The store query protocol operates as a query protocol for a key-value store of historical messages,
|
||||
with each entry having a [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
and associated `pubsub_topic` as the value,
|
||||
and [deterministic message hash](/waku/standards/core/14/message.md#deterministic-message-hashing) as the key.
|
||||
The store can be queried to return either a set of keys or a set of key-value pairs.
|
||||
|
||||
#### PagingInfo
|
||||
Within the store query protocol,
|
||||
the [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) keys and
|
||||
values MUST be represented in a `WakuMessageKeyValue` message.
|
||||
This message MUST contain the deterministic `message_hash` as the key.
|
||||
It MAY contain the full [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) and
|
||||
associated pubsub topic as the value in the `message` and
|
||||
`pubsub_topic` fields, depending on the use case as set out below.
|
||||
|
||||
`PagingInfo` holds the information required for pagination.
|
||||
It consists of the following components.
|
||||
If the message contains a value entry in addition to the key,
|
||||
both the `message` and `pubsub_topic` fields MUST be populated.
|
||||
The message MUST NOT have either `message` or `pubsub_topic` populated with the other unset.
|
||||
Both fields MUST either be set or unset.
|
||||
|
||||
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s
|
||||
in a `HistoryQuery`
|
||||
(or retrieved `WakuMessage`s in a `HistoryResponse`).
|
||||
- `cursor`: holds the `Index` of a `WakuMessage`.
|
||||
- `direction`: indicates the direction of paging
|
||||
which can be either `FORWARD` or `BACKWARD`.
|
||||
#### Waku Message Store Eligibility
|
||||
|
||||
#### ContentFilter
|
||||
In order for a message to be eligible for storage:
|
||||
|
||||
`ContentFilter` carries the information required for filtering historical messages.
|
||||
- it MUST be a _valid_ [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md).
|
||||
- the `timestamp` field MUST be populated with the Unix epoch time,
|
||||
at which the message was generated in nanoseconds.
|
||||
If at the time of storage the `timestamp` deviates by more than 20 seconds
|
||||
either into the past or the future when compared to the store node’s internal clock,
|
||||
the store node MAY reject the message.
|
||||
- the `ephemeral` field MUST be set to `false`.
|
||||
|
||||
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
|
||||
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](../14/message.md).
|
||||
|
||||
#### HistoryQuery
|
||||
#### Waku message sorting
|
||||
|
||||
RPC call to query historical messages.
|
||||
The key-value entries in the store MUST be time-sorted by the [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) `timestamp` attribute.
|
||||
Where two or more key-value entries have identical `timestamp` values,
|
||||
the entries MUST be further sorted by the natural order of their message hash keys.
|
||||
Within the context of traversing over key-value entries in the store,
|
||||
_"forward"_ indicates traversing the entries in ascending order,
|
||||
whereas _"backward"_ indicates traversing the entries in descending order.
|
||||
|
||||
- The `pubsubTopic` field MUST indicate the pubsub topic
|
||||
of the historical messages to be retrieved.
|
||||
This field denotes the pubsub topic on which `WakuMessage`s are published.
|
||||
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](../11/relay.md).
|
||||
Leaving this field empty means no filter on the pubsub topic
|
||||
of message history is requested.
|
||||
This field SHOULD be left empty in order to retrieve the historical `WakuMessage`
|
||||
regardless of the pubsub topics on which they are published.
|
||||
- The `contentFilters` field MUST indicate the list of content filters
|
||||
based on which the historical messages are to be retrieved.
|
||||
Leaving this field empty means no filter on the content topic
|
||||
of message history is required.
|
||||
This field SHOULD be left empty in order
|
||||
to retrieve historical `WakuMessage` regardless of their content topics.
|
||||
- `PagingInfo` holds the information required for pagination.
|
||||
Its `pageSize` field indicates the number of `WakuMessage`s
|
||||
to be included in the corresponding `HistoryResponse`.
|
||||
It is RECOMMENDED that the queried node defines a maximum page size internally.
|
||||
If the querying node leaves the `pageSize` unspecified,
|
||||
or if the `pageSize` exceeds the maximum page size,
|
||||
the queried node SHOULD auto-paginate the `HistoryResponse`
|
||||
to no more than the configured maximum page size.
|
||||
This allows mitigation of long response time for `HistoryQuery`.
|
||||
In the forward pagination request,
|
||||
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
|
||||
the `pageSize` amount of `WakuMessage` whose `Index`
|
||||
values are larger than the given `cursor`
|
||||
(and vise versa for the backward pagination).
|
||||
Note that the `cursor` of a `HistoryQuery` MAY be empty
|
||||
(e.g., for the initial query), as such, and
|
||||
depending on whether the `direction` is `BACKWARD` or
|
||||
`FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned,
|
||||
respectively.
|
||||
#### Pagination
|
||||
|
||||
#### Sorting Messages
|
||||
If a large number of entries in the store service node match the query criteria provided in a `StoreQueryRequest`,
|
||||
the client MAY make use of pagination
|
||||
in a chain of store query request and response transactions
|
||||
to retrieve the full response in smaller batches termed _"pages"_.
|
||||
Pagination can be performed either in [a _forward_ or _backward_ direction](#waku-message-sorting).
|
||||
|
||||
The queried node MUST sort the `WakuMessage` based on their `Index`,
|
||||
where the `senderTime` constitutes the most significant part and
|
||||
the `digest` comes next, and
|
||||
then perform pagination on the sorted result.
|
||||
As such, the retrieved page contains an ordered list of `WakuMessage`
|
||||
from the oldest messages to the most recent one.
|
||||
Alternatively, the `receiverTime` (instead of `senderTime`)
|
||||
MAY be used to sort messages during the paging process.
|
||||
However, it is RECOMMENDED the use of the `senderTime`
|
||||
for sorting as it is invariant and
|
||||
consistent across all the nodes.
|
||||
This has the benefit of `cursor` reusability i.e.,
|
||||
a `cursor` obtained from one node can be consistently used
|
||||
to query from another node.
|
||||
However, this `cursor` reusability does not hold when the `receiverTime` is utilized
|
||||
as the receiver time is affected by the network delay and
|
||||
nodes' clock asynchrony.
|
||||
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
|
||||
by setting the page size limit in the `pagination_limit` field.
|
||||
Note that a store service node MAY enforce its own limit
|
||||
if the `pagination_limit` is unset
|
||||
or larger than the service node's internal page size limit.
|
||||
|
||||
#### HistoryResponse
|
||||
A `StoreQueryResponse` with a populated `pagination_cursor` indicates that more stored entries match the query than included in the response.
|
||||
|
||||
RPC call to respond to a HistoryQuery call.
|
||||
A `StoreQueryResponse` without a populated `pagination_cursor` indicates that
|
||||
there are no more matching entries in the store.
|
||||
|
||||
- The `messages` field MUST contain the messages found,
|
||||
these are [14/WAKU2-MESSAGE](../14/message.md) types.
|
||||
- `PagingInfo` holds the paging information based
|
||||
on which the querying node can resume its further history queries.
|
||||
The `pageSize` indicates the number of returned Waku messages
|
||||
(i.e., the number of messages included in the `messages` field of `HistoryResponse`).
|
||||
The `direction` is the same direction as in the corresponding `HistoryQuery`.
|
||||
In the forward pagination, the `cursor` holds the `Index` of the last message
|
||||
in the `HistoryResponse` `messages` (and the first message in the backward paging).
|
||||
Regardless of the paging direction,
|
||||
the retrieved `messages` are always sorted in ascending order
|
||||
based on their timestamp as explained in the [sorting messages](#sorting-messages)section,
|
||||
that is, from the oldest to the most recent.
|
||||
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery`
|
||||
to retrieve the next page of the [14/WAKU2-MESSAGE](../14/message.md).
|
||||
The `cursor` obtained from one node SHOULD NOT be used in a request to another node
|
||||
because the result may be different.
|
||||
- The `error` field contains information about any error that has occurred
|
||||
while processing the corresponding `HistoryQuery`.
|
||||
`NONE` stands for no error.
|
||||
This is also the default value.
|
||||
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery`
|
||||
does not match with the `Index` of any of the `WakuMessage`
|
||||
persisted by the queried node.
|
||||
The client MAY request the next page of entries from the store service node
|
||||
by populating a subsequent `StoreQueryRequest` with the `pagination_cursor`
|
||||
received in the `StoreQueryResponse`.
|
||||
All other fields and query criteria MUST be the same as in the preceding `StoreQueryRequest`.
|
||||
|
||||
## Security Consideration
|
||||
A `StoreQueryRequest` without a populated `pagination_cursor` indicates that
|
||||
the client wants to retrieve the "first page" of the stored entries matching the query.
|
||||
|
||||
The main security consideration to take into account
|
||||
while using this protocol is that a querying node
|
||||
have to reveal their content filters of interest to the queried node,
|
||||
### Store Query Request
|
||||
|
||||
A client node MUST send all historical message queries within a `StoreQueryRequest` message.
|
||||
This request MUST contain a `request_id`.
|
||||
The `request_id` MUST be a uniquely generated string.
|
||||
|
||||
If the store query client requires the store service node to include [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) values in the query response,
|
||||
it MUST set `include_data` to `true`.
|
||||
If the store query client requires the store service node to return only message hash keys in the query response,
|
||||
it SHOULD set `include_data` to `false`.
|
||||
By default, therefore, the store service node assumes `include_data` to be `false`.
|
||||
|
||||
A store query client MAY include query filter criteria in the `StoreQueryRequest`.
|
||||
There are two types of filter use cases:
|
||||
|
||||
1. Content filtered queries and
|
||||
2. Message hash lookup queries
|
||||
|
||||
#### Content filtered queries
|
||||
|
||||
A store query client MAY request the store service node to filter historical entries by a content filter.
|
||||
Such a client MAY create a filter on content topic, on time range or on both.
|
||||
|
||||
To filter on content topic,
|
||||
the client MUST populate _both_ the `pubsub_topic` _and_ `content_topics` field.
|
||||
The client MUST NOT populate either `pubsub_topic` or
|
||||
`content_topics` and leave the other unset.
|
||||
Both fields MUST either be set or unset.
|
||||
A mixed content topic filter with just one of either `pubsub_topic` or
|
||||
`content_topics` set, SHOULD be regarded as an invalid request.
|
||||
|
||||
To filter on time range, the client MUST set `time_start`, `time_end` or both.
|
||||
Each `time_` field should contain a Unix epoch timestamp in nanoseconds.
|
||||
An unset `time_start` SHOULD be interpreted as "from the oldest stored entry".
|
||||
An unset `time_end` SHOULD be interpreted as "up to the youngest stored entry".
|
||||
|
||||
If any of the content filter fields are set,
|
||||
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`,
|
||||
the client MUST NOT set the `message_hashes` field.
|
||||
|
||||
#### Message hash lookup queries
|
||||
|
||||
A store query client MAY request the store service node to filter historical entries by one or
|
||||
more matching message hash keys.
|
||||
This type of query acts as a "lookup" against a message hash key or
|
||||
set of keys already known to the client.
|
||||
|
||||
In order to perform a lookup query,
|
||||
the store query client MUST populate the `message_hashes` field with the list of message hash keys it wants to lookup in the store service node.
|
||||
|
||||
If the `message_hashes` field is set,
|
||||
the client MUST NOT set any of the content filter fields,
|
||||
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`.
|
||||
|
||||
#### Presence queries
|
||||
|
||||
A presence query is a special type of lookup query that allows a client to check for the presence of one or
|
||||
more messages in the store service node,
|
||||
without retrieving the full contents (values) of the messages.
|
||||
This can, for example, be used as part of a reliability mechanism,
|
||||
whereby store query clients verify that previously published messages have been successfully stored.
|
||||
|
||||
In order to perform a presence query,
|
||||
the store query client MUST populate the `message_hashes` field in the `StoreQueryRequest` with the list of message hashes
|
||||
for which it wants to verify presence in the store service node.
|
||||
The `include_data` property MUST be set to `false`.
|
||||
The client SHOULD interpret every `message_hash` returned in the `messages` field of the `StoreQueryResponse` as present in the store.
|
||||
The client SHOULD assume that all other message hashes included in the original `StoreQueryRequest` but
|
||||
not in the `StoreQueryResponse` is not present in the store.
|
||||
|
||||
#### Pagination info
|
||||
|
||||
The store query client MAY include a message hash as `pagination_cursor`,
|
||||
to indicate at which key-value entry a store service node SHOULD start the query.
|
||||
The `pagination_cursor` is treated as exclusive
|
||||
and the corresponding entry will not be included in subsequent store query responses.
|
||||
|
||||
For forward queries,
|
||||
only messages following (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor`
|
||||
will be returned.
|
||||
For backward queries,
|
||||
only messages preceding (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor`
|
||||
will be returned.
|
||||
|
||||
If the store query client requires the store service node to perform a forward query,
|
||||
it MUST set `pagination_forward` to `true`.
|
||||
If the store query client requires the store service node to perform a backward query,
|
||||
it SHOULD set `pagination_forward` to `false`.
|
||||
By default, therefore, the store service node assumes pagination to be backward.
|
||||
|
||||
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
|
||||
by setting the page size limit in the `pagination_limit` field.
|
||||
Note that a store service node MAY enforce its own limit
|
||||
if the `pagination_limit` is unset
|
||||
or larger than the service node's internal page size limit.
|
||||
|
||||
See [pagination](#pagination) for more on how the pagination info is used in store transactions.
|
||||
|
||||
### Store Query Response
|
||||
|
||||
In response to any `StoreQueryRequest`,
|
||||
a store service node SHOULD respond with a `StoreQueryResponse` with a `requestId` matching that of the request.
|
||||
This response MUST contain a `status_code` indicating if the request was successful or not.
|
||||
Successful status codes are in the `2xx` range.
|
||||
A client node SHOULD consider all other status codes as error codes and
|
||||
assume that the requested operation had failed.
|
||||
In addition,
|
||||
the store service node MAY choose to provide a more detailed status description in the `status_desc` field.
|
||||
|
||||
#### Filter matching
|
||||
|
||||
For [content filtered queries](#content-filtered-queries),
|
||||
an entry in the store service node matches the filter criteria in a `StoreQueryRequest` if each of the following conditions are met:
|
||||
|
||||
- its `content_topic` is in the request `content_topics` set
|
||||
and it was published on a matching `pubsub_topic` OR the request `content_topics` and
|
||||
`pubsub_topic` fields are unset
|
||||
- its `timestamp` is _larger or equal_ than the request `start_time` OR the request `start_time` is unset
|
||||
- its `timestamp` is _smaller_ than the request `end_time` OR the request `end_time` is unset
|
||||
|
||||
Note that for content filtered queries, `start_time` is treated as _inclusive_ and
|
||||
`end_time` is treated as _exclusive_.
|
||||
|
||||
For [message hash lookup queries](#message-hash-lookup-queries),
|
||||
an entry in the store service node matches the filter criteria if its `message_hash` is in the request `message_hashes` set.
|
||||
|
||||
The store service node SHOULD respond with an error code and
|
||||
discard the request if the store query request contains both content filter criteria
|
||||
and message hashes.
|
||||
|
||||
#### Populating response messages
|
||||
|
||||
The store service node SHOULD populate the `messages` field in the response
|
||||
only with entries matching the filter criteria provided in the corresponding request.
|
||||
Regardless of whether the response is to a _forward_ or _backward_ query,
|
||||
the `messages` field in the response MUST be ordered in a forward direction
|
||||
according to the [message sorting rules](#waku-message-sorting).
|
||||
|
||||
If the corresponding `StoreQueryRequest` has `include_data` set to true,
|
||||
the service node SHOULD populate both the `message_hash` and
|
||||
`message` for each entry in the response.
|
||||
In all other cases,
|
||||
the store service node SHOULD populate only the `message_hash` field for each entry in the response.
|
||||
|
||||
#### Paginating the response
|
||||
|
||||
The response SHOULD NOT contain more `messages` than the `pagination_limit` provided in the corresponding `StoreQueryRequest`.
|
||||
It is RECOMMENDED that the store node defines its own maximum page size internally.
|
||||
If the `pagination_limit` in the request is unset,
|
||||
or exceeds this internal maximum page size,
|
||||
the store service node SHOULD ignore the `pagination_limit` field and
|
||||
apply its own internal maximum page size.
|
||||
|
||||
In response to a _forward_ `StoreQueryRequest`:
|
||||
|
||||
- if the `pagination_cursor` is set,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries following the `pagination_cursor` (exclusive).
|
||||
- if the `pagination_cursor` is unset,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries from the first entry in the store.
|
||||
- if there are still more matching entries in the store
|
||||
after the maximum page size is reached while populating the response,
|
||||
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
|
||||
with the message hash key of the _last_ entry _included_ in the response.
|
||||
|
||||
In response to a _backward_ `StoreQueryRequest`:
|
||||
|
||||
- if the `pagination_cursor` is set,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries preceding the `pagination_cursor` (exclusive).
|
||||
- if the `pagination_cursor` is unset,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries from the last entry in the store.
|
||||
- if there are still more matching entries in the store
|
||||
after the maximum page size is reached while populating the response,
|
||||
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
|
||||
with the message hash key of the _first_ entry _included_ in the response.
|
||||
|
||||
### Security Consideration
|
||||
|
||||
The main security consideration while using this protocol is that a querying node has to reveal its content filters of interest to the queried node,
|
||||
hence potentially compromising their privacy.
|
||||
|
||||
## Future Work
|
||||
#### Adversarial Model
|
||||
|
||||
- **Anonymous query**: This feature guarantees that nodes
|
||||
can anonymously query historical messages from other nodes i.e.,
|
||||
without disclosing the exact topics of [14/WAKU2-MESSAGE](../14/message.md)
|
||||
they are interested in.
|
||||
As such, no adversary in the `13/WAKU2-STORE` protocol
|
||||
would be able to learn which peer is interested in which content filters i.e.,
|
||||
content topics of [14/WAKU2-MESSAGE](../14/message.md).
|
||||
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity
|
||||
for historical queries,
|
||||
as the querying node needs to directly connect to another node
|
||||
in the `13/WAKU2-STORE` protocol and
|
||||
explicitly disclose the content filters of its interest
|
||||
to retrieve the corresponding messages.
|
||||
Any peer running the `WAKU2-STORE` protocol, i.e.
|
||||
both the querying node and the queried node, are considered as an adversary.
|
||||
Furthermore,
|
||||
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
|
||||
it does so without violating protocol definitions and instructions.
|
||||
As we evolve the protocol,
|
||||
further adversarial models will be considered.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node hides or
|
||||
lies about the history of messages as it is against the description of the `WAKU2-STORE` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
|
||||
Specifically, the communication channels are assumed to be secure.
|
||||
|
||||
### Future Work
|
||||
|
||||
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
|
||||
without disclosing the exact topics of [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) they are interested in.
|
||||
As such, no adversary in the `WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
|
||||
content topics of [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md).
|
||||
The current version of the `WAKU2-STORE` protocol does not provide anonymity for historical queries,
|
||||
as the querying node needs to directly connect to another node in the `WAKU2-STORE` protocol and
|
||||
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
|
||||
However, one can consider preserving anonymity through one of the following ways:
|
||||
- By hiding the source of the request i.e., anonymous communication.
|
||||
That is the querying node shall hide all its PII in its history request
|
||||
e.g., its IP address.
|
||||
This can happen by the utilization of a proxy server or by using Tor.
|
||||
Note that the current structure of historical requests
|
||||
does not embody any piece of PII, otherwise,
|
||||
such data fields must be treated carefully to achieve query anonymity.
|
||||
<!-- TODO: if nodes have to disclose their PeerIDs
|
||||
(e.g., for authentication purposes) when connecting to other nodes
|
||||
in the store protocol,
|
||||
then Tor does not preserve anonymity since it only helps in hiding the IP.
|
||||
So, the PeerId usage in switches must be investigated further.
|
||||
Depending on how PeerId is used, one may be able to link between a querying node
|
||||
and its queried topics despite hiding the IP address-->
|
||||
- By deploying secure 2-party computations in which the querying node
|
||||
obtains the historical messages of a certain topic,
|
||||
the queried node learns nothing about the query.
|
||||
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
|
||||
<!-- TODO: add a reference for PSIs? -->
|
||||
<!-- TODO: more techniques to be included -->
|
||||
|
||||
- By hiding the source of the request i.e., anonymous communication.
|
||||
That is the querying node shall hide all its PII in its history request e.g.,
|
||||
its IP address.
|
||||
This can happen by the utilization of a proxy server or by using Tor.
|
||||
Note that the current structure of historical requests does not embody any piece of PII, otherwise,
|
||||
such data fields must be treated carefully to achieve query anonymity.
|
||||
<!-- TODO: if nodes have to disclose their PeerIDs
|
||||
(e.g., for authentication purposes) when connecting to other nodes in the store protocol,
|
||||
then Tor does not preserve anonymity since it only helps in hiding the IP.
|
||||
So, the PeerId usage in switches must be investigated further.
|
||||
Depending on how PeerId is used, one may be able to link between a querying node
|
||||
and its queried topics despite hiding the IP address-->
|
||||
- By deploying secure 2-party computations
|
||||
in which the querying node obtains the historical messages of a certain topic,
|
||||
the queried node learns nothing about the query.
|
||||
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
|
||||
<!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included -->
|
||||
<!-- TODO: Censorship resistant:
|
||||
this is about a node that hides the historical messages from other nodes.
|
||||
This attack is not included in the specs
|
||||
since it does not fit the passive adversarial model
|
||||
(the attacker needs to deviate from the store protocol).-->
|
||||
This attack is not included in the specs since it does not fit the
|
||||
passive adversarial model (the attacker needs to deviate from the store protocol).-->
|
||||
|
||||
- **Robust and verifiable timestamps**:
|
||||
Messages timestamp is a way to show that the message existed
|
||||
prior to some point in time.
|
||||
- **Robust and verifiable timestamps**: Messages timestamp is a way to show that
|
||||
the message existed prior to some point in time.
|
||||
However, the lack of timestamp verifiability can create room for a range of attacks,
|
||||
including injecting messages with invalid timestamps pointing to the far future.
|
||||
To better understand the attack,
|
||||
@@ -317,31 +409,25 @@ that are sorted based on their timestamp.
|
||||
An attacker sends a message with an arbitrary large timestamp e.g.,
|
||||
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
|
||||
The store node places `m'` at the end of the list,
|
||||
|
||||
```text
|
||||
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),(m',2021-01-01 10:00:30).
|
||||
```
|
||||
|
||||
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),
|
||||
(m',2021-01-01 10:00:30)`.
|
||||
Now another message arrives with a valid timestamp e.g.,
|
||||
`(m11, 2021-01-01 00:00:45)`.
|
||||
However, since its timestamp precedes the malicious message `m'`,
|
||||
it gets placed before `m'` in the list i.e.,
|
||||
|
||||
```text
|
||||
(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30).
|
||||
```
|
||||
|
||||
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20),
|
||||
(m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
|
||||
In fact, for the next 10 hours,
|
||||
`m'` will always be considered as the most recent message and
|
||||
served as the last message to the querying nodes irrespective
|
||||
of how many other messages arrive afterward.
|
||||
served as the last message to the querying nodes irrespective of how many other
|
||||
messages arrive afterward.
|
||||
|
||||
A robust and verifiable timestamp allows the receiver of a message
|
||||
to verify that a message has been generated prior to the claimed timestamp.
|
||||
A robust and verifiable timestamp allows the receiver of a message to verify that
|
||||
a message has been generated prior to the claimed timestamp.
|
||||
One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
|
||||
block height in Blockchain-based timestamps.
|
||||
That is, messages contain the most recent block height
|
||||
perceived by their senders at the time of message generation.
|
||||
That is, messages contain the most recent block height perceived by their senders
|
||||
at the time of message generation.
|
||||
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
|
||||
seconds (e.g., in Ethereum 2.0) from the time of origination.
|
||||
|
||||
@@ -352,7 +438,6 @@ Copyright and related rights waived via
|
||||
|
||||
## References
|
||||
|
||||
1. [14/WAKU2-MESSAGE](../14/message.md)
|
||||
1. [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
|
||||
3. [11/WAKU2-RELAY](../11/relay.md)
|
||||
4. [Open timestamps](https://opentimestamps.org/)
|
||||
3. [Open timestamps](https://opentimestamps.org/)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
slug: 14
|
||||
title: 14/WAKU2-MESSAGE
|
||||
name: Waku v2 Message
|
||||
status: draft
|
||||
status: stable
|
||||
category: Standards Track
|
||||
tags: waku/core-protocol
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
@@ -16,15 +16,17 @@ contributors:
|
||||
|
||||
## Abstract
|
||||
|
||||
Waku v2 is a family of modular peer-to-peer protocols for secure communication.
|
||||
[10/WAKU2](/waku/standards/core/10/waku2.md) is a family of modular peer-to-peer protocols
|
||||
for secure communication.
|
||||
These protocols are designed to be secure, privacy-preserving,
|
||||
and censorship-resistant and can run in resource-restricted environments.
|
||||
At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and
|
||||
At a high level,
|
||||
[10/WAKU2](/waku/standards/core/10/waku2.md) implements a publish/subscribe messaging pattern over libp2p and
|
||||
adds capabilities.
|
||||
|
||||
The present document specifies the Waku v2 message format,
|
||||
a way to encapsulate the messages sent with specific information security goals,
|
||||
and Whisper/Waku v1 backward compatibility.
|
||||
The present document specifies the [10/WAKU2](/waku/standards/core/10/waku2.md) message format.
|
||||
A way to encapsulate the messages sent with specific information security goals,
|
||||
and Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) backward compatibility.
|
||||
|
||||
## Motivation
|
||||
|
||||
@@ -32,7 +34,7 @@ When sending messages over Waku, there are multiple requirements:
|
||||
|
||||
- One may have a separate encryption layer as part of the application.
|
||||
- One may want to provide efficient routing for resource-restricted devices.
|
||||
- One may want to provide compatibility with [Waku v1 envelopes](../../legacy/6/waku1.md).
|
||||
- One may want to provide compatibility with [6/WAKU1](/waku/standards/legacy/6/waku1.md) envelopes.
|
||||
- One may want encrypted payloads by default.
|
||||
- One may want to provide unlinkability to get metadata protection.
|
||||
|
||||
@@ -40,16 +42,20 @@ This specification attempts to provide for these various requirements.
|
||||
|
||||
## Semantics
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”,
|
||||
“SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
### Waku Message
|
||||
|
||||
A Waku message is constituted by the combination of data payload and
|
||||
A `WakuMessage` is constituted by the combination of data payload and
|
||||
attributes that, for example, a *publisher* sends to a *topic* and
|
||||
is eventually delivered to *subscribers*.
|
||||
|
||||
Waku message attributes are key-value pairs of metadata associated with a message.
|
||||
And the message data payload is the part of the transmitted Waku message
|
||||
The `WakuMessage` attributes are key-value pairs of metadata associated with a message.
|
||||
The message data payload is the part of the transmitted `WakuMessage`
|
||||
that is the actual message information.
|
||||
The data payload is also treated as a Waku message attribute for convenience.
|
||||
The data payload is also treated as a `WakuMessage` attribute for convenience.
|
||||
|
||||
### Message Attributes
|
||||
|
||||
@@ -57,13 +63,13 @@ The data payload is also treated as a Waku message attribute for convenience.
|
||||
|
||||
- The `content_topic` attribute MUST specify a string identifier
|
||||
that can be used for content-based filtering,
|
||||
as described in [23/WAKU2-TOPICS](../../../informational/23/topics.md).
|
||||
as described in [23/WAKU2-TOPICS](/waku/informational/23/topics.md).
|
||||
|
||||
- The `meta` attribute, if present,
|
||||
contains an arbitrary application-specific variable-length byte array
|
||||
with a maximum length limit of 64 bytes.
|
||||
This attribute can be utilized to convey supplementary details
|
||||
to various Waku protocols,
|
||||
to various [10/WAKU2](/waku/standards/core/10/waku2.md) protocols,
|
||||
thereby enabling customized processing based on its contents.
|
||||
|
||||
- The `version` attribute, if present,
|
||||
@@ -75,15 +81,18 @@ signifies the time at which the message was generated by its sender.
|
||||
This attribute MAY contain the Unix epoch time in nanoseconds.
|
||||
If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
|
||||
|
||||
- The `rate_limit_proof` attribute, if present,
|
||||
contains a rate limit proof encoded as per [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md).
|
||||
|
||||
- The `ephemeral` attribute, if present, signifies the transient nature of the message.
|
||||
For example, an ephemeral message SHOULD not be persisted by the Waku network.
|
||||
For example, an ephemeral message SHOULD not be persisted by other nodes on the same network.
|
||||
If this attribute is set to `true`, the message SHOULD be interpreted as ephemeral.
|
||||
If, instead, the attribute is omitted or set to `false`,
|
||||
the message SHOULD be interpreted as non-ephemeral.
|
||||
|
||||
## Wire Format
|
||||
|
||||
The Waku message wire format is specified using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
The `WakuMessage` wire format is specified using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
@@ -94,6 +103,7 @@ message WakuMessage {
|
||||
optional uint32 version = 3;
|
||||
optional sint64 timestamp = 10;
|
||||
optional bytes meta = 11;
|
||||
optional bytes rate_limit_proof = 21;
|
||||
optional bool ephemeral = 31;
|
||||
}
|
||||
```
|
||||
@@ -102,7 +112,7 @@ An example proto file following this specification can be found [here (vacp2p/wa
|
||||
|
||||
## Payload encryption
|
||||
|
||||
The Waku message payload MAY be encrypted.
|
||||
The `WakuMessage` payload MAY be encrypted.
|
||||
The message `version` attribute indicates
|
||||
the schema used to encrypt the payload data.
|
||||
|
||||
@@ -112,28 +122,28 @@ the schema used to encrypt the payload data.
|
||||
at the application layer.
|
||||
|
||||
- **Version 1:**
|
||||
The payload SHOULD be encrypted using Waku v1 payload encryption specified in [26/WAKU-PAYLOAD](../../application/26/payload.md).
|
||||
The payload SHOULD be encrypted using [6/WAKU1](/waku/standards/legacy/6/waku1.md) payload encryption specified in [26/WAKU-PAYLOAD](/waku/standards/application/26/payload.md).
|
||||
This provides asymmetric and symmetric encryption.
|
||||
The key agreement is performed out of band.
|
||||
And provides an encrypted signature and padding for some form of unlinkability.
|
||||
|
||||
- **Version 2:**
|
||||
The payload SHOULD be encoded according to [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md).
|
||||
Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
|
||||
The Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
|
||||
|
||||
Any `version` value not included in this list is reserved for future specification.
|
||||
And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
|
||||
|
||||
## Whisper/Waku v1 envelope compatibility
|
||||
## Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) envelope compatibility
|
||||
|
||||
Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
|
||||
Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) envelopes are compatible with Waku messages format.
|
||||
|
||||
- Whisper/Waku v1 `topic` field
|
||||
SHOULD be mapped to Waku v2 message's `content_topic` attribute.
|
||||
- Whisper/Waku v1 `data` field SHOULD be mapped to Waku v2 message's `payload` attribute.
|
||||
- Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) `topic` field
|
||||
SHOULD be mapped to Waku message's `content_topic` attribute.
|
||||
- Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) `data` field SHOULD be mapped to Waku message's `payload` attribute.
|
||||
|
||||
Waku v2 implements a pub/sub messaging pattern over libp2p.
|
||||
This makes redundant some Whisper/Waku v1 envelope fields
|
||||
[10/WAKU2](/waku/standards/core/10/waku2.md) implements a publish/subscribe messaging pattern over libp2p.
|
||||
This makes some Whisper/[6/WAKU1](/waku/standards/legacy/6/waku1.md) envelope fields redundant
|
||||
(e.g., `expiry`, `ttl`, `topic`, etc.), so they can be ignored.
|
||||
|
||||
## Deterministic message hashing
|
||||
@@ -144,7 +154,7 @@ and languages.
|
||||
It is also unstable across different builds with schema changes due to unknown fields.
|
||||
|
||||
To overcome this interoperability limitation,
|
||||
a Waku v2 message's hash MUST be computed following this schema:
|
||||
a [10/WAKU2](/waku/standards/core/10/waku2.md) message's hash MUST be computed following this schema:
|
||||
|
||||
```js
|
||||
message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp))
|
||||
@@ -164,7 +174,7 @@ coupled with using a SHA-2 (256-bit) hashing algorithm.
|
||||
|
||||
### Test vectors
|
||||
|
||||
Waku message hash computation (`meta` size of 12 bytes):
|
||||
The `WakuMessage` hash computation (`meta` size of 12 bytes):
|
||||
|
||||
```js
|
||||
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
|
||||
@@ -176,7 +186,7 @@ message.timestamp = 0x175789bfa23f8400
|
||||
message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05
|
||||
```
|
||||
|
||||
Waku message hash computation (`meta` size of 64 bytes):
|
||||
The `WakuMessage` hash computation (`meta` size of 64 bytes):
|
||||
|
||||
```js
|
||||
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
|
||||
@@ -188,7 +198,7 @@ message.timestamp = 0x175789bfa23f8400
|
||||
message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27
|
||||
```
|
||||
|
||||
Waku message hash computation (`meta` attribute not present):
|
||||
The `WakuMessage` hash computation (`meta` attribute not present):
|
||||
|
||||
```js
|
||||
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
|
||||
@@ -200,7 +210,7 @@ message.timestamp = 0x175789bfa23f8400
|
||||
message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8
|
||||
```
|
||||
|
||||
Waku message hash computation (`payload` length 0):
|
||||
The `WakuMessage` hash computation (`payload` length 0):
|
||||
|
||||
```js
|
||||
pubsub_topic = "/waku/2/default-waku/proto" (0x2f77616b752f322f64656661756c742d77616b752f70726f746f)
|
||||
@@ -217,28 +227,29 @@ message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de
|
||||
### Confidentiality, integrity, and authenticity
|
||||
|
||||
The level of confidentiality, integrity, and
|
||||
authenticity of the Waku message payload is discretionary.
|
||||
authenticity of the `WakuMessage` payload is discretionary.
|
||||
Accordingly, the application layer shall utilize the encryption and
|
||||
signature schemes supported by Waku v2 to meet the application-specific privacy needs.
|
||||
signature schemes supported by [10/WAKU2](/waku/standards/core/10/waku2.md),
|
||||
to meet the application-specific privacy needs.
|
||||
|
||||
### Reliability of the `timestamp` attribute
|
||||
|
||||
The Waku message `timestamp` attribute is set by the sender.
|
||||
The message `timestamp` attribute is set by the sender.
|
||||
Therefore, because message timestamps aren’t independently verified,
|
||||
this attribute is prone to exploitation and misuse.
|
||||
It should not solely be relied upon for operations such as message ordering.
|
||||
For example, a malicious actor can arbitrarily set the `timestamp` of a Waku message
|
||||
For example, a malicious actor can arbitrarily set the `timestamp` of a `WakuMessage`
|
||||
to a high value so that it always shows up as the most recent message
|
||||
in a chat application.
|
||||
Applications using Waku messages’ `timestamp` attribute
|
||||
are recommended to use additional methods for more robust message ordering.
|
||||
Applications using [10/WAKU2](/waku/standards/core/10/waku2.md) messages’ `timestamp` attribute
|
||||
are RECOMMENDED to use additional methods for more robust message ordering.
|
||||
An example of how to deal with message ordering against adversarial message timestamps
|
||||
can be found in the Status protocol,
|
||||
see [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering).
|
||||
see [62/STATUS-PAYLOADS](/status/62/payloads.md/#clock-vs-timestamp-and-message-ordering).
|
||||
|
||||
### Reliability of the `ephemeral` attribute
|
||||
|
||||
The Waku message `ephemeral` attribute is set by the sender.
|
||||
The message `ephemeral` attribute is set by the sender.
|
||||
Since there is currently no incentive mechanism
|
||||
for network participants to behave correctly,
|
||||
this attribute is inherently insecure.
|
||||
@@ -251,8 +262,12 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
## References
|
||||
|
||||
- [6/WAKU1](../../legacy/6/waku1.md)
|
||||
- [Google Protocol buffers v3](https://developers.google.com/protocol-buffers/)
|
||||
- [26/WAKU-PAYLOAD](../../application/26/payload.md)
|
||||
- [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
- [6/WAKU1](/waku/standards/legacy/6/waku1.md)
|
||||
- [23/WAKU2-TOPICS](/waku/informational/23/topics.md)
|
||||
- [17/WAKU2-RLN-RELAY](/waku/standards/core/17/rln-relay.md)
|
||||
- [64/WAKU2-NETWORK](/waku/standards/core/64/network.md)
|
||||
- [protocol buffers v3](https://developers.google.com/protocol-buffers/)
|
||||
- [26/WAKU-PAYLOAD](/waku/standards/application/26/payload.md)
|
||||
- [WAKU2-NOISE](https://github.com/waku-org/specs/blob/master/standards/application/noise.md)
|
||||
- [62/STATUS-PAYLOADS](../../../../status/62/payloads.md/#clock-vs-timestamp-and-message-ordering)
|
||||
- [62/STATUS-PAYLOADS](/status/62/payloads.md/#clock-vs-timestamp-and-message-ordering)
|
||||
|
||||
@@ -7,24 +7,38 @@ tags: waku-core
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
---
|
||||
|
||||
A bridge between Waku v1 and Waku v2.
|
||||
## Abstract
|
||||
|
||||
## Bridge
|
||||
This specification describes how [6/WAKU1](/waku/standards/legacy/6/waku1.md)
|
||||
traffic can be used with [10/WAKU2](/waku/standards/core/10/waku2.md) networks.
|
||||
|
||||
## Wire Format
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
|
||||
“SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and
|
||||
“OPTIONAL” in this document are to be interpreted as described in [2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
A bridge requires supporting both Waku versions:
|
||||
|
||||
* Waku v1 - using devp2p RLPx protocol
|
||||
* Waku v2 - using libp2p protocols
|
||||
* [6/WAKU1](/waku/standards/legacy/6/waku1.md) - using devp2p RLPx protocol
|
||||
* [10/WAKU2](/waku/standards/core/10/waku2.md) - using libp2p protocols
|
||||
|
||||
Packets received on the Waku v1 network SHOULD be published just once on the
|
||||
Waku v2 network. More specifically, the bridge SHOULD publish
|
||||
this through the Waku Relay (PubSub domain).
|
||||
## Publishing Packets
|
||||
|
||||
Publishing such packet will require the creation of a new `Message` with a
|
||||
new `WakuMessage` as data field. The `data` and `topic` field from the Waku v1
|
||||
`Envelope` MUST be copied to the `payload` and `contentTopic` fields of the
|
||||
`WakuMessage`. Other fields such as nonce, expiry and ttl will be dropped as
|
||||
they become obsolete in Waku v2.
|
||||
Packets received on [6/WAKU1](/waku/standards/legacy/6/waku1.md) networks
|
||||
SHOULD be published just once on [10/WAKU2](/waku/standards/core/10/waku2.md) networks.
|
||||
More specifically, the bridge SHOULD publish
|
||||
this through [11/WAKU2-RELAY](/waku/standards/core/11/relay.md) (PubSub domain).
|
||||
|
||||
When publishing such packet,
|
||||
the creation of a new `Message` with a new `WakuMessage` as data field is REQUIRED.
|
||||
The `data` and
|
||||
`topic` field, from the [6/WAKU1](/waku/standards/legacy/6/waku1.md) `Envelope`,
|
||||
MUST be copied to the `payload` and `content_topic` fields of the `WakuMessage`.
|
||||
See [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md#wire-format)
|
||||
for message format details.
|
||||
Other fields such as `nonce`, `expiry` and
|
||||
`ttl` will be dropped as they become obsolete in [10/WAKU2](/waku/standards/core/10/waku2.md).
|
||||
|
||||
Before this is done, the usual envelope verification still applies:
|
||||
|
||||
@@ -32,37 +46,54 @@ Before this is done, the usual envelope verification still applies:
|
||||
* PoW verification
|
||||
* Size verification
|
||||
|
||||
Bridging SHOULD occur through the `WakuRelay`, but it MAY also be done on other Waku
|
||||
v2 protocols (e.g. `WakuFilter`). The latter is however not advised as it will
|
||||
increase the complexity of the bridge and because of the
|
||||
[Security Considerations](#security-considerations) explained further below.
|
||||
Bridging SHOULD occur through the [11/WAKU2-RELAY](/waku/standards/core/11/relay.md),
|
||||
but it MAY also be done on other [10/WAKU2](/waku/standards/core/10/waku2.md) protocols
|
||||
(e.g. [12/WAKU2-FILTER](/waku/standards/core/12/filter.md)).
|
||||
The latter is however not advised as it will
|
||||
increase the complexity of the bridge and
|
||||
because of the [Security Considerations](#security-considerations) explained further below.
|
||||
|
||||
Packets received on the Waku v2 network SHOULD be posted just once on the Waku
|
||||
v1 network. The Waku v2 `WakuMessage` contains only the `payload` and
|
||||
`contentTopic` fields. The bridge MUST create a new Waku v1 `Envelope` and
|
||||
copy over the `payload` and `contentFilter` fields to the `data` and `topic`
|
||||
fields. Next, before posting on the network, the bridge MUST set a new expiry
|
||||
and ttl and do the PoW nonce calculation.
|
||||
Packets received on [10/WAKU2](/waku/standards/core/10/waku2.md) networks,
|
||||
SHOULD be posted just once on [6/WAKU1](/waku/standards/legacy/6/waku1.md) networks.
|
||||
The [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md) contains only the `payload` and
|
||||
`contentTopic` fields.
|
||||
The bridge MUST create a new [6/WAKU1](/waku/standards/legacy/6/waku1.md) `Envelope` and
|
||||
copy over the `payload` and `contentFilter`
|
||||
fields to the `data` and `topic` fields.
|
||||
Next, before posting on the network,
|
||||
the bridge MUST set a new `expiry`, `ttl` and do the PoW `nonce` calculation.
|
||||
|
||||
### Security Considerations
|
||||
|
||||
As mentioned above, a bridge will be posting new Waku v1 envelopes, which
|
||||
requires doing the PoW nonce calculation.
|
||||
As mentioned above,
|
||||
a bridge will be posting new [6/WAKU1](/waku/standards/legacy/6/waku1.md) envelopes,
|
||||
which requires doing the PoW `nonce` calculation.
|
||||
|
||||
This could be a DoS attack vector, as the PoW calculation will make it more
|
||||
expensive to post the message compared to the original publishing on the Waku v2
|
||||
network. Low PoW setting will lower this problem, but it is likely that it is
|
||||
still more expensive.
|
||||
This could be a DoS attack vector,
|
||||
as the PoW calculation will make it more expensive to post the message
|
||||
compared to the original publishing on [10/WAKU2](/waku/standards/core/10/waku2.md) networks.
|
||||
Low PoW setting will lower this problem,
|
||||
but it is likely that it is still more expensive.
|
||||
|
||||
For this reason, bridges SHOULD probably be run independently of other nodes, so
|
||||
that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
|
||||
For this reason, it is RECOMMENDED to run bridges independently of other nodes,
|
||||
so that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
|
||||
traffic.
|
||||
|
||||
Bridging functionality SHOULD also be carefully implemented so that messages do
|
||||
not bounce back and forth between the two networks. The bridge SHOULD properly
|
||||
track messages with a seen filter so that no amplification can be achieved here.
|
||||
not bounce back and forth between the [10/WAKU2](/waku/standards/core/10/waku2.md) and
|
||||
[6/WAKU1](/waku/standards/legacy/6/waku1.md) networks.
|
||||
The bridge SHOULD properly track messages with a seen filter,
|
||||
so that no amplification occurs.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
* [6/WAKU1](/waku/standards/legacy/6/waku1.md)
|
||||
* [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
* [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
* [14/WAKU2-MESSAGE](/waku/standards/core/14/message.md)
|
||||
* [12/WAKU2-FILTER](/waku/standards/core/12/filter.md)
|
||||
|
||||
@@ -4,7 +4,8 @@ title: 33/WAKU2-DISCV5
|
||||
name: Waku v2 Discv5 Ambient Peer Discovery
|
||||
status: draft
|
||||
editor: Daniel Kaiser <danielkaiser@status.im>
|
||||
contributors:
|
||||
contributors:
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
@@ -12,7 +13,7 @@ contributors:
|
||||
`33/WAKU2-DISCV5` specifies a modified version of
|
||||
[Ethereum's Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
|
||||
as a means for ambient node discovery.
|
||||
[10/WAKU2](../10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network
|
||||
[10/WAKU2](/waku/standards/core/10/waku2.md) uses the `33/WAKU2-DISCV5` ambient node discovery network
|
||||
for establishing a decentralized network of interconnected Waku2 nodes.
|
||||
In its current version,
|
||||
the `33/WAKU2-DISCV5` discovery network
|
||||
@@ -24,7 +25,7 @@ compared to the total number of Ethereum nodes.
|
||||
## Disclaimer
|
||||
|
||||
This version of `33/WAKU2-DISCV5` has a focus on timely deployment
|
||||
of an efficient discovery method for [10/WAKU2](../10/waku2.md).
|
||||
of an efficient discovery method for [10/WAKU2](/waku/standards/core/10/waku2.md).
|
||||
Establishing a separate discovery network is in line with this focus.
|
||||
However, we are aware of potential resilience problems
|
||||
(see section on security considerations) and
|
||||
@@ -33,11 +34,11 @@ and researching hybrid approaches.
|
||||
|
||||
## Background and Rationale
|
||||
|
||||
[11/WAKU2-RELAY](../11/relay.md) assumes the existence of a network of Waku2 nodes.
|
||||
[11/WAKU2-RELAY](/waku/standards/core/11/relay.md) assumes the existence of a network of Waku2 nodes.
|
||||
For establishing and growing this network,
|
||||
new nodes trying to join the Waku2 network need a means of discovering nodes
|
||||
within the network.
|
||||
[10/WAKU2](../10/waku2.md) supports the following discovery methods
|
||||
new nodes trying to join the Waku2 network
|
||||
need a means of discovering nodes within the network.
|
||||
[10/WAKU2](/waku/standards/core/10/waku2.md) supports the following discovery methods
|
||||
in order of increasing decentralization
|
||||
|
||||
* hard coded bootstrap nodes
|
||||
@@ -45,7 +46,7 @@ in order of increasing decentralization
|
||||
* [`34/WAKU2-PEER-EXCHANGE`](/waku/standards/core/34/peer-exchange.md)
|
||||
* `33/WAKU2-DISCV5` (specified in this document)
|
||||
|
||||
The purpose of ambient node discovery within [10/WAKU2](../10/waku2.md)
|
||||
The purpose of ambient node discovery within [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
is discovering Waku2 nodes in a decentralized way.
|
||||
The unique selling point of `33/WAKU2-DISCV5` is its holistic view of the network,
|
||||
which allows avoiding hotspots and allows merging the network after a split.
|
||||
@@ -62,7 +63,7 @@ during which the querying node was offline.
|
||||
|
||||
`33/WAKU2-DISCV5` spans an overlay network separate from the
|
||||
[GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
|
||||
network [11/WAKU2-RELAY](../11/relay.md) builds on.
|
||||
network [11/WAKU2-RELAY](/waku/standards/core/11/relay.md) builds on.
|
||||
Because it is a P2P network on its own, it also depends on bootstrap nodes.
|
||||
Having a separate discovery network reduces load on the bootstrap nodes,
|
||||
because the actual work is done by randomly discovered nodes.
|
||||
@@ -204,9 +205,9 @@ raises research questions that we will address in future stages of our discv5 ro
|
||||
|
||||
## References
|
||||
|
||||
1. [10/WAKU2](../10/waku2.md)
|
||||
1. [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
2. [`34/WAKU2-PEER-EXCHANGE`](/waku/standards/core/34/peer-exchange.md)
|
||||
3. [11/WAKU2-RELAY](../11/relay.md)
|
||||
3. [11/WAKU2-RELAY](/waku/standards/core/11/relay.md)
|
||||
4. [WAKU2-ENR](https://github.com/waku-org/specs/blob/master/standards/core/enr.md)
|
||||
5. [Node Discovery Protocol v5 (`discv5`)](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
|
||||
6. [`discv5` semantics](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md).
|
||||
|
||||
@@ -50,16 +50,16 @@ routed on the combination of the eight pubsub topics:
|
||||
/waku/2/rs/1/7
|
||||
```
|
||||
|
||||
A node MUST use [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
|
||||
A node MUST use [66/WAKU2-METADATA](../66/metadata.md)
|
||||
protocol to identify the `<cluster_id>` that every
|
||||
inbound/outbound peer that attempts to connect supports.
|
||||
In any of the following cases, the node MUST trigger a disconnection:
|
||||
|
||||
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
|
||||
* [66/WAKU2-METADATA](../66/metadata.md)
|
||||
dial fails.
|
||||
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
|
||||
* [66/WAKU2-METADATA](../66/metadata.md)
|
||||
reports an empty `<cluster_id>`.
|
||||
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
|
||||
* [66/WAKU2-METADATA](../66/metadata.md)
|
||||
reports a `<cluster_id>` different than `1`.
|
||||
|
||||
## Roles
|
||||
@@ -370,7 +370,7 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
* [17/WAKU2-RLN-RELAY](../17/rln-relay.md)
|
||||
* [11/WAKU2-RELAY](../11/relay.md)
|
||||
* [WAKU2-RELAY-SHARDING](../../core/relay-sharding.md)
|
||||
* [WAKU-METADATA](https://github.com/waku-org/specs/blob/master/standards/core/metadata.md)
|
||||
* [66/WAKU2-METADATA](../66/metadata.md)
|
||||
* [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459)
|
||||
* [33/WAKU2-DISCV5](../33/discv5.md)
|
||||
* [12/WAKU2-FILTER](../12/filter.md)
|
||||
|
||||
@@ -3,27 +3,32 @@ slug: 66
|
||||
title: 66/WAKU2-METADATA
|
||||
name: Waku Metadata Protocol
|
||||
status: draft
|
||||
editor: Alvaro Revuelta <alrevuelta@status.im>
|
||||
editor: Franck Royer <franck@status.im>
|
||||
contributors:
|
||||
- Filip Dimitrijevic <filip@status.im>
|
||||
- Alvaro Revuelta <alrevuelta@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
This specification describes the metadata
|
||||
that can be associated with a [10/WAKU2](../10/waku2.md) node.
|
||||
that can be associated with a [10/WAKU2](/waku/standards/core/10/waku2.md) node.
|
||||
|
||||
## Metadata Protocol
|
||||
|
||||
Waku specifies a req/resp protocol that provides information about the node's medatadata.
|
||||
Such metadata is meant to be used by the node to decide if a peer is worth connecting
|
||||
or not.
|
||||
The keywords “MUST”, // List style “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”,
|
||||
“NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
Waku specifies a req/resp protocol that provides information about the node's capabilities.
|
||||
Such metadata MAY be used by other peers for subsequent actions such as light protocol requests or disconnection.
|
||||
|
||||
The node that makes the request,
|
||||
includes its metadata so that the receiver is aware of it,
|
||||
without requiring an extra interaction.
|
||||
without requiring another round trip.
|
||||
The parameters are the following:
|
||||
|
||||
* `clusterId`: Unique identifier of the cluster that the node is running in.
|
||||
* `shards`: Shard indexes that the node is subscribed to.
|
||||
* `shards`: Shard indexes that the node is subscribed to via [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md).
|
||||
|
||||
***Protocol Identifier***
|
||||
|
||||
@@ -47,10 +52,56 @@ message WakuMetadataResponse {
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Suggestions
|
||||
|
||||
### Triggering Metadata Request
|
||||
|
||||
A node SHOULD proceed with metadata request upon first connection to a remote node.
|
||||
A node SHOULD use the remote node's libp2p peer id as identifier for this heuristic.
|
||||
|
||||
A node MAY proceed with metadata request upon reconnection to a remote peer.
|
||||
|
||||
A node SHOULD store the remote peer's metadata information for future reference.
|
||||
A node MAY implement a TTL regarding a remote peer's metadata, and refresh it upon expiry by initiating another metadata request.
|
||||
It is RECOMMENDED to set the TTL to 6 hours.
|
||||
|
||||
A node MAY trigger a metadata request after receiving an error response from a remote note
|
||||
stating they do not support a specific cluster or shard.
|
||||
For example, when using a request-response service such as [`19/WAKU2-LIGHTPUSH`](/waku/standards/core/19/lightpush.md).
|
||||
|
||||
### Providing Cluster Id
|
||||
|
||||
A node MUST include their cluster id into their metadata payload.
|
||||
It is RECOMMENDED for a node to operate on a single cluster id.
|
||||
|
||||
### Providing Shard Information
|
||||
|
||||
* Nodes that mount [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md) MAY include the shards they are subscribed to in their metadata payload.
|
||||
* Shard-relevant services are message related services,
|
||||
such as [`13/WAKU2-STORE`](/waku/standards/core/13/store.md), [12/WAKU2-FILTER](/waku/standards/core/12/filter.md)
|
||||
and [`19/WAKU2-LIGHTPUSH`](/waku/standards/core/19/lightpush.md)
|
||||
but not [`34/WAKU2-PEER-EXCHANGE`](/waku/standards/core/34/peer-exchange.md)
|
||||
* Nodes that mount [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md) and a shard-relevant service SHOULD include the shards they are subscribed to in their metadata payload.
|
||||
* Nodes that do not mount [`11/WAKU2-RELAY`](/waku/standards/core/11/relay.md) SHOULD NOT include any shard information
|
||||
|
||||
### Using Cluster Id
|
||||
|
||||
When reading the cluster id of a remote peer, the local node MAY disconnect if their cluster id is different from the remote peer.
|
||||
|
||||
### Using Shard Information
|
||||
|
||||
It is NOT RECOMMENDED to disconnect from a peer based on the fact that their shard information is different from the local node.
|
||||
|
||||
Ahead of doing a shard-relevant request,
|
||||
a node MAY use the previously received metadata shard information to select a peer that support the targeted shard.
|
||||
|
||||
For non-shard-relevant requests, a node SHOULD NOT discriminate a peer based on medata shard information.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## References
|
||||
|
||||
* [10/WAKU2](../10/waku2.md)
|
||||
* [10/WAKU2](/waku/standards/core/10/waku2.md)
|
||||
|
||||
Reference in New Issue
Block a user