AtHeartEngineer/specs
1
0
Fork 0
mirror of https://github.com/vacp2p/specs.git synced 2026-01-09 15:28:03 -05:00
Code Issues Packages Projects Releases Wiki Activity

refactor the peer ID spec

Browse Source 
* I reorganized it to put all the key encoding logic together (instead of
  sticking the peer ID stuff in the middle).
* Described how the protobufs must should be deterministically encoded.
* Fixed some misconceptions around CIDv0 versus CIDv1. Qm... peer IDs are not
  CIDv0, they're just bare multihashes.
* Instructed users to continue encoding peer IDs as bare multihashes by default
  for now. Not all implementations support decoding CID peer IDs.
This commit is contained in:
Steven Allen
2020-01-19 09:45:55 -08:00
parent 7467aecfae
commit 44ba63e46f
1 changed files with 138 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files

260
peer-ids/peer-ids.md
View File

@@ -20,20 +20,18 @@ about maturity level and spec status.
## Table of Contents
- [Peer Ids and Keys](#peer-ids-and-keys)
- [Table of Contents](#table-of-contents)
- [Overview](#overview)
- [Keys](#keys)
- [Where are keys used?](#where-are-keys-used)
- [Peer Ids](#peer-ids)
- [Note about deterministic encoding](#note-about-deterministic-encoding)
- [String representation](#string-representation)
- [How Keys are Encoded and Messages Signed](#how-keys-are-encoded-and-messages-signed)
- [Overview](#overview)
- [Keys](#keys)
- [Where are keys used?](#where-are-keys-used)
- [Key Types](#key-types)
- [RSA](#rsa)
- [Ed25519](#ed25519)
- [Secp256k1](#secp256k1)
- [ECDSA](#ecdsa)
- [Peer Ids](#peer-ids)
- [String representation](#string-representation)
- [Encoding](#encoding)
- [Decoding](#decoding)
## Overview
@@ -53,8 +51,10 @@ Key encodings and message signing semantics are
## Keys
Our key pairs are wrapped in a [simple protobuf](https://github.com/libp2p/go-libp2p-core/blob/master/crypto/pb/crypto.proto),
defined using the [Protobuf version 2 syntax](https://developers.google.com/protocol-buffers/docs/proto):
Libp2p encodes keys in a [protobuf](https://github.com/libp2p/go-libp2p-core/blob/master/crypto/pb/crypto.proto)
containing a key _type_ and the encoded key (where the encoding depends on the type).
Specifically:
```protobuf
syntax = "proto2";
@@ -81,7 +81,17 @@ The `PublicKey` and `PrivateKey` messages contain a `Data` field with serialized
keys, and a `Type` enum that specifies the type of key.
Each key type has its own serialization format within the `Data` field,
[described below](#how-keys-are-encoded-and-messages-signed).
[described below](#key-types).
Finally, libp2p places a stronger requirement on the protobuf encoder than the
protobuf spec: encoding must be deterministic. To achieve this, libp2p imposes
two additional requirements:
1. Fields must be minimally encoded. That is, varints must use the minimal
representation (fewest bytes that can encode the given number).
2. Fields must be encoded in tag order (i.e., key type, then the key data).
3. All fields must be included.
4. No additional fields may be defined.
Note that `PrivateKey` messages are never transmitted over the wire.
Current libp2p implementations store private keys on disk as a serialized
@@ -91,96 +101,15 @@ files.
### Where are keys used?
Keys are used in two places in libp2p. The first is for signing messages. Here are some examples of messages we sign:
Keys are used in two places in libp2p. The first is for signing messages. Here
are some examples of messages we sign:
- IPNS records
- PubSub messages
- SECIO handshake
The second is for generating peer ids; this is discussed in the section below.
## Peer Ids
Here is the process by which we generate peer ids based on the public component of the keypairs described above:
1. Encode the public key into a byte array according to the rules [described below](#how-keys-are-encoded-and-messages-signed)
2. Construct a `PublicKey` protobuf message, setting the `Data` field to the encoded public key bytes, and also setting the correct `Type` field.
3. Serialize the protobuf containing the public key into bytes using the [canonical protobuf encoding](https://developers.google.com/protocol-buffers/docs/encoding).
4. If the length of the serialized bytes <= 42, then we compute the "identity" multihash of the serialized bytes. In other words, no hashing is performed, but the [multihash format is still followed](https://github.com/multiformats/multihash) (byte plus varint plus serialized bytes). The idea here is that if the serialized byte array is short enough, we can fit it in a multihash verbatim without having to condense it using a hash function.
5. If the length is >42, then we hash it using it using the SHA256 multihash.
### Note about deterministic encoding
Deterministic encoding of the `PublicKey` message is desirable, as it ensures
the same public key will always result in the same peer id.
The Protobuf specification does not provide sufficient guidance to ensure
deterministic serialization of messages. Specifically, there are no guarantees
about field ordering, and it's possible to add any number of unknown fields to a
message.
A future revision to this spec may define a canonical encoding that is more
strict than the Protobuf encoding spec.
In the meantime, implementors SHOULD use a protobuf encoder that orders fields
by their field number. This is the default behavior in many protobuf encoders,
but should be verified before committing to an implementation.
Additionally, implementors MUST avoid extending the `PublicKey` message with
additional fields, as the ordering of unknown fields is currently undefined
behavior.
### String representation
Peer Ids are [multihashes][multihash] canonically represented with [CIDs](https://github.com/ipld/cid) when encoded into strings.
Encoding and decoding of string representation MUST follow [CID specification][cid-decoding].
Implementations parsing IDs from text MUST support both base58 CIDv0 and CIDv1 in base32, and they MUST generate base32-encoded CIDv1 by default. Generating CIDv0 is allowed as an opt-in (behind a flag).
CIDv0 is a multihash encoded in Base58.
CIDv1 is a multihash with a prefix that specifies things like base encoding, cid version and the type of data behind it:
```
<cidv1> ::= <multibase><cid-version><multicodec><multihash>
```
#### libp2p-key CID
The canonical string representation of a Peer Id is a CID v1
with `base32` [multibase][multibase] ([RFC4648](https://tools.ietf.org/html/rfc4648), without padding) and `libp2p-key` [multicodec][multicodec]:
| multibase | cid version | multicodec |
| --------- | ----------- | ------------ |
| `base32` | `1` | `libp2p-key` |
- `libp2p-key` multicodec is mandatory when serializing to text (ensures Peer Id is self-describing)
- `base32` is the default multibase encoding: projects are free to use a different one if it is more suited to their needs
##### Decoding string representation
To decode a CID, follow the following algorithm:
- If it is 46 characters long and starts with `Qm...`, it's a CIDv0. Decode it as base58btc multihash.
- Otherwise, decode it according to the multibase and [CID spec][cid-decoding].
Examples:
- SHA256 Peer Id encoded as canonical [CIDv1][cid-versions]:
`bafzbeie5745rpv2m6tjyuugywy4d5ewrqgqqhfnf445he3omzpjbx5xqxe` ([inspect](http://cid.ipfs.io/#bafzbeie5745rpv2m6tjyuugywy4d5ewrqgqqhfnf445he3omzpjbx5xqxe))
- Peer Ids that do not start with a valid multibase prefix are assumed to be legacy [CIDv0][cid-versions]
(a multihash with implicit [`base58btc`][base58btc] encoding, without any prefix).
An example of the same Peer Id as a legacy CIDv0: `QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N`
[multihash]: https://github.com/multiformats/multihash
[multicodec]: https://github.com/multiformats/multicodec
[multibase]: https://github.com/multiformats/multibase
[base58btc]: https://en.bitcoinwiki.org/wiki/Base58#Alphabet_Base58
[cid-decoding]: https://github.com/multiformats/cid#decoding-algorithm
[cid-versions]: https://github.com/multiformats/cid#versions
## How Keys are Encoded and Messages Signed
### Key Types
Four key types are supported:
- RSA
@@ -188,57 +117,144 @@ Four key types are supported:
- Secp256k1
- ECDSA
Implementations MUST support RSA and Ed25519. Implementations MAY support Secp256k1 and ECDSA, but nodes using those keys may not be able to connect to all other nodes.
Implementations MUST support RSA and Ed25519. Implementations MAY support
Secp256k1 and ECDSA, but nodes using those keys may not be able to connect to
all other nodes.
In all cases, implementation MAY allow the user to enable/disable specific key types via configuration.
Note that disabling support for compulsory key types may hinder connectivity.
In all cases, implementation MAY allow the user to enable/disable specific key
types via configuration. Note that disabling support for compulsory key types
may hinder connectivity.
Keys are encoded into byte arrays and serialized into the `Data` field of the
protobuf messages described above.
The following sections describe:
The following sections describe each key type's encoding rules.
1. How each key type is encoded into the libp2p key's Data field.
2. How each key type creates and validates signatures.
libp2p implementations MUST use the encoding described below when embedding
keys into the `PublicKey` and `PrivateKey` messages described above, whether
for transmission, Peer Id generation, or storage.
Implementations can use whatever in-memory representation is convenient,
Implementations may use whatever in-memory representation is convenient,
provided the encodings described below are used at the "I/O boundary".
In addition to key encodings, the sections below cover the signing method used
when signing and verifying messages with each key type.
### RSA
#### RSA
We encode the public key using the DER-encoded PKIX format.
We encode the private key as a PKCS1 key using ASN.1 DER.
To sign a message, we first hash it with SHA-256 and then sign it using the [RSASSA-PKCS1-V1.5-SIGN](https://tools.ietf.org/html/rfc3447#section-8.2) method, as originally defined in [RSA PKCS#1 v1.5](https://tools.ietf.org/html/rfc2313).
To sign a message, we first hash it with SHA-256 and then sign it using the
[RSASSA-PKCS1-V1.5-SIGN](https://tools.ietf.org/html/rfc3447#section-8.2)
method, as originally defined in [RSA PKCS#1
v1.5](https://tools.ietf.org/html/rfc2313).
### Ed25519
#### Ed25519
Ed25519 specifies the exact format for keys and signatures, so we do not do much additional encoding, except as noted below.
Ed25519 specifies the exact format for keys and signatures, so we do not do much
additional encoding, except as noted below.
We do not do any special additional encoding for Ed25519 public keys.
The encoding for Ed25519 private keys is a little unusual. There are two formats that we encourage implementors to support:
The encoding for Ed25519 private keys is a little unusual. There are two formats
that we encourage implementors to support:
- Preferred method is a simple concatenation: `[private key bytes][public key bytes]` (64 bytes)
- Older versions of the libp2p code used the following format: `[private key][public key][public key]` (96 bytes). If you encounter this type of encoding, the proper way to process it is to compare the two public key strings (32 bytes each) and verify they are identical. If they are, then proceed as you would with the preferred method. If they do not match, reject or error out because the byte array is invalid.
- Preferred method is a simple concatenation: `[private key bytes][public key
bytes]` (64 bytes)
- Older versions of the libp2p code used the following format: `[private
key][public key][public key]` (96 bytes). If you encounter this type of
encoding, the proper way to process it is to compare the two public key
strings (32 bytes each) and verify they are identical. If they are, then
proceed as you would with the preferred method. If they do not match, reject
or error out because the byte array is invalid.
Ed25519 signatures follow the normal [Ed25519 standard](https://tools.ietf.org/html/rfc8032#section-5.1).
### Secp256k1
#### Secp256k1
We use the standard Bitcoin EC encoding for Secp256k1 public and private keys.
To sign a message, we hash the message with SHA 256, then sign it using the standard [Bitcoin EC signature algorithm (BIP0062)](https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki), and then use [standard Bitcoin encoding](https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki#der-encoding).
To sign a message, we hash the message with SHA 256, then sign it using the
standard [Bitcoin EC signature algorithm
(BIP0062)](https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki), and
then use [standard Bitcoin
encoding](https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki#der-encoding).
### ECDSA
#### ECDSA
We encode the public key using ASN.1 DER.
We encode the private key using DER-encoded PKIX.
To sign a message, we hash the message with SHA 256, and then sign it with the [ECDSA standard algorithm](https://tools.ietf.org/html/rfc6979), then we encode it using [DER-encoded ASN.1.](https://wiki.openssl.org/index.php/DER)
To sign a message, we hash the message with SHA 256, and then sign it with the
[ECDSA standard algorithm](https://tools.ietf.org/html/rfc6979), then we encode
it using [DER-encoded ASN.1.](https://wiki.openssl.org/index.php/DER)
## Peer Ids
Peer IDs are derived by hashing the encoded public key with
[multihash][multihash]. Keys that serialize to more than 42 bytes must be hashed
using sha256 multihash, keys that serialize to at most 42 bytes must be hashed
using the "identity" mulish codec.
Specifically, to compute a peer ID of a key:
1. Encode the public key as described in the [keys](#keys) section.
4. If the length of the serialized bytes is less than or equal to 42, compute
the "identity" multihash of the serialized bytes. In other words, no
hashing is performed, but the [multihash format is still
followed][multihash] (byte plus varint plus serialized bytes). The idea
here is that if the serialized byte array is short enough, we can fit it in
a multihash verbatim without having to condense it using a hash function.
5. If the length is greater than 42, then hash it using it using the SHA256
multihash.
### String representation
There are two ways to represent peer IDs in text: as a raw
[base58btc][base58btc] encoded multihash (e.g., `Qm...`, `1...`) and as a
[multibase][multibase] encoded [CID][cid] (e.g., `bafz...`). Libp2p is slowly
transitioning from the first (legacy) format to the second (new).
Implementations MUST support parsing both forms of peer IDs. Implementations
SHOULD display peer IDs using the first (raw base58btc encoded multihash) format
until the second format is widely supported.
Peer IDs encoded as CIDs must be encoded using CIDv1 and must use the
`libp2p-key` [multicodec][multicodec] (0x72). By default, such peer IDs SHOULD
be encoded in using the base32 multibase
([RFC4648](https://tools.ietf.org/html/rfc4648), without padding).
For reference, CIDs (encoded in text) have the following format
```
<multibase-prefix><cid-version><multicodec><multihash>
```
#### Encoding
To encode a peer ID using the legacy format, simply encode it with base58btc.
To encode a peer ID using the new format, create a CID with the `libp2p-key` multicodec and encode it using multibase.
#### Decoding
To decode a peer ID:
* If it starts with `1` or `Qm`, it's a bare [base58btc][base58btc] encoded
[multihash][multihash]. Decode it according to the base58btc algorithm.
* If it starts with a [multibase][multibase] prefix, it's a CIDv1
CID. Decode it according to the multibase and [CID spec][cid-decoding].
* Once decoded, verify that the CIDs multicodec is `libp2p-key`.
* Finally, extract the multihash from the CID. This is the peer ID.
* Otherwise, it's not a valid peer ID.
Examples:
- `bafzbeie5745rpv2m6tjyuugywy4d5ewrqgqqhfnf445he3omzpjbx5xqxe` -- Peer ID (sha256) encoded as a CID
([inspect](http://cid.ipfs.io/#bafzbeie5745rpv2m6tjyuugywy4d5ewrqgqqhfnf445he3omzpjbx5xqxe)).
- `QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N` -- Peer ID (sha256) encoded as a raw base58btc multihash.
- `12D3KooWD3eckifWpRn9wQpMG9R9hX3sD158z7EqHWmweQAJU5SA` -- Peer ID (ed25519, using the "identity" multihash") encoded as a raw base58btc multihash.
[multihash]: https://github.com/multiformats/multihash
[multicodec]: https://github.com/multiformats/multicodec
[multibase]: https://github.com/multiformats/multibase
[base58btc]: https://en.bitcoinwiki.org/wiki/Base58#Alphabet_Base58
[cid]: https://github.com/multiformats/cid
[cid-decoding]: https://github.com/multiformats/cid#decoding-algorithm
[protobuf-encoding]: https://developers.google.com/protocol-buffers/docs/encoding