From 5dae252f56f4e780dceb35476407d00a0ada597b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 10 Feb 2020 15:53:10 -0700 Subject: [PATCH 1/4] add eth2 key/value ENR to phase 0 p2p --- specs/phase0/p2p-interface.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 7ad8759d3..651938a54 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -604,6 +604,12 @@ Nonetheless, ENRs MUST carry a generic `eth2` key with nil value, denoting that #### Mainnet +ENRs MUST carry a generic `eth2` with a 4-byte value of the node's current fork version to ensure connections are made with peers on the intended eth2 network. + +| Key | Value | +|:-------------|:--------------------| +| `eth2` | SSZ `Bytes4` | + On mainnet, ENRs MUST include a structure enumerating the capabilities offered by the peer in an efficient manner. The concrete solution is currently undefined. Proposals include using namespaced bloom filters mapping capabilities to specific protocol IDs supported under that capability. ### Topic advertisement From 37b1fed8ff03db4ad6d0113ea69551609067fbf6 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 4 Mar 2020 14:01:50 -0700 Subject: [PATCH 2/4] update eth2 ENR field to use ENRForkID --- specs/phase0/p2p-interface.md | 39 +++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 651938a54..c8d4bc1b4 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -55,6 +55,8 @@ It consists of four main sections: - [Attestation subnet bitfield](#attestation-subnet-bitfield) - [Interop](#interop-5) - [Mainnet](#mainnet-5) + - [`eth2` field](#eth2-field) + - [General capabilities](#general-capabilities) - [Topic advertisement](#topic-advertisement) - [Mainnet](#mainnet-6) - [Design decision rationale](#design-decision-rationale) @@ -88,6 +90,7 @@ It consists of four main sections: - [Why are we sending entire objects in the pubsub and not just hashes?](#why-are-we-sending-entire-objects-in-the-pubsub-and-not-just-hashes) - [Should clients gossip blocks if they *cannot* validate the proposer signature due to not yet being synced, not knowing the head block, etc?](#should-clients-gossip-blocks-if-they-cannot-validate-the-proposer-signature-due-to-not-yet-being-synced-not-knowing-the-head-block-etc) - [How are we going to discover peers in a gossipsub topic?](#how-are-we-going-to-discover-peers-in-a-gossipsub-topic) + - [How should fork version be used in practice?](#how-should-fork-version-be-used-in-practice) - [Req/Resp](#reqresp) - [Why segregate requests into dedicated protocol IDs?](#why-segregate-requests-into-dedicated-protocol-ids) - [Why are messages length-prefixed with a protobuf varint in the SSZ-encoding?](#why-are-messages-length-prefixed-with-a-protobuf-varint-in-the-ssz-encoding) @@ -604,11 +607,35 @@ Nonetheless, ENRs MUST carry a generic `eth2` key with nil value, denoting that #### Mainnet -ENRs MUST carry a generic `eth2` with a 4-byte value of the node's current fork version to ensure connections are made with peers on the intended eth2 network. +##### `eth2` field + +ENRs MUST carry a generic `eth2` key with an 16-byte value of the node's current fork version, next fork version, and next fork epoch to ensure connections are made with peers on the intended eth2 network. | Key | Value | |:-------------|:--------------------| -| `eth2` | SSZ `Bytes4` | +| `eth2` | SSZ `ENRForkID` | + +Specifically, the value of the `eth2` key MUST be the following SSZ encoded object (`ENRForkID`), where + +``` +( + current_fork_version: Fork + next_fork_version: Fork + next_fork_epoch: Epoch +) +``` + +where the fields of `ENRForkID` are defined as + +* `current_fork_version` is the fork version at the node's current epoch defined by the wall-clock time (not necessarily the epoch to which the node is sync) +* `next_fork_version` is the fork version corresponding to the next planned hard fork at a future epoch. If no future fork is planned, set `next_fork_version = current_fork_version` to signal this fact +* `next_fork_epoch` is the epoch at which the next fork is planned and the `current_fork_version` will be updated. If no future fork is planned, set `next_fork_epoch = FAR_FUTURE_EPOCH` to signal this fact + +Clients SHOULD connect to peers with `current_fork_version`, `next_fork_version`, and `next_fork_epoch` that match local values. + +Clients MAY connect to peers with the same `current_fork_version` but a different `next_fork_version`/`next_fork_epoch`. Unless `ENRForkID` is manually updated to matching prior to the earlier `next_fork_epoch` of the two clients, these type of connecting clients will be unable to successfully interact starting at the earlier `next_fork_epoch`. + +##### General capabilities On mainnet, ENRs MUST include a structure enumerating the capabilities offered by the peer in an efficient manner. The concrete solution is currently undefined. Proposals include using namespaced bloom filters mapping capabilities to specific protocol IDs supported under that capability. @@ -829,6 +856,14 @@ In Phase 0, peers for attestation subnets will be found using the `attnets` entr Although this method will be sufficient for early phases of Eth2, we aim to use the more appropriate discv5 topics for this and other similar tasks in the future. ENRs should ultimately not be used for this purpose. They are best suited to store identity, location, and capability information, rather than more volatile advertisements. +### How should fork version be used in practice? + +Fork versions are to be manually updated (likely via incrementing or using the less collision-prone git spec hash) at each hard fork. This is to provide native domain separation for signatures as well as to aid in usefulness for identitying peers (via ENRs) and versioning network protocols (e.g. using fork version to naturally version gossipsub topics). + +To reap the full benefit of the native versioning scheme, networks SHOULD avoid collisions. For example, a testnet might us mainnet versioning but use a unique higher order byte to signal the testnet. + +A node locally stores all previous and future planned fork versions along with the each fork epoch. This allows for handling sync starting from past forks/epochs and for connections to safely be made with peers syncing from past forks/epochs. + ## Req/Resp ### Why segregate requests into dedicated protocol IDs? From 7e04989e29306c8d6049338457e3ba89c7959218 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 5 Mar 2020 09:21:32 -0700 Subject: [PATCH 3/4] add genesis_validators_root to beaconstate and utilize in sig domain separation as well as fork separation --- setup.py | 4 ++-- specs/phase0/beacon-chain.md | 14 +++++++---- specs/phase0/p2p-interface.md | 24 +++++++++++++++---- specs/phase1/beacon-chain.md | 1 + .../pyspec/eth2spec/test/helpers/genesis.py | 3 +++ 5 files changed, 35 insertions(+), 11 deletions(-) diff --git a/setup.py b/setup.py index 444489d2e..6dd4de861 100644 --- a/setup.py +++ b/setup.py @@ -95,7 +95,7 @@ from dataclasses import ( from eth2spec.utils.ssz.ssz_impl import hash_tree_root from eth2spec.utils.ssz.ssz_typing import ( View, boolean, Container, List, Vector, uint64, - Bytes1, Bytes4, Bytes8, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, + Bytes1, Bytes4, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, ) from eth2spec.utils import bls @@ -117,7 +117,7 @@ from dataclasses import ( from eth2spec.utils.ssz.ssz_impl import hash_tree_root from eth2spec.utils.ssz.ssz_typing import ( View, boolean, Container, List, Vector, uint64, uint8, bit, - ByteList, Bytes1, Bytes4, Bytes8, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, + ByteList, Bytes1, Bytes4, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, ) from eth2spec.utils import bls diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index c7a76d066..731b02b1a 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -149,7 +149,7 @@ We define the following Python custom types for type hinting and readability: | `Root` | `Bytes32` | a Merkle root | | `Version` | `Bytes4` | a fork version number | | `DomainType` | `Bytes4` | a domain type | -| `Domain` | `Bytes8` | a signature domain | +| `Domain` | `Bytes32` | a signature domain | | `BLSPubkey` | `Bytes48` | a BLS12-381 public key | | `BLSSignature` | `Bytes96` | a BLS12-381 signature | @@ -473,6 +473,7 @@ class BeaconBlock(Container): class BeaconState(Container): # Versioning genesis_time: uint64 + genesis_validators_root: Root slot: Slot fork: Fork # History @@ -795,13 +796,15 @@ def compute_activation_exit_epoch(epoch: Epoch) -> Epoch: #### `compute_domain` ```python -def compute_domain(domain_type: DomainType, fork_version: Optional[Version]=None) -> Domain: +def compute_domain(domain_type: DomainType, fork_version: Optional[Version]=None, genesis_root: Root=None) -> Domain: """ Return the domain for the ``domain_type`` and ``fork_version``. """ if fork_version is None: fork_version = GENESIS_FORK_VERSION - return Domain(domain_type + fork_version) + if genesis_root is None: + genesis_root = Root() + return Domain(domain_type + fork_version + genesis_root[:24]) ``` #### `compute_signing_root` @@ -977,7 +980,7 @@ def get_domain(state: BeaconState, domain_type: DomainType, epoch: Epoch=None) - """ epoch = get_current_epoch(state) if epoch is None else epoch fork_version = state.fork.previous_version if epoch < state.fork.epoch else state.fork.current_version - return compute_domain(domain_type, fork_version) + return compute_domain(domain_type, fork_version, state.genesis_validators_root) ``` #### `get_indexed_attestation` @@ -1122,6 +1125,9 @@ def initialize_beacon_state_from_eth1(eth1_block_hash: Bytes32, validator.activation_eligibility_epoch = GENESIS_EPOCH validator.activation_epoch = GENESIS_EPOCH + # Set genesis validators root for domain separation and chain versioning + state.genesis_validators_root = hash_tree_root(state.validators) + return state ``` diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index c8d4bc1b4..f2dcc2324 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -615,23 +615,37 @@ ENRs MUST carry a generic `eth2` key with an 16-byte value of the node's current |:-------------|:--------------------| | `eth2` | SSZ `ENRForkID` | -Specifically, the value of the `eth2` key MUST be the following SSZ encoded object (`ENRForkID`), where +First we define `current_fork` as the following SSZ encoded object ``` ( - current_fork_version: Fork - next_fork_version: Fork + current_fork_version: Version + genesis_validators_root: Root +) +``` + +where + +* `current_fork_version` is the fork version at the node's current epoch defined by the wall-clock time (not necessarily the epoch to which the node is sync) +* `genesis_validators_root` is the static `Root` found in `state.genesis_validators_root` + +Specifically, the value of the `eth2` key MUST be the following SSZ encoded object (`ENRForkID`) + +``` +( + current_fork_digest: Bytes4 + next_fork_version: Version next_fork_epoch: Epoch ) ``` where the fields of `ENRForkID` are defined as -* `current_fork_version` is the fork version at the node's current epoch defined by the wall-clock time (not necessarily the epoch to which the node is sync) +* `current_fork_digest` is `hash_tree_root(current_fork)[:4]` * `next_fork_version` is the fork version corresponding to the next planned hard fork at a future epoch. If no future fork is planned, set `next_fork_version = current_fork_version` to signal this fact * `next_fork_epoch` is the epoch at which the next fork is planned and the `current_fork_version` will be updated. If no future fork is planned, set `next_fork_epoch = FAR_FUTURE_EPOCH` to signal this fact -Clients SHOULD connect to peers with `current_fork_version`, `next_fork_version`, and `next_fork_epoch` that match local values. +Clients SHOULD connect to peers with `current_fork_digest`, `next_fork_version`, and `next_fork_epoch` that match local values. Clients MAY connect to peers with the same `current_fork_version` but a different `next_fork_version`/`next_fork_epoch`. Unless `ENRForkID` is manually updated to matching prior to the earlier `next_fork_epoch` of the two clients, these type of connecting clients will be unable to successfully interact starting at the earlier `next_fork_epoch`. diff --git a/specs/phase1/beacon-chain.md b/specs/phase1/beacon-chain.md index 78b3b3d25..095bbafc1 100644 --- a/specs/phase1/beacon-chain.md +++ b/specs/phase1/beacon-chain.md @@ -244,6 +244,7 @@ Note that aside from the new additions, `Validator` and `PendingAttestation` hav class BeaconState(Container): # Versioning genesis_time: uint64 + genesis_validators_root: Root slot: Slot fork: Fork # History diff --git a/tests/core/pyspec/eth2spec/test/helpers/genesis.py b/tests/core/pyspec/eth2spec/test/helpers/genesis.py index c60787b92..46bc62fe5 100644 --- a/tests/core/pyspec/eth2spec/test/helpers/genesis.py +++ b/tests/core/pyspec/eth2spec/test/helpers/genesis.py @@ -43,4 +43,7 @@ def create_genesis_state(spec, validator_balances, activation_threshold): validator.activation_eligibility_epoch = spec.GENESIS_EPOCH validator.activation_epoch = spec.GENESIS_EPOCH + # Set genesis validators root for domain separation and chain versioning + state.genesis_validators_root = spec.hash_tree_root(state.validators) + return state From 1818f349ad9200c0d9e1d8b9633fa4efc9330950 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 10 Mar 2020 11:59:34 -0600 Subject: [PATCH 4/4] add ForkDigest type, clarify how genesis_validators_root is mixed into domains for chain isolation in p2p faq --- specs/phase0/beacon-chain.md | 3 ++- specs/phase0/p2p-interface.md | 10 +++++----- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index 731b02b1a..8c93df7ef 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -149,6 +149,7 @@ We define the following Python custom types for type hinting and readability: | `Root` | `Bytes32` | a Merkle root | | `Version` | `Bytes4` | a fork version number | | `DomainType` | `Bytes4` | a domain type | +| `ForkDigest` | `Bytes4` | a digest of the current fork data | | `Domain` | `Bytes32` | a signature domain | | `BLSPubkey` | `Bytes48` | a BLS12-381 public key | | `BLSSignature` | `Bytes96` | a BLS12-381 signature | @@ -803,7 +804,7 @@ def compute_domain(domain_type: DomainType, fork_version: Optional[Version]=None if fork_version is None: fork_version = GENESIS_FORK_VERSION if genesis_root is None: - genesis_root = Root() + genesis_root = Root() # all bytes zero by default return Domain(domain_type + fork_version + genesis_root[:24]) ``` diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index f2dcc2324..0b6a5798a 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -615,7 +615,7 @@ ENRs MUST carry a generic `eth2` key with an 16-byte value of the node's current |:-------------|:--------------------| | `eth2` | SSZ `ENRForkID` | -First we define `current_fork` as the following SSZ encoded object +First we define `current_fork_data` as the following SSZ encoded object ``` ( @@ -633,7 +633,7 @@ Specifically, the value of the `eth2` key MUST be the following SSZ encoded obje ``` ( - current_fork_digest: Bytes4 + current_fork_digest: ForkDigest next_fork_version: Version next_fork_epoch: Epoch ) @@ -641,7 +641,7 @@ Specifically, the value of the `eth2` key MUST be the following SSZ encoded obje where the fields of `ENRForkID` are defined as -* `current_fork_digest` is `hash_tree_root(current_fork)[:4]` +* `current_fork_digest` is `ForkDigest(hash_tree_root(current_fork_data)[:4])` * `next_fork_version` is the fork version corresponding to the next planned hard fork at a future epoch. If no future fork is planned, set `next_fork_version = current_fork_version` to signal this fact * `next_fork_epoch` is the epoch at which the next fork is planned and the `current_fork_version` will be updated. If no future fork is planned, set `next_fork_epoch = FAR_FUTURE_EPOCH` to signal this fact @@ -872,9 +872,9 @@ Although this method will be sufficient for early phases of Eth2, we aim to use ### How should fork version be used in practice? -Fork versions are to be manually updated (likely via incrementing or using the less collision-prone git spec hash) at each hard fork. This is to provide native domain separation for signatures as well as to aid in usefulness for identitying peers (via ENRs) and versioning network protocols (e.g. using fork version to naturally version gossipsub topics). +Fork versions are to be manually updated (likely via incrementing) at each hard fork. This is to provide native domain separation for signatures as well as to aid in usefulness for identitying peers (via ENRs) and versioning network protocols (e.g. using fork version to naturally version gossipsub topics). -To reap the full benefit of the native versioning scheme, networks SHOULD avoid collisions. For example, a testnet might us mainnet versioning but use a unique higher order byte to signal the testnet. +`BeaconState.genesis_validators_root` is mixed into signature and ENR fork domains to aid in the ease of domain separation between chains. This allows fork versions to safely be reused across chains except for the case of contentious forks using the same genesis. In these cases, extra care should be taken to isolate fork versions (e.g. flip a high order bit in all future versions of one of the chains). A node locally stores all previous and future planned fork versions along with the each fork epoch. This allows for handling sync starting from past forks/epochs and for connections to safely be made with peers syncing from past forks/epochs.