From 17221c806501e11943fd8359754f79b65713468d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 12 Nov 2020 10:26:25 -0700 Subject: [PATCH 1/6] more clearly define min epoch range for blocksbyrange requests --- README.md | 1 + specs/phase0/p2p-interface.md | 41 ++++++++++++++++++++++++++++--- specs/phase0/weak-subjectivity.md | 5 +++- 3 files changed, 42 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index a3ca7c586..8104ec7cc 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,7 @@ Core specifications for Eth2 clients be found in [specs](specs/). These are divi * [Deposit Contract](specs/phase0/deposit-contract.md) * [Honest Validator](specs/phase0/validator.md) * [P2P Networking](specs/phase0/p2p-interface.md) +* [Weak Subjectivity](specs/phase0/weak-subjectivity.md.md) ### Phase 1 * [From Phase 0 to Phase 1](specs/phase1/phase1-fork.md) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 5dc892991..8104b54c7 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -93,6 +93,7 @@ It consists of four main sections: - [Why is it called Req/Resp and not RPC?](#why-is-it-called-reqresp-and-not-rpc) - [Why do we allow empty responses in block requests?](#why-do-we-allow-empty-responses-in-block-requests) - [Why does `BeaconBlocksByRange` let the server choose which branch to send blocks from?](#why-does-beaconblocksbyrange-let-the-server-choose-which-branch-to-send-blocks-from) + - [Why are `BlocksByRange` requests only required to be served for the latest `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs?](#why-are-blocksbyrange-requests-only-required-to-be-served-for-the-latest-min_epochs_for_block_requests-epochs) - [What's the effect of empty slots on the sync algorithm?](#whats-the-effect-of-empty-slots-on-the-sync-algorithm) - [Discovery](#discovery) - [Why are we using discv5 and not libp2p Kademlia DHT?](#why-are-we-using-discv5-and-not-libp2p-kademlia-dht) @@ -171,6 +172,7 @@ This section outlines constants that are used in this spec. |---|---|---| | `GOSSIP_MAX_SIZE` | `2**20` (= 1048576, 1 MiB) | The maximum allowed size of uncompressed gossip messages. | | `MAX_REQUEST_BLOCKS` | `2**10` (= 1024) | Maximum number of blocks in a single request | +| `MIN_EPOCHS_FOR_BLOCK_REQUESTS` | `MIN_VALIDATOR_WITHDRAWABILITY_DELAY + CHURN_LIMIT_QUOTIENT // 2` (= 33024, ~5 months) | The minimum epoch range over which a node must serve blocks | | `MAX_CHUNK_SIZE` | `2**20` (1048576, 1 MiB) | The maximum allowed size of uncompressed req/resp chunked responses. | | `TTFB_TIMEOUT` | `5s` | The maximum time to wait for first byte of request response (time-to-first-byte). | | `RESP_TIMEOUT` | `10s` | The maximum time for complete response transfer. | @@ -179,7 +181,6 @@ This section outlines constants that are used in this spec. | `MESSAGE_DOMAIN_INVALID_SNAPPY` | `0x00000000` | 4-byte domain for gossip message-id isolation of *invalid* snappy messages | | `MESSAGE_DOMAIN_VALID_SNAPPY` | `0x01000000` | 4-byte domain for gossip message-id isolation of *valid* snappy messages | - ## MetaData Clients MUST locally store the following `MetaData`: @@ -746,10 +747,17 @@ The request MUST be encoded as an SSZ-container. The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `SignedBeaconBlock` payload. -Clients MUST keep a record of signed blocks seen since the start of the weak subjectivity period -and MUST support serving requests of blocks up to their own `head_block_root`. +Clients MUST keep a record of signed blocks seen on the epoch range +`[max(GENESIS_EPOCH, current_epoch - MIN_EPOCHS_FOR_BLOCK_REQUESTS), current_epoch]` +where `current_epoch` is defined by the current wall-clock time, +and clients MUST support serving requests of blocks on this range. -Clients MUST respond with at least the first block that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOCKS` blocks. +*Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint +MUST backfill the local block database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOCK_REQUESTS` +to be compliant with `BlocksByRange` requests. + +Clients MUST respond with at least the first block that exists in the range, if they have it, +and no more than `MAX_REQUEST_BLOCKS` blocks. The following blocks, where they exist, MUST be sent in consecutive order. @@ -1394,6 +1402,31 @@ To avoid this race condition, we allow the responding side to choose which branc The requesting client then goes on to validate the blocks and incorporate them in their own database -- because they follow the same rules, they should at this point arrive at the same canonical chain. +### Why are `BlocksByRange` requests only required to be served for the latest `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs? + +Due to economic finality and weak subjectivity requirements of a proof-of-stake blockchain, for a new node to safely join the network +the node must provide a recent checkpoint found out-of-band. This checkpoint can be in the form of a `root` & `epoch` or it can be the entire +beacon state and then a simple block sync from there to the head. We expect the latter to be the dominant UX strategy. + +These checkpoints *in the worst case* (i.e. very large validator set and maximal allowed safety decay) must be from the +most recent `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs, and thus a user must be able to block sync to the head from this starting point. +Thus, this defines the epoch range outside which nodes may prune blocks, and +the epoch range that a new node syncing from a checkpoint must backfill. + +`MIN_EPOCHS_FOR_BLOCK_REQUESTS` is calculated using the arithmetic from `compute_weak_subjectivity_period` found in the +[weak subjectivity guide](./weak-subjectivity.md). Specifically to find this max epoch range, we use the worst case event of a very large validator size +(`>= MIN_PER_EPOCH_CHURN_LIMIT * CHURN_LIMIT_QUOTIENT`). + +```python +MIN_EPOCHS_FOR_BLOCK_REQUESTS = ( + MIN_VALIDATOR_WITHDRAWABILITY_DELAY + + MAX_SAFETY_DECAY * CHURN_LIMIT_QUOTIENT // (2 * 100) +) +``` + +Where `MAX_SAFETY_DECAY = 100` and thus `MIN_EPOCHS_FOR_BLOCK_REQUESTS = 33024` (~5 months). + + ### What's the effect of empty slots on the sync algorithm? When syncing one can only tell that a slot has been skipped on a particular branch diff --git a/specs/phase0/weak-subjectivity.md b/specs/phase0/weak-subjectivity.md index b4d78cb12..797f972a1 100644 --- a/specs/phase0/weak-subjectivity.md +++ b/specs/phase0/weak-subjectivity.md @@ -94,7 +94,9 @@ A brief reference for what these values look like in practice: ## Weak Subjectivity Sync -Clients should allow users to input a Weak Subjectivity Checkpoint at startup, and guarantee that any successful sync leads to the given Weak Subjectivity Checkpoint along the canonical chain. If such a sync is not possible, the client should treat this as a critical and irrecoverable failure. +Clients should allow users to input a Weak Subjectivity Checkpoint at startup, +and guarantee that any successful sync leads to the given Weak Subjectivity Checkpoint along the canonical chain. +If such a sync is not possible, the client should treat this as a critical and irrecoverable failure. ### Weak Subjectivity Sync Procedure @@ -112,6 +114,7 @@ Clients should allow users to input a Weak Subjectivity Checkpoint at startup, a Emit descriptive critical error if this assert fails, then exit client process. ### Checking for Stale Weak Subjectivity Checkpoint + Clients may choose to validate that the input Weak Subjectivity Checkpoint is not stale at the time of startup. To support this mechanism, the client needs to take the state at the Weak Subjectivity Checkpoint as a CLI parameter input (or fetch the state associated with the input Weak Subjectivity Checkpoint from some source). From 56aafbe533c6f5ed7881a65b22c41c1d7367c50e Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 9 Dec 2020 12:37:56 -0700 Subject: [PATCH 2/6] add note about signature check when backfilling beaconblocks --- specs/phase0/p2p-interface.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 8104b54c7..df17258ab 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -94,6 +94,7 @@ It consists of four main sections: - [Why do we allow empty responses in block requests?](#why-do-we-allow-empty-responses-in-block-requests) - [Why does `BeaconBlocksByRange` let the server choose which branch to send blocks from?](#why-does-beaconblocksbyrange-let-the-server-choose-which-branch-to-send-blocks-from) - [Why are `BlocksByRange` requests only required to be served for the latest `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs?](#why-are-blocksbyrange-requests-only-required-to-be-served-for-the-latest-min_epochs_for_block_requests-epochs) + - [Why must the proposer signature be checked when backfilling blocks in the database?](#why-must-the-proposer-signature-be-checked-when-backfilling-blocks-in-the-database) - [What's the effect of empty slots on the sync algorithm?](#whats-the-effect-of-empty-slots-on-the-sync-algorithm) - [Discovery](#discovery) - [Why are we using discv5 and not libp2p Kademlia DHT?](#why-are-we-using-discv5-and-not-libp2p-kademlia-dht) @@ -754,7 +755,10 @@ and clients MUST support serving requests of blocks on this range. *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint MUST backfill the local block database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOCK_REQUESTS` -to be compliant with `BlocksByRange` requests. +to be compliant with `BlocksByRange` requests. To safely perform such a +backfill of blocks to the recent state, the node MUST validate both (1) the +proposer signatures and (2) that the blocks form a valid chain up to the most +recent block referenced in the weak subjectivity state. Clients MUST respond with at least the first block that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOCKS` blocks. @@ -1426,6 +1430,20 @@ MIN_EPOCHS_FOR_BLOCK_REQUESTS = ( Where `MAX_SAFETY_DECAY = 100` and thus `MIN_EPOCHS_FOR_BLOCK_REQUESTS = 33024` (~5 months). +### Why must the proposer signature be checked when backfilling blocks in the database? + +When backfilling blocks in a database from a know safe block/state (e.g. when starting from a weak subjectivity state), +the node not only must ensure the `BeaconBlock`s form a chain to the known safe block, +but also must check that the proposer signature is valid in the `SignedBeaconBlock` wrapper. + +This is because the signature is not part of the `BeaconBlock` hash chain, and +thus could be corrupted by an attacker serving valid `BeaconBlock`s but invalid +signatures contained in `SignedBeaconBlock`. + +Although in this particular use case this does not represent a decay in safety +(due to the assumptions of starting at a weak subjectivity checkpoint), it +would represent invalid historic data and could be unwittingly transmitted to +additional nodes. ### What's the effect of empty slots on the sync algorithm? From 2ad8fdb818f58778391d93540892d35570f33237 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 13 Jan 2021 18:03:40 -0700 Subject: [PATCH 3/6] add ability for node to randomly request and descore if not serving blocks on WS period --- specs/phase0/p2p-interface.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index df17258ab..4a412ac19 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -753,6 +753,11 @@ Clients MUST keep a record of signed blocks seen on the epoch range where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blocks on this range. +Synced clients unable to reply to Block requests within the +`MIN_EPOCHS_FOR_BLOCK_REQUESTS` epoch range MAY get descored or disconnected at any time. +Note, due to this it is risky behaviour to begin participating as a full node at the head if having +not yet backfilled on this range. + *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint MUST backfill the local block database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOCK_REQUESTS` to be compliant with `BlocksByRange` requests. To safely perform such a From 488ceed4f9de63898d8ebc490d6b044b5a27fec4 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 11 May 2021 11:29:37 -0600 Subject: [PATCH 4/6] add notes about repeatedly failing tos erve blocks as being disconncetable --- specs/phase0/p2p-interface.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index a0bd2d4e2..f2c7d8610 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -752,18 +752,20 @@ Clients MUST keep a record of signed blocks seen on the epoch range where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blocks on this range. -Synced clients unable to reply to Block requests within the +Peers are *repeatedly* unable to reply to Block requests within the `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epoch range MAY get descored or disconnected at any time. -Note, due to this it is risky behaviour to begin participating as a full node at the head if having -not yet backfilled on this range. *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint MUST backfill the local block database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOCK_REQUESTS` -to be compliant with `BlocksByRange` requests. To safely perform such a +to be fully compliant with `BlocksByRange` requests. To safely perform such a backfill of blocks to the recent state, the node MUST validate both (1) the proposer signatures and (2) that the blocks form a valid chain up to the most recent block referenced in the weak subjectivity state. +*Note*: Although clients that bootstrap from a weak subjectivity checkpoint can begin +participating in the networking immediately, other peers that are actively block syncing MAY +disconnect and/or temporarily ban such an un-synced or semi-synced client. + Clients MUST respond with at least the first block that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOCKS` blocks. From 82b7a7be3b5cf6046ad046d4c499bb2721931bf2 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 12 May 2021 08:29:42 -0600 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Alex Stokes Co-authored-by: Jacek Sieka --- README.md | 2 +- specs/phase0/p2p-interface.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8a340e9ad..b74102c30 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ The current features are: * [Deposit Contract](specs/phase0/deposit-contract.md) * [Honest Validator](specs/phase0/validator.md) * [P2P Networking](specs/phase0/p2p-interface.md) -* [Weak Subjectivity](specs/phase0/weak-subjectivity.md.md) +* [Weak Subjectivity](specs/phase0/weak-subjectivity.md) ### Altair diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index f2c7d8610..69ff7e120 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -752,7 +752,7 @@ Clients MUST keep a record of signed blocks seen on the epoch range where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blocks on this range. -Peers are *repeatedly* unable to reply to Block requests within the +Peers that are unable to reply to block requests within the `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epoch range MAY get descored or disconnected at any time. *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint @@ -763,7 +763,7 @@ proposer signatures and (2) that the blocks form a valid chain up to the most recent block referenced in the weak subjectivity state. *Note*: Although clients that bootstrap from a weak subjectivity checkpoint can begin -participating in the networking immediately, other peers that are actively block syncing MAY +participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client. Clients MUST respond with at least the first block that exists in the range, if they have it, From f52f067b8ea3f8adbebc936207b06459d1956e72 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 12 May 2021 08:36:27 -0600 Subject: [PATCH 6/6] add resourceunavailable error code --- specs/phase0/p2p-interface.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 69ff7e120..297d93948 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -567,6 +567,8 @@ The response code can have one of the following values, encoded as a single unsi The response payload adheres to the `ErrorMessage` schema (described below). - 2: **ServerError** -- the responder encountered an error while processing the request. The response payload adheres to the `ErrorMessage` schema (described below). +- 3: **ResourceUnavailable** -- the responder does not have requested resource. + The response payload adheres to the `ErrorMessage` schema (described below). Clients MAY use response codes above `128` to indicate alternative, erroneous request-specific responses.