From 21cf61ea8f339c36c276003844f002e70f491741 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 15 Dec 2022 11:28:20 +0700 Subject: [PATCH 01/11] De-duplicate range references in BlobSidecarsByRange --- specs/deneb/p2p-interface.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 107c90b9a..51180f8a2 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -256,18 +256,18 @@ The request MUST be encoded as an SSZ-container. The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. -Clients MUST keep a record of signed blobs sidecars seen on the epoch range -`[max(current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]` +Let `blob_serve_range` be `[max(current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]`. +Clients MUST keep a record of signed blob sidecars seen on the epoch range `blob_serve_range` where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blobs on this range. -Peers that are unable to reply to blob sidecar requests within the `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` -epoch range SHOULD respond with error code `3: ResourceUnavailable`. +Peers that are unable to reply to blob sidecar requests within the +range `blob_serve_range` SHOULD respond with error code `3: ResourceUnavailable`. Such peers that are unable to successfully reply to this range of requests MAY get descored or disconnected at any time. *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint -MUST backfill the local blobs database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` +MUST backfill the local blobs database to at least the range `blob_serve_range` to be fully compliant with `BlobSidecarsByRange` requests. *Note*: Although clients that bootstrap from a weak subjectivity checkpoint can begin From f5e00848d0f74cd440b996ae716d58947b868173 Mon Sep 17 00:00:00 2001 From: Mikhail Kalinin Date: Wed, 7 Jun 2023 11:59:51 +0600 Subject: [PATCH 02/11] Remove assert from is_data_available --- specs/deneb/fork-choice.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 283e76b1d..8b3c22423 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -42,8 +42,7 @@ def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZ if isinstance(blobs, str) or isinstance(proofs, str): return True - assert verify_blob_kzg_proof_batch(blobs, blob_kzg_commitments, proofs) - return True + return verify_blob_kzg_proof_batch(blobs, blob_kzg_commitments, proofs) ``` ## Updated fork-choice handlers From b7e21d6caf94980318f4892d35096c2061b10f2c Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Thu, 8 Jun 2023 15:05:46 +0800 Subject: [PATCH 03/11] PR feedback and make the marks italic --- specs/deneb/beacon-chain.md | 8 ++++---- specs/deneb/fork-choice.md | 2 +- specs/deneb/p2p-interface.md | 12 ++++++------ specs/deneb/polynomial-commitments.md | 4 ++-- specs/deneb/validator.md | 10 +++++++--- 5 files changed, 20 insertions(+), 16 deletions(-) diff --git a/specs/deneb/beacon-chain.md b/specs/deneb/beacon-chain.md index 5c3b4e8a4..41ce76bd2 100644 --- a/specs/deneb/beacon-chain.md +++ b/specs/deneb/beacon-chain.md @@ -48,8 +48,8 @@ Deneb is a consensus-layer upgrade containing a number of features. Including: | Name | SSZ equivalent | Description | | - | - | - | -| `VersionedHash` | `Bytes32` | [New in Deneb:EIP4844] | -| `BlobIndex` | `uint64` | [New in Deneb:EIP4844] | +| `VersionedHash` | `Bytes32` | *[New in Deneb:EIP4844]* | +| `BlobIndex` | `uint64` | *[New in Deneb:EIP4844]* | ## Constants @@ -72,8 +72,8 @@ Deneb is a consensus-layer upgrade containing a number of features. Including: | Name | Value | Description | | - | - | - | -| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `uint64(2**12)` (= 4096) | [New in Deneb:EIP4844] hardfork independent fixed theoretical limit same as `LIMIT_BLOBS_PER_TX` (see EIP 4844) | -| `MAX_BLOBS_PER_BLOCK` | `uint64(2**2)` (= 4) | [New in Deneb:EIP4844] Maximum number of blobs in a single block limited by `MAX_BLOB_COMMITMENTS_PER_BLOCK` | +| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `uint64(2**12)` (= 4096) | *[New in Deneb:EIP4844]* hardfork independent fixed theoretical limit same as `LIMIT_BLOBS_PER_TX` (see EIP 4844) | +| `MAX_BLOBS_PER_BLOCK` | `uint64(2**2)` (= 4) | *[New in Deneb:EIP4844]* Maximum number of blobs in a single block limited by `MAX_BLOB_COMMITMENTS_PER_BLOCK` | *Note*: The blob transactions are packed into the execution payload by the EL/builder with their corresponding blobs being independently transmitted and are limited by `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB`. However the CL limit is independently defined by `MAX_BLOBS_PER_BLOCK`. diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 60e05f826..cc7373c09 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -25,7 +25,7 @@ This is the modification of the fork choice accompanying the Deneb upgrade. #### `is_data_available` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* The implementation of `is_data_available` will become more sophisticated during later scaling upgrades. Initially, verification requires every verifying actor to retrieve all matching `Blob`s and `KZGProof`s, and validate them with `verify_blob_kzg_proof_batch`. diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index e9123c309..0a61ef800 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -50,7 +50,7 @@ The specification of these changes continues in the same format as the network s #### `BlobSidecar` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* ```python class BlobSidecar(Container): @@ -66,7 +66,7 @@ class BlobSidecar(Container): #### `SignedBlobSidecar` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* ```python class SignedBlobSidecar(Container): @@ -76,7 +76,7 @@ class SignedBlobSidecar(Container): #### `BlobIdentifier` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* ```python class BlobIdentifier(Container): @@ -130,7 +130,7 @@ New validation: ###### `blob_sidecar_{subnet_id}` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* This topic is used to propagate signed blob sidecars, where each blob index maps to some `subnet_id`. @@ -199,7 +199,7 @@ No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_root/1/` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* The `` field is calculated as `context = compute_fork_digest(fork_version, genesis_validators_root)`: @@ -248,7 +248,7 @@ Clients MAY limit the number of blocks and sidecars in the response. **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/` -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* The `` field is calculated as `context = compute_fork_digest(fork_version, genesis_validators_root)`: diff --git a/specs/deneb/polynomial-commitments.md b/specs/deneb/polynomial-commitments.md index 8a26b9ade..94299aaa2 100644 --- a/specs/deneb/polynomial-commitments.md +++ b/specs/deneb/polynomial-commitments.md @@ -49,7 +49,7 @@ ## Introduction -This document specifies basic polynomial operations and KZG polynomial commitment operations as they are needed for the Deneb specification. The implementations are not optimized for performance, but readability. All practical implementations should optimize the polynomial operations. +This document specifies basic polynomial operations and KZG polynomial commitment operations that are essential for the implementation of the EIP-4844 feature in the Deneb specification. The implementations are not optimized for performance, but readability. All practical implementations should optimize the polynomial operations. Functions flagged as "Public method" MUST be provided by the underlying KZG library as public functions. All other functions are private functions used internally by the KZG library. @@ -336,7 +336,7 @@ def evaluate_polynomial_in_evaluation_form(polynomial: Polynomial, ### KZG -KZG core functions. These are also defined in Deneb execution specs for EIP-4844. +KZG core functions. These are also defined in Deneb execution specs. #### `blob_to_kzg_commitment` diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index 5af7c4938..b53c0a042 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -46,12 +46,14 @@ Please see related Beacon Chain doc before continuing and use them as a referenc | Name | Value | Unit | | - | - | :-: | -| `BLOB_SIDECAR_SUBNET_COUNT` | `4` | The number of blob sidecar subnets used in the gossipsub protocol. | +| `BLOB_SIDECAR_SUBNET_COUNT` | `4` | *[New in Deneb:EIP4844]* The number of blob sidecar subnets used in the gossipsub protocol. | ## Helpers ### `BlobsBundle` +*[New in Deneb:EIP4844]* + ```python @dataclass class BlobsBundle(object): @@ -67,7 +69,7 @@ class BlobsBundle(object): class GetPayloadResponse(object): execution_payload: ExecutionPayload block_value: uint256 - blobs_bundle: BlobsBundle + blobs_bundle: BlobsBundle # [New in Deneb:EIP4844] ``` ## Protocol @@ -98,7 +100,7 @@ All validator responsibilities remain unchanged other than those noted below. ##### Blob KZG commitments -[New in Deneb:EIP4844] +*[New in Deneb:EIP4844]* 1. After retrieving the execution payload from the execution engine as specified in Capella, use the `payload_id` to retrieve `blobs`, `blob_kzg_commitments`, and `blob_kzg_proofs` @@ -107,6 +109,8 @@ via `get_payload(payload_id).blobs_bundle`. #### Constructing the `SignedBlobSidecar`s +*[New in Deneb:EIP4844]* + To construct a `SignedBlobSidecar`, a `signed_blob_sidecar` is defined with the necessary context for block and sidecar proposal. ##### Sidecar From 241e52a20230a970a156b7d9501d9a98ed8954c1 Mon Sep 17 00:00:00 2001 From: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 8 Jun 2023 10:35:03 +0300 Subject: [PATCH 04/11] Whisk (SSLE) with Curdleproofs - rebased (#3342) * Introduce consensus code for Whisk * polish, simplify, clean up (~100 fewer lines) @asn-d6: As discussed, I fixed a few bugs along the way but likely also introduced some bugs :) * minor cleanups and fixes * simplify is_k_commitment_unique * Update beacon-chain.md * Update beacon-chain.md * Initialize `k` in `get_validator_from_deposit()` * minor cleanups * Update beacon-chain.md * Create beacon-chain.md This PR changes the Whisk tracker format to be of the form `(r * pubkey, r * BLS_GT_GENERATOR)` instead of `(r * k * BLS_G1_GENERATOR, r * BLS_G1_GENERATOR)`. This allows for non-interactive tracker registrations from validator pubkeys, removing ~50 lines the code. It also significantly reduces the amount of state overhead. This PR also removes permutation commitments, though those can be easily readded if deemed necessary. * A couple of fixes to the no-registration simplification @asn-d6: Readded a consistency check for `IsValidWhiskOpeningProof` (involving `pubkey` instead of `k_commitment`). * remove unused helpers * use Mary's suggested tracker * Update beacon-chain.md * Revert G_t element optimization This needs its own ethresearch post, and some additional analysis to see if we can do the shuffle ZKP in the allowed timeframe. This reverts commit 8517acabfc1dbb1a35789e6170b5db0bb2c19c9a. * Implement new shuffling strategy Ditch the Feistel logic and instead have each shuffler pick the row they shuffle using their RANDAO reveal. * Curdleproofs edits * working whisk eth2spec * working whisk dummy test * add more boilerplate set up code * rebase constants * Implement even newer and simplified shuffling strategy This commit further simplifies 0faef30fc131d1b471b63a7f16772eeeef548ef8 by removing the entire squareshuffle. The latest version of https://eprint.iacr.org/2022/560 proposes that each shuffler picks random indices from the entire candidate set instead of organizing validators into a square. * Move to _features * remove dummy test * Run doctoc * Change Whisk's previous fork to Capella instead of Bellatrix. Make linter happier. * Fix lint * Fix pylint * Fix mypy issues * Clean-up get_beacon_proposer_index * Fix doc headers * Fix capella link * Update apply_deposit * Rename process_shuffled_trackers --------- Co-authored-by: George Kadianakis Co-authored-by: Justin Co-authored-by: Nalin Bhardwaj Co-authored-by: Hsiao-Wei Wang --- .gitignore | 1 + Makefile | 2 +- setup.py | 39 ++- specs/_features/whisk/beacon-chain.md | 482 ++++++++++++++++++++++++++ specs/_features/whisk/fork.md | 138 ++++++++ 5 files changed, 656 insertions(+), 6 deletions(-) create mode 100644 specs/_features/whisk/beacon-chain.md create mode 100644 specs/_features/whisk/fork.md diff --git a/.gitignore b/.gitignore index 82026c27b..c7012d51b 100644 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,7 @@ tests/core/pyspec/eth2spec/bellatrix/ tests/core/pyspec/eth2spec/capella/ tests/core/pyspec/eth2spec/deneb/ tests/core/pyspec/eth2spec/eip6110/ +tests/core/pyspec/eth2spec/whisk/ # coverage reports .htmlcov diff --git a/Makefile b/Makefile index ab5521663..5c25d59d3 100644 --- a/Makefile +++ b/Makefile @@ -34,7 +34,7 @@ MARKDOWN_FILES = $(wildcard $(SPEC_DIR)/*/*.md) \ $(wildcard $(SPEC_DIR)/_features/*/*/*.md) \ $(wildcard $(SSZ_DIR)/*.md) -ALL_EXECUTABLE_SPECS = phase0 altair bellatrix capella deneb eip6110 +ALL_EXECUTABLE_SPECS = phase0 altair bellatrix capella deneb eip6110 whisk # The parameters for commands. Use `foreach` to avoid listing specs again. COVERAGE_SCOPE := $(foreach S,$(ALL_EXECUTABLE_SPECS), --cov=eth2spec.$S.$(TEST_PRESET_TYPE)) PYLINT_SCOPE := $(foreach S,$(ALL_EXECUTABLE_SPECS), ./eth2spec/$S) diff --git a/setup.py b/setup.py index c9041010d..f4d0c8719 100644 --- a/setup.py +++ b/setup.py @@ -48,6 +48,7 @@ BELLATRIX = 'bellatrix' CAPELLA = 'capella' DENEB = 'deneb' EIP6110 = 'eip6110' +WHISK = 'whisk' # The helper functions that are used when defining constants @@ -733,9 +734,31 @@ from eth2spec.deneb import {preset_name} as deneb ''' +# +# WhiskSpecBuilder +# +class WhiskSpecBuilder(CapellaSpecBuilder): + fork: str = WHISK + + @classmethod + def imports(cls, preset_name: str): + return super().imports(preset_name) + f''' +from eth2spec.capella import {preset_name} as capella +''' + + @classmethod + def hardcoded_custom_type_dep_constants(cls, spec_object) -> str: + # Necessary for custom types `WhiskShuffleProof` and `WhiskTrackerProof` + constants = { + 'WHISK_MAX_SHUFFLE_PROOF_SIZE': spec_object.constant_vars['WHISK_MAX_SHUFFLE_PROOF_SIZE'].value, + 'WHISK_MAX_OPENING_PROOF_SIZE': spec_object.constant_vars['WHISK_MAX_OPENING_PROOF_SIZE'].value, + } + return {**super().hardcoded_custom_type_dep_constants(spec_object), **constants} + + spec_builders = { builder.fork: builder - for builder in (Phase0SpecBuilder, AltairSpecBuilder, BellatrixSpecBuilder, CapellaSpecBuilder, DenebSpecBuilder, EIP6110SpecBuilder) + for builder in (Phase0SpecBuilder, AltairSpecBuilder, BellatrixSpecBuilder, CapellaSpecBuilder, DenebSpecBuilder, EIP6110SpecBuilder, WhiskSpecBuilder) } @@ -1045,7 +1068,7 @@ class PySpecCommand(Command): if len(self.md_doc_paths) == 0: print("no paths were specified, using default markdown file paths for pyspec" " build (spec fork: %s)" % self.spec_fork) - if self.spec_fork in (PHASE0, ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110): + if self.spec_fork in (PHASE0, ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): self.md_doc_paths = """ specs/phase0/beacon-chain.md specs/phase0/fork-choice.md @@ -1053,7 +1076,7 @@ class PySpecCommand(Command): specs/phase0/weak-subjectivity.md specs/phase0/p2p-interface.md """ - if self.spec_fork in (ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110): + if self.spec_fork in (ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): self.md_doc_paths += """ specs/altair/light-client/full-node.md specs/altair/light-client/light-client.md @@ -1065,7 +1088,7 @@ class PySpecCommand(Command): specs/altair/validator.md specs/altair/p2p-interface.md """ - if self.spec_fork in (BELLATRIX, CAPELLA, DENEB, EIP6110): + if self.spec_fork in (BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): self.md_doc_paths += """ specs/bellatrix/beacon-chain.md specs/bellatrix/fork.md @@ -1074,7 +1097,7 @@ class PySpecCommand(Command): specs/bellatrix/p2p-interface.md sync/optimistic.md """ - if self.spec_fork in (CAPELLA, DENEB, EIP6110): + if self.spec_fork in (CAPELLA, DENEB, EIP6110, WHISK): self.md_doc_paths += """ specs/capella/light-client/fork.md specs/capella/light-client/full-node.md @@ -1104,6 +1127,11 @@ class PySpecCommand(Command): specs/_features/eip6110/beacon-chain.md specs/_features/eip6110/fork.md """ + if self.spec_fork == WHISK: + self.md_doc_paths += """ + specs/_features/whisk/beacon-chain.md + specs/_features/whisk/fork.md + """ if len(self.md_doc_paths) == 0: raise Exception('no markdown files specified, and spec fork "%s" is unknown', self.spec_fork) @@ -1259,5 +1287,6 @@ setup( "lru-dict==1.1.8", MARKO_VERSION, "py_arkworks_bls12381==0.3.4", + "curdleproofs @ git+https://github.com/nalinbhardwaj/curdleproofs.pie@master#egg=curdleproofs&subdirectory=curdleproofs", ] ) diff --git a/specs/_features/whisk/beacon-chain.md b/specs/_features/whisk/beacon-chain.md new file mode 100644 index 000000000..7d2ee078f --- /dev/null +++ b/specs/_features/whisk/beacon-chain.md @@ -0,0 +1,482 @@ +# Whisk -- The Beacon Chain + +**Notice**: This document is a work-in-progress for researchers and implementers. + +## Table of contents + + + + + +- [Introduction](#introduction) +- [Constants](#constants) +- [Cryptography](#cryptography) + - [BLS](#bls) + - [Curdleproofs and opening proofs](#curdleproofs-and-opening-proofs) +- [Epoch processing](#epoch-processing) + - [`WhiskTracker`](#whisktracker) + - [`Validator`](#validator) + - [`BeaconState`](#beaconstate) +- [Block processing](#block-processing) + - [Block header](#block-header) + - [`BeaconBlock`](#beaconblock) + - [Whisk](#whisk) + - [`BeaconBlockBody`](#beaconblockbody) + - [Deposits](#deposits) + - [`get_beacon_proposer_index`](#get_beacon_proposer_index) + + + + +## Introduction + +This document details the beacon chain additions and changes of to support the Whisk SSLE, + +*Note:* This specification is built upon [Capella](../../capella/beacon-chain.md) and is under active development. + +## Constants + +| Name | Value | Description | +| ---------------------------------- | -------------------------- | ----------------------------------------------------------- | +| `WHISK_CANDIDATE_TRACKERS_COUNT` | `uint64(2**14)` (= 16,384) | number of candidate trackers | +| `WHISK_PROPOSER_TRACKERS_COUNT` | `uint64(2**13)` (= 8,192) | number of proposer trackers | +| `WHISK_EPOCHS_PER_SHUFFLING_PHASE` | `Epoch(2**8)` (= 256) | epochs per shuffling phase | +| `WHISK_VALIDATORS_PER_SHUFFLE` | `uint64(2**7)` (= 128) | number of validators shuffled per shuffle step | +| `WHISK_PROPOSER_SELECTION_GAP` | `Epoch(2)` | gap between proposer selection and the block proposal phase | +| `WHISK_MAX_SHUFFLE_PROOF_SIZE` | `uint64(2**15)` | max size of a shuffle proof | +| `WHISK_MAX_OPENING_PROOF_SIZE` | `uint64(2**10)` | max size of a opening proof | + +| Name | Value | +| ---------------------------------- | -------------------------- | +| `DOMAIN_WHISK_CANDIDATE_SELECTION` | `DomainType('0x07000000')` | +| `DOMAIN_WHISK_SHUFFLE` | `DomainType('0x07100000')` | +| `DOMAIN_WHISK_PROPOSER_SELECTION` | `DomainType('0x07200000')` | + +## Cryptography + +### BLS + +| Name | SSZ equivalent | Description | +| ------------------- | ---------------------------------------- | ----------------------------- | +| `BLSFieldElement` | `uint256` | BLS12-381 scalar | +| `BLSG1Point` | `Bytes48` | compressed BLS12-381 G1 point | +| `WhiskShuffleProof` | `ByteList[WHISK_MAX_SHUFFLE_PROOF_SIZE]` | Serialized shuffle proof | +| `WhiskTrackerProof` | `ByteList[WHISK_MAX_OPENING_PROOF_SIZE]` | Serialized tracker proof | + +*Note*: A subgroup check MUST be performed when deserializing a `BLSG1Point` for use in any of the functions below. + +```python +def BLSG1ScalarMultiply(scalar: BLSFieldElement, point: BLSG1Point) -> BLSG1Point: + return bls.G1_to_bytes48(bls.multiply(bls.bytes48_to_G1(point), scalar)) +``` + +```python +def bytes_to_bls_field(b: Bytes32) -> BLSFieldElement: + """ + Convert bytes to a BLS field scalar. The output is not uniform over the BLS field. + TODO: Deneb will introduces this helper too. Should delete it once it's rebased to post-Deneb. + """ + field_element = int.from_bytes(b, ENDIANNESS) + assert field_element < BLS_MODULUS + return BLSFieldElement(field_element) +``` + +| Name | Value | +| ------------------ | ------------------------------------------------------------------------------- | +| `BLS_G1_GENERATOR` | `bls.G1_to_bytes48(bls.G1)` | +| `BLS_MODULUS` | `52435875175126190479447740508185965837690552500527637822603658699938581184513` | + +### Curdleproofs and opening proofs + +Note that Curdleproofs (Whisk Shuffle Proofs), the tracker opening proofs and all related data structures and verifier code (along with tests) is specified in [curdleproofs.pie](https://github.com/nalinbhardwaj/curdleproofs.pie/tree/verifier-only) repository. + +```python +def IsValidWhiskShuffleProof(pre_shuffle_trackers: Sequence[WhiskTracker], + post_shuffle_trackers: Sequence[WhiskTracker], + M: BLSG1Point, + shuffle_proof: WhiskShuffleProof) -> bool: + """ + Verify `post_shuffle_trackers` is a permutation of `pre_shuffle_trackers`. + Defined in https://github.com/nalinbhardwaj/curdleproofs.pie/tree/verifier-only. + """ + # pylint: disable=unused-argument + return True +``` + +```python +def IsValidWhiskOpeningProof(tracker: WhiskTracker, + k_commitment: BLSG1Point, + tracker_proof: WhiskTrackerProof) -> bool: + """ + Verify knowledge of `k` such that `tracker.k_r_G == k * tracker.r_G` and `k_commitment == k * BLS_G1_GENERATOR`. + Defined in https://github.com/nalinbhardwaj/curdleproofs.pie/tree/verifier-only. + """ + # pylint: disable=unused-argument + return True +``` + +## Epoch processing + +### `WhiskTracker` + +```python +class WhiskTracker(Container): + r_G: BLSG1Point # r * G + k_r_G: BLSG1Point # k * r * G +``` + +### `Validator` + +```python +class Validator(Container): + pubkey: BLSPubkey + withdrawal_credentials: Bytes32 # Commitment to pubkey for withdrawals + effective_balance: Gwei # Balance at stake + slashed: boolean + # Status epochs + activation_eligibility_epoch: Epoch # When criteria for activation were met + activation_epoch: Epoch + exit_epoch: Epoch + withdrawable_epoch: Epoch # When validator can withdraw funds + whisk_tracker: WhiskTracker # Whisk tracker (r * G, k * r * G) [New in Whisk] + whisk_k_commitment: BLSG1Point # Whisk k commitment k * BLS_G1_GENERATOR [New in Whisk] +``` + +### `BeaconState` + +```python +class BeaconState(Container): + # Versioning + genesis_time: uint64 + genesis_validators_root: Root + slot: Slot + fork: Fork + # History + latest_block_header: BeaconBlockHeader + block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] # Frozen in Capella, replaced by historical_summaries + # Eth1 + eth1_data: Eth1Data + eth1_data_votes: List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH] + eth1_deposit_index: uint64 + # Registry + validators: List[Validator, VALIDATOR_REGISTRY_LIMIT] # [Modified in Whisk] + balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT] + # Randomness + randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR] + # Slashings + slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances + # Participation + previous_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT] + current_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT] + # Finality + justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch + previous_justified_checkpoint: Checkpoint + current_justified_checkpoint: Checkpoint + finalized_checkpoint: Checkpoint + # Inactivity + inactivity_scores: List[uint64, VALIDATOR_REGISTRY_LIMIT] + # Sync + current_sync_committee: SyncCommittee + next_sync_committee: SyncCommittee + # Execution + latest_execution_payload_header: ExecutionPayloadHeader + # Withdrawals + next_withdrawal_index: WithdrawalIndex + next_withdrawal_validator_index: ValidatorIndex + # Deep history valid from Capella onwards + historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT] + whisk_candidate_trackers: Vector[WhiskTracker, WHISK_CANDIDATE_TRACKERS_COUNT] # [New in Whisk] + whisk_proposer_trackers: Vector[WhiskTracker, WHISK_PROPOSER_TRACKERS_COUNT] # [New in Whisk] +``` + +```python +def select_whisk_trackers(state: BeaconState, epoch: Epoch) -> None: + # Select proposer trackers from candidate trackers + proposer_seed = get_seed(state, epoch - WHISK_PROPOSER_SELECTION_GAP, DOMAIN_WHISK_PROPOSER_SELECTION) + for i in range(WHISK_PROPOSER_TRACKERS_COUNT): + index = compute_shuffled_index(uint64(i), uint64(len(state.whisk_candidate_trackers)), proposer_seed) + state.whisk_proposer_trackers[i] = state.whisk_candidate_trackers[index] + + # Select candidate trackers from active validator trackers + active_validator_indices = get_active_validator_indices(state, epoch) + for i in range(WHISK_CANDIDATE_TRACKERS_COUNT): + seed = hash(get_seed(state, epoch, DOMAIN_WHISK_CANDIDATE_SELECTION) + uint_to_bytes(i)) + candidate_index = compute_proposer_index(state, active_validator_indices, seed) # sample by effective balance + state.whisk_candidate_trackers[i] = state.validators[candidate_index].whisk_tracker +``` + +```python +def process_whisk_updates(state: BeaconState) -> None: + next_epoch = Epoch(get_current_epoch(state) + 1) + if next_epoch % WHISK_EPOCHS_PER_SHUFFLING_PHASE == 0: # select trackers at the start of shuffling phases + select_whisk_trackers(state, next_epoch) +``` + +```python +def process_epoch(state: BeaconState) -> None: + process_justification_and_finalization(state) + process_inactivity_updates(state) + process_rewards_and_penalties(state) + process_registry_updates(state) + process_slashings(state) + process_eth1_data_reset(state) + process_effective_balance_updates(state) + process_slashings_reset(state) + process_randao_mixes_reset(state) + process_historical_summaries_update(state) + process_participation_flag_updates(state) + process_sync_committee_updates(state) + process_whisk_updates(state) # [New in Whisk] +``` + +## Block processing + +### Block header + +#### `BeaconBlock` + +```python +class BeaconBlock(Container): + slot: Slot + proposer_index: ValidatorIndex + parent_root: Root + state_root: Root + body: BeaconBlockBody + whisk_opening_proof: WhiskTrackerProof # [New in Whisk] +``` + +```python +def process_whisk_opening_proof(state: BeaconState, block: BeaconBlock) -> None: + tracker = state.whisk_proposer_trackers[state.slot % WHISK_PROPOSER_TRACKERS_COUNT] + k_commitment = state.validators[block.proposer_index].whisk_k_commitment + assert IsValidWhiskOpeningProof(tracker, k_commitment, block.whisk_opening_proof) +``` + +Removed `assert block.proposer_index == get_beacon_proposer_index(state)` check in Whisk. + +```python +def process_block_header(state: BeaconState, block: BeaconBlock) -> None: + # Verify that the slots match + assert block.slot == state.slot + # Verify that the block is newer than latest block header + assert block.slot > state.latest_block_header.slot + + # # Verify that proposer index is the correct index + # assert block.proposer_index == get_beacon_proposer_index(state) + + # Verify that the parent matches + assert block.parent_root == hash_tree_root(state.latest_block_header) + # Cache current block as the new latest block + state.latest_block_header = BeaconBlockHeader( + slot=block.slot, + proposer_index=block.proposer_index, + parent_root=block.parent_root, + state_root=Bytes32(), # Overwritten in the next process_slot call + body_root=hash_tree_root(block.body), + ) + + # Verify proposer is not slashed + proposer = state.validators[block.proposer_index] + assert not proposer.slashed + process_whisk_opening_proof(state, block) # [New in Whisk] +``` + +### Whisk + +#### `BeaconBlockBody` + +```python +class BeaconBlockBody(capella.BeaconBlockBody): + randao_reveal: BLSSignature + eth1_data: Eth1Data # Eth1 data vote + graffiti: Bytes32 # Arbitrary data + # Operations + proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS] + attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS] + attestations: List[Attestation, MAX_ATTESTATIONS] + deposits: List[Deposit, MAX_DEPOSITS] + voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] + sync_aggregate: SyncAggregate + # Execution + execution_payload: ExecutionPayload + # Capella operations + bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] + whisk_post_shuffle_trackers: Vector[WhiskTracker, WHISK_VALIDATORS_PER_SHUFFLE] # [New in Whisk] + whisk_shuffle_proof: WhiskShuffleProof # [New in Whisk] + whisk_shuffle_proof_M_commitment: BLSG1Point # [New in Whisk] + whisk_registration_proof: WhiskTrackerProof # [New in Whisk] + whisk_tracker: WhiskTracker # [New in Whisk] + whisk_k_commitment: BLSG1Point # [New in Whisk] +``` + +```python +def get_shuffle_indices(randao_reveal: BLSSignature) -> Sequence[uint64]: + """ + Given a `randao_reveal` return the list of indices that got shuffled from the entire candidate set + """ + indices = [] + for i in WHISK_VALIDATORS_PER_SHUFFLE: + # XXX ensure we are not suffering from modulo bias + shuffle_index = uint256(hash(randao_reveal + uint_to_bytes(i))) % WHISK_CANDIDATE_TRACKERS_COUNT + indices.append(shuffle_index) + + return indices +``` + +```python +def process_shuffled_trackers(state: BeaconState, body: BeaconBlockBody) -> None: + # Check the shuffle proof + shuffle_indices = get_shuffle_indices(body.randao_reveal) + pre_shuffle_trackers = [state.whisk_candidate_trackers[i] for i in shuffle_indices] + post_shuffle_trackers = body.whisk_post_shuffle_trackers + + shuffle_epoch = get_current_epoch(state) % WHISK_EPOCHS_PER_SHUFFLING_PHASE + if shuffle_epoch + WHISK_PROPOSER_SELECTION_GAP + 1 >= WHISK_EPOCHS_PER_SHUFFLING_PHASE: + # Require unchanged trackers during cooldown + assert pre_shuffle_trackers == post_shuffle_trackers + else: + # Require shuffled trackers during shuffle + assert IsValidWhiskShuffleProof( + pre_shuffle_trackers, + post_shuffle_trackers, + body.whisk_shuffle_proof_M_commitment, + body.whisk_shuffle_proof, + ) + + # Shuffle candidate trackers + for i, shuffle_index in enumerate(shuffle_indices): + state.whisk_candidate_trackers[shuffle_index] = post_shuffle_trackers[i] +``` + +```python +def is_k_commitment_unique(state: BeaconState, k_commitment: BLSG1Point) -> bool: + return all([validator.whisk_k_commitment != k_commitment for validator in state.validators]) +``` + +```python +def process_whisk(state: BeaconState, body: BeaconBlockBody) -> None: + process_shuffled_trackers(state, body) + + # Overwrite all validator Whisk fields (first Whisk proposal) or just the permutation commitment (next proposals) + proposer = state.validators[get_beacon_proposer_index(state)] + if proposer.whisk_tracker.r_G == BLS_G1_GENERATOR: # first Whisk proposal + assert body.whisk_tracker.r_G != BLS_G1_GENERATOR + assert is_k_commitment_unique(state, body.whisk_k_commitment) + assert IsValidWhiskOpeningProof( + body.whisk_tracker, + body.whisk_k_commitment, + body.whisk_registration_proof, + ) + proposer.whisk_tracker = body.whisk_tracker + proposer.whisk_k_commitment = body.whisk_k_commitment + else: # next Whisk proposals + assert body.whisk_registration_proof == WhiskTrackerProof() + assert body.whisk_tracker == WhiskTracker() + assert body.whisk_k_commitment == BLSG1Point() + assert body.whisk_shuffle_proof_M_commitment == BLSG1Point() +``` + +```python +def process_block(state: BeaconState, block: BeaconBlock) -> None: + process_block_header(state, block) + if is_execution_enabled(state, block.body): + process_withdrawals(state, block.body.execution_payload) + process_execution_payload(state, block.body.execution_payload, EXECUTION_ENGINE) + process_randao(state, block.body) + process_eth1_data(state, block.body) + process_operations(state, block.body) + process_sync_aggregate(state, block.body.sync_aggregate) + process_whisk(state, block.body) # [New in Whisk] +``` + +### Deposits + +```python +def get_unique_whisk_k(state: BeaconState, validator_index: ValidatorIndex) -> BLSFieldElement: + counter = 0 + while True: + # hash `validator_index || counter` + k = BLSFieldElement(bytes_to_bls_field(hash(uint_to_bytes(validator_index) + uint_to_bytes(uint64(counter))))) + if is_k_commitment_unique(state, BLSG1ScalarMultiply(k, BLS_G1_GENERATOR)): + return k # unique by trial and error + counter += 1 +``` + +```python +def get_initial_commitments(k: BLSFieldElement) -> Tuple[BLSG1Point, WhiskTracker]: + return ( + BLSG1ScalarMultiply(k, BLS_G1_GENERATOR), + WhiskTracker(r_G=BLS_G1_GENERATOR, k_r_G=BLSG1ScalarMultiply(k, BLS_G1_GENERATOR)) + ) +``` + +```python +def get_validator_from_deposit_whisk( + state: BeaconState, + pubkey: BLSPubkey, + withdrawal_credentials: Bytes32, + amount: uint64 +) -> Validator: + effective_balance = min(amount - amount % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE) + k = get_unique_whisk_k(state, ValidatorIndex(len(state.validators))) + whisk_k_commitment, whisk_tracker = get_initial_commitments(k) + + validator = Validator( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + activation_eligibility_epoch=FAR_FUTURE_EPOCH, + activation_epoch=FAR_FUTURE_EPOCH, + exit_epoch=FAR_FUTURE_EPOCH, + withdrawable_epoch=FAR_FUTURE_EPOCH, + effective_balance=effective_balance, + # Whisk fields + whisk_tracker=whisk_tracker, + whisk_k_commitment=whisk_k_commitment, + ) + return validator +``` + +```python +def apply_deposit(state: BeaconState, + pubkey: BLSPubkey, + withdrawal_credentials: Bytes32, + amount: uint64, + signature: BLSSignature) -> None: + validator_pubkeys = [validator.pubkey for validator in state.validators] + if pubkey not in validator_pubkeys: + # Verify the deposit signature (proof of possession) which is not checked by the deposit contract + deposit_message = DepositMessage( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + amount=amount, + ) + domain = compute_domain(DOMAIN_DEPOSIT) # Fork-agnostic domain since deposits are valid across forks + signing_root = compute_signing_root(deposit_message, domain) + # Initialize validator if the deposit signature is valid + if bls.Verify(pubkey, signing_root, signature): + index = get_index_for_new_validator(state) + validator = get_validator_from_deposit_whisk(state, pubkey, withdrawal_credentials, amount) + set_or_append_list(state.validators, index, validator) + set_or_append_list(state.balances, index, amount) + # [New in Altair] + set_or_append_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) + set_or_append_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) + set_or_append_list(state.inactivity_scores, index, uint64(0)) + else: + # Increase balance by deposit amount + index = ValidatorIndex(validator_pubkeys.index(pubkey)) + increase_balance(state, index, amount) +``` + +### `get_beacon_proposer_index` + +```python +def get_beacon_proposer_index(state: BeaconState) -> ValidatorIndex: + """ + Return the beacon proposer index at the current slot. + """ + assert state.latest_block_header.slot == state.slot # sanity check `process_block_header` has been called + return state.latest_block_header.proposer_index +``` diff --git a/specs/_features/whisk/fork.md b/specs/_features/whisk/fork.md new file mode 100644 index 000000000..189d8c5b3 --- /dev/null +++ b/specs/_features/whisk/fork.md @@ -0,0 +1,138 @@ +# Whisk -- Fork Logic + +**Notice**: This document is a work-in-progress for researchers and implementers. + +## Table of contents + + + + + +- [Introduction](#introduction) +- [Configuration](#configuration) +- [Fork to Whisk](#fork-to-whisk) + + + + +## Introduction + +This document describes the process of Whisk upgrade. + + +``` +""" + WHISK_FORK_EPOCH + | cooldown + | | || + v vsvv + --+~~~~~~~~~~~~~~~~~~~~~----+- + shuffling ^ + | + | + proposer selection + candidate selection +""" +``` + +## Configuration + +Warning: this configuration is not definitive. + +| Name | Value | +| -------------------- | ----------------------- | +| `WHISK_FORK_VERSION` | `Version('0x05000000')` | +| `WHISK_FORK_EPOCH` | `Epoch(18446744073709551615)` **TBD** | + +## Fork to Whisk + +If `state.slot % SLOTS_PER_EPOCH == 0` and `compute_epoch_at_slot(state.slot) == WHISK_FORK_EPOCH`, an irregular state change is made to upgrade to Whisk. `WHISK_FORK_EPOCH` must be a multiple of `WHISK_RUN_DURATION_IN_EPOCHS`. + +The upgrade occurs after the completion of the inner loop of `process_slots` that sets `state.slot` equal to `WHISK_FORK_EPOCH * SLOTS_PER_EPOCH`. + +This ensures that we drop right into the beginning of the shuffling phase but without `process_whisk_epoch()` triggering for this Whisk run. Hence we handle all the setup ourselves in `upgrade_to_whisk()` below. + +```python +def whisk_candidate_selection(state: BeaconState, epoch: Epoch) -> None: + # TODO + # pylint: disable=unused-argument + pass +``` + +```python +def whisk_proposer_selection(state: BeaconState, epoch: Epoch) -> None: + # TODO + # pylint: disable=unused-argument + pass +``` + +```python +def upgrade_to_whisk(pre: bellatrix.BeaconState) -> BeaconState: + epoch = bellatrix.get_current_epoch(pre) + post = BeaconState( + # Versioning + genesis_time=pre.genesis_time, + genesis_validators_root=pre.genesis_validators_root, + slot=pre.slot, + fork=Fork( + previous_version=pre.fork.current_version, + current_version=WHISK_FORK_VERSION, + epoch=epoch, + ), + # History + latest_block_header=pre.latest_block_header, + block_roots=pre.block_roots, + state_roots=pre.state_roots, + historical_roots=pre.historical_roots, + # Eth1 + eth1_data=pre.eth1_data, + eth1_data_votes=pre.eth1_data_votes, + eth1_deposit_index=pre.eth1_deposit_index, + # Registry + validators=[], + balances=pre.balances, + # Randomness + randao_mixes=pre.randao_mixes, + # Slashings + slashings=pre.slashings, + # Participation + previous_epoch_participation=pre.previous_epoch_participation, + current_epoch_participation=pre.current_epoch_participation, + # Finality + justification_bits=pre.justification_bits, + previous_justified_checkpoint=pre.previous_justified_checkpoint, + current_justified_checkpoint=pre.current_justified_checkpoint, + finalized_checkpoint=pre.finalized_checkpoint, + # Inactivity + inactivity_scores=pre.inactivity_Scores, + ) + + # Initialize all validators with predictable commitments + for val_index, pre_validator in enumerate(pre.validators): + whisk_commitment, whisk_tracker = get_initial_commitments(get_unique_whisk_k(post, ValidatorIndex(val_index))) + + post_validator = Validator( + pubkey=pre_validator.pubkey, + withdrawal_credentials=pre_validator.withdrawal_credentials, + effective_balance=pre_validator.effective_balance, + slashed=pre_validator.slashed, + activation_eligibility_epoch=pre_validator.activation_eligibility_epoch, + activation_epoch=pre_validator.activation_epoch, + exit_epoch=pre_validator.exit_epoch, + withdrawable_epoch=pre_validator.withdrawable_epoch, + whisk_commitment=whisk_commitment, + whisk_tracker=whisk_tracker, + ) + post.validators.append(post_validator) + + # Do a candidate selection followed by a proposer selection so that we have proposers for the upcoming day + # Use an old epoch when selecting candidates so that we don't get the same seed as in the next candidate selection + whisk_candidate_selection(post, epoch - WHISK_PROPOSER_SELECTION_GAP - 1) + whisk_proposer_selection(post, epoch) + + # Do a final round of candidate selection. + # We need it so that we have something to shuffle over the upcoming shuffling phase. + whisk_candidate_selection(post, epoch) + + return post +``` From 68bb5ee5079ababf7be4cf7b6a1e0a9cada09f43 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 8 Jun 2023 15:00:49 +0300 Subject: [PATCH 05/11] Ensure make lint resolves to the right pylint and mypy versions --- Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 5c25d59d3..6c852a1e9 100644 --- a/Makefile +++ b/Makefile @@ -146,8 +146,8 @@ codespell: lint: pyspec . venv/bin/activate; cd $(PY_SPEC_DIR); \ flake8 --config $(LINTER_CONFIG_FILE) ./eth2spec \ - && pylint --rcfile $(LINTER_CONFIG_FILE) $(PYLINT_SCOPE) \ - && mypy --config-file $(LINTER_CONFIG_FILE) $(MYPY_SCOPE) + && python -m pylint --rcfile $(LINTER_CONFIG_FILE) $(PYLINT_SCOPE) \ + && python -m mypy --config-file $(LINTER_CONFIG_FILE) $(MYPY_SCOPE) lint_generators: pyspec . venv/bin/activate; cd $(TEST_GENERATORS_DIR); \ From 706824cf959df3aaa5aef543d573aaae00c411a3 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 8 Jun 2023 20:25:03 +0300 Subject: [PATCH 06/11] Drop need to declare spec feature files --- setup.py | 121 ++++++++++++++++++++++++++----------------------------- 1 file changed, 57 insertions(+), 64 deletions(-) diff --git a/setup.py b/setup.py index f4d0c8719..f9b2438d4 100644 --- a/setup.py +++ b/setup.py @@ -50,6 +50,25 @@ DENEB = 'deneb' EIP6110 = 'eip6110' WHISK = 'whisk' +PREVIOUS_FORK_OF = { + PHASE0: None, + ALTAIR: PHASE0, + BELLATRIX: ALTAIR, + CAPELLA: BELLATRIX, + DENEB: CAPELLA, + EIP6110: DENEB, + WHISK: CAPELLA, +} + +ALL_FORKS = list(PREVIOUS_FORK_OF.keys()) + +IGNORE_SPEC_FILES = [ + "specs/phase0/deposit-contract.md" +] + +EXTRA_SPEC_FILES = { + BELLATRIX: "sync/optimistic.md" +} # The helper functions that are used when defining constants CONSTANT_DEP_SUNDRY_CONSTANTS_FUNCTIONS = ''' @@ -96,6 +115,30 @@ class SpecObject(NamedTuple): dataclasses: Dict[str, str] +def is_post_fork(a, b) -> bool: + """ + Returns true if fork a is after b, or if a == b + """ + if a == b: + return True + + prev_fork = PREVIOUS_FORK_OF[a] + if prev_fork == b: + return True + elif prev_fork == None: + return False + else: + return is_post_fork(prev_fork, b) + +def get_fork_directory(fork): + dir1 = f'specs/{fork}' + if os.path.exists(dir1): + return dir1 + dir2 = f'specs/_features/{fork}' + if os.path.exists(dir2): + return dir2 + raise FileNotFoundError(f"No directory found for fork: {fork}") + def _get_name_from_heading(heading: Heading) -> Optional[str]: last_child = heading.children[-1] if isinstance(last_child, CodeSpan): @@ -1068,70 +1111,20 @@ class PySpecCommand(Command): if len(self.md_doc_paths) == 0: print("no paths were specified, using default markdown file paths for pyspec" " build (spec fork: %s)" % self.spec_fork) - if self.spec_fork in (PHASE0, ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): - self.md_doc_paths = """ - specs/phase0/beacon-chain.md - specs/phase0/fork-choice.md - specs/phase0/validator.md - specs/phase0/weak-subjectivity.md - specs/phase0/p2p-interface.md - """ - if self.spec_fork in (ALTAIR, BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): - self.md_doc_paths += """ - specs/altair/light-client/full-node.md - specs/altair/light-client/light-client.md - specs/altair/light-client/p2p-interface.md - specs/altair/light-client/sync-protocol.md - specs/altair/beacon-chain.md - specs/altair/bls.md - specs/altair/fork.md - specs/altair/validator.md - specs/altair/p2p-interface.md - """ - if self.spec_fork in (BELLATRIX, CAPELLA, DENEB, EIP6110, WHISK): - self.md_doc_paths += """ - specs/bellatrix/beacon-chain.md - specs/bellatrix/fork.md - specs/bellatrix/fork-choice.md - specs/bellatrix/validator.md - specs/bellatrix/p2p-interface.md - sync/optimistic.md - """ - if self.spec_fork in (CAPELLA, DENEB, EIP6110, WHISK): - self.md_doc_paths += """ - specs/capella/light-client/fork.md - specs/capella/light-client/full-node.md - specs/capella/light-client/p2p-interface.md - specs/capella/light-client/sync-protocol.md - specs/capella/beacon-chain.md - specs/capella/fork.md - specs/capella/fork-choice.md - specs/capella/validator.md - specs/capella/p2p-interface.md - """ - if self.spec_fork in (DENEB, EIP6110): - self.md_doc_paths += """ - specs/deneb/light-client/fork.md - specs/deneb/light-client/full-node.md - specs/deneb/light-client/p2p-interface.md - specs/deneb/light-client/sync-protocol.md - specs/deneb/beacon-chain.md - specs/deneb/fork.md - specs/deneb/fork-choice.md - specs/deneb/polynomial-commitments.md - specs/deneb/p2p-interface.md - specs/deneb/validator.md - """ - if self.spec_fork == EIP6110: - self.md_doc_paths += """ - specs/_features/eip6110/beacon-chain.md - specs/_features/eip6110/fork.md - """ - if self.spec_fork == WHISK: - self.md_doc_paths += """ - specs/_features/whisk/beacon-chain.md - specs/_features/whisk/fork.md - """ + self.md_doc_paths = "" + + for fork in ALL_FORKS: + if is_post_fork(self.spec_fork, fork): + # Append all files in fork directory recursively + for root, dirs, files in os.walk(get_fork_directory(fork)): + for filename in files: + filepath = os.path.join(root, filename) + if filepath.endswith('.md') and filepath not in IGNORE_SPEC_FILES: + self.md_doc_paths += filepath + "\n" + # Append extra files if any + if fork in EXTRA_SPEC_FILES: + self.md_doc_paths += EXTRA_SPEC_FILES[fork] + "\n" + if len(self.md_doc_paths) == 0: raise Exception('no markdown files specified, and spec fork "%s" is unknown', self.spec_fork) From efd2429b3f14f7bccaad342a75fdf843b21c0951 Mon Sep 17 00:00:00 2001 From: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri, 9 Jun 2023 11:28:06 +0300 Subject: [PATCH 07/11] Move whisk_opening_proof to block body (#3408) --- specs/_features/whisk/beacon-chain.md | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/specs/_features/whisk/beacon-chain.md b/specs/_features/whisk/beacon-chain.md index 7d2ee078f..bad818837 100644 --- a/specs/_features/whisk/beacon-chain.md +++ b/specs/_features/whisk/beacon-chain.md @@ -19,7 +19,6 @@ - [`BeaconState`](#beaconstate) - [Block processing](#block-processing) - [Block header](#block-header) - - [`BeaconBlock`](#beaconblock) - [Whisk](#whisk) - [`BeaconBlockBody`](#beaconblockbody) - [Deposits](#deposits) @@ -235,23 +234,11 @@ def process_epoch(state: BeaconState) -> None: ### Block header -#### `BeaconBlock` - -```python -class BeaconBlock(Container): - slot: Slot - proposer_index: ValidatorIndex - parent_root: Root - state_root: Root - body: BeaconBlockBody - whisk_opening_proof: WhiskTrackerProof # [New in Whisk] -``` - ```python def process_whisk_opening_proof(state: BeaconState, block: BeaconBlock) -> None: tracker = state.whisk_proposer_trackers[state.slot % WHISK_PROPOSER_TRACKERS_COUNT] k_commitment = state.validators[block.proposer_index].whisk_k_commitment - assert IsValidWhiskOpeningProof(tracker, k_commitment, block.whisk_opening_proof) + assert IsValidWhiskOpeningProof(tracker, k_commitment, block.body.whisk_opening_proof) ``` Removed `assert block.proposer_index == get_beacon_proposer_index(state)` check in Whisk. @@ -303,6 +290,8 @@ class BeaconBlockBody(capella.BeaconBlockBody): execution_payload: ExecutionPayload # Capella operations bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES] + # Whisk + whisk_opening_proof: WhiskTrackerProof # [New in Whisk] whisk_post_shuffle_trackers: Vector[WhiskTracker, WHISK_VALIDATORS_PER_SHUFFLE] # [New in Whisk] whisk_shuffle_proof: WhiskShuffleProof # [New in Whisk] whisk_shuffle_proof_M_commitment: BLSG1Point # [New in Whisk] From 7b69f176e2e836ccff5b58f655b39b9c583f00cd Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 9 Jun 2023 08:54:16 -0600 Subject: [PATCH 08/11] bump VERSION.txt --- tests/core/pyspec/eth2spec/VERSION.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/core/pyspec/eth2spec/VERSION.txt b/tests/core/pyspec/eth2spec/VERSION.txt index 2a6fa03c0..54dd11e46 100644 --- a/tests/core/pyspec/eth2spec/VERSION.txt +++ b/tests/core/pyspec/eth2spec/VERSION.txt @@ -1 +1 @@ -1.4.0-alpha.1 +1.4.0-alpha.2 From 45f3432019336b5cb1196037e6f5d92a6dc5aeba Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Fri, 9 Jun 2023 23:10:00 +0800 Subject: [PATCH 09/11] Update `MAX_BLOBS_PER_BLOCK` to `6` and add Deneb networking configs to yaml files (#3410) --- configs/mainnet.yaml | 8 ++++++++ configs/minimal.yaml | 8 ++++++++ presets/mainnet/deneb.yaml | 4 ++-- presets/minimal/deneb.yaml | 4 ++-- specs/deneb/beacon-chain.md | 2 +- specs/deneb/fork-choice.md | 3 ++- .../test/deneb/unittests/test_config_invariants.py | 8 ++++++++ 7 files changed, 31 insertions(+), 6 deletions(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index baf489739..80a240185 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -124,3 +124,11 @@ ATTESTATION_SUBNET_COUNT: 64 ATTESTATION_SUBNET_EXTRA_BITS: 0 # ceillog2(ATTESTATION_SUBNET_COUNT) + ATTESTATION_SUBNET_EXTRA_BITS ATTESTATION_SUBNET_PREFIX_BITS: 6 + +# Deneb +# `2**7` (=128) +MAX_REQUEST_BLOCKS_DENEB: 128 +# MAX_REQUEST_BLOCKS_DENEB * MAX_BLOBS_PER_BLOCK +MAX_REQUEST_BLOB_SIDECARS: 768 +# `2**12` (= 4096 epochs, ~18 days) +MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 4096 diff --git a/configs/minimal.yaml b/configs/minimal.yaml index 43f1fc83f..82a813a9b 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -125,3 +125,11 @@ ATTESTATION_SUBNET_COUNT: 64 ATTESTATION_SUBNET_EXTRA_BITS: 0 # ceillog2(ATTESTATION_SUBNET_COUNT) + ATTESTATION_SUBNET_EXTRA_BITS ATTESTATION_SUBNET_PREFIX_BITS: 6 + +# Deneb +# `2**7` (=128) +MAX_REQUEST_BLOCKS_DENEB: 128 +# MAX_REQUEST_BLOCKS_DENEB * MAX_BLOBS_PER_BLOCK +MAX_REQUEST_BLOB_SIDECARS: 768 +# `2**12` (= 4096 epochs, ~18 days) +MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 4096 diff --git a/presets/mainnet/deneb.yaml b/presets/mainnet/deneb.yaml index 10c5025ed..23889fd18 100644 --- a/presets/mainnet/deneb.yaml +++ b/presets/mainnet/deneb.yaml @@ -6,5 +6,5 @@ FIELD_ELEMENTS_PER_BLOB: 4096 # `uint64(2**12)` (= 4096) MAX_BLOB_COMMITMENTS_PER_BLOCK: 4096 -# `uint64(2**2)` (= 4) -MAX_BLOBS_PER_BLOCK: 4 +# `uint64(6)` +MAX_BLOBS_PER_BLOCK: 6 diff --git a/presets/minimal/deneb.yaml b/presets/minimal/deneb.yaml index 91120f9da..e21d38777 100644 --- a/presets/minimal/deneb.yaml +++ b/presets/minimal/deneb.yaml @@ -6,5 +6,5 @@ FIELD_ELEMENTS_PER_BLOB: 4 # [customized] MAX_BLOB_COMMITMENTS_PER_BLOCK: 16 -# `uint64(2**2)` (= 4) -MAX_BLOBS_PER_BLOCK: 4 +# `uint64(6)` +MAX_BLOBS_PER_BLOCK: 6 diff --git a/specs/deneb/beacon-chain.md b/specs/deneb/beacon-chain.md index 41ce76bd2..148dc96d8 100644 --- a/specs/deneb/beacon-chain.md +++ b/specs/deneb/beacon-chain.md @@ -73,7 +73,7 @@ Deneb is a consensus-layer upgrade containing a number of features. Including: | Name | Value | Description | | - | - | - | | `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `uint64(2**12)` (= 4096) | *[New in Deneb:EIP4844]* hardfork independent fixed theoretical limit same as `LIMIT_BLOBS_PER_TX` (see EIP 4844) | -| `MAX_BLOBS_PER_BLOCK` | `uint64(2**2)` (= 4) | *[New in Deneb:EIP4844]* Maximum number of blobs in a single block limited by `MAX_BLOB_COMMITMENTS_PER_BLOCK` | +| `MAX_BLOBS_PER_BLOCK` | `uint64(6)` | *[New in Deneb:EIP4844]* maximum number of blobs in a single block limited by `MAX_BLOB_COMMITMENTS_PER_BLOCK` | *Note*: The blob transactions are packed into the execution payload by the EL/builder with their corresponding blobs being independently transmitted and are limited by `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB`. However the CL limit is independently defined by `MAX_BLOBS_PER_BLOCK`. diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index c67906259..bbc7fa0f8 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -36,7 +36,8 @@ The block MUST NOT be considered valid until all valid `Blob`s have been downloa def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: # `retrieve_blobs_and_proofs` is implementation and context dependent # It returns all the blobs for the given block root, and raises an exception if not available - # Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` + # Note: the p2p network does not guarantee sidecar retrieval outside of + # `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` blobs, proofs = retrieve_blobs_and_proofs(beacon_block_root) # For testing, `retrieve_blobs_and_proofs` returns ("TEST", "TEST"). diff --git a/tests/core/pyspec/eth2spec/test/deneb/unittests/test_config_invariants.py b/tests/core/pyspec/eth2spec/test/deneb/unittests/test_config_invariants.py index 13a54225e..75955ed16 100644 --- a/tests/core/pyspec/eth2spec/test/deneb/unittests/test_config_invariants.py +++ b/tests/core/pyspec/eth2spec/test/deneb/unittests/test_config_invariants.py @@ -10,3 +10,11 @@ from eth2spec.test.context import ( @single_phase def test_length(spec): assert spec.MAX_BLOBS_PER_BLOCK < spec.MAX_BLOB_COMMITMENTS_PER_BLOCK + + +@with_deneb_and_later +@spec_test +@single_phase +def test_networking(spec): + assert spec.MAX_BLOBS_PER_BLOCK < spec.MAX_BLOB_COMMITMENTS_PER_BLOCK + assert spec.config.MAX_REQUEST_BLOB_SIDECARS == spec.config.MAX_REQUEST_BLOCKS_DENEB * spec.MAX_BLOBS_PER_BLOCK From 5576d0e6850b0ddbffd1ec122841b35e31e66fee Mon Sep 17 00:00:00 2001 From: Paul Harris Date: Sat, 10 Jun 2023 01:12:38 +1000 Subject: [PATCH 10/11] Moved configuration into network preset instead of constants. (#3394) * Moved configuration into network preset instead of constants. Now that `MAX_CHUNK_SIZE` and `GOSSIP_MAX_SIZE` are in configuration, we no longer need separate constants to represent them in the spec when they change in Bellatrix. I've changed the usage, and put the values into the presets, but I'm not sure if I've updated the descriptions in the best way... This is following on from the work in #3375 where a number of constants got moved into configuration, so we no longer need these constants to be separately represented, they can simply be updated in presets. * Update presets/minimal/bellatrix.yaml Co-authored-by: Hsiao-Wei Wang * Update presets/mainnet/bellatrix.yaml Co-authored-by: Hsiao-Wei Wang * Moved preset items into the correct section and updated TOC. It looked like the items listed in configuration about the max size and chunk size were no longer needed since we're updating preset values now and the preset changes seem to only be listed in the changes at the top. * review feedback * hopefully correct this time! Moved the 2 fields from configs into presets completely as suggested. * WIP - changing back to being in config and updating the phase 0 value... I think this should be close but want to see what's outstanding. * fix intellij's formatting of table. * more fixes --------- Co-authored-by: Hsiao-Wei Wang --- configs/mainnet.yaml | 8 ++++---- configs/minimal.yaml | 8 ++++---- presets/mainnet/phase0.yaml | 2 +- presets/minimal/phase0.yaml | 2 +- specs/bellatrix/p2p-interface.md | 28 ++-------------------------- specs/phase0/p2p-interface.md | 4 ++-- 6 files changed, 14 insertions(+), 38 deletions(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index 80a240185..fe405cb38 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -98,16 +98,16 @@ DEPOSIT_CONTRACT_ADDRESS: 0x00000000219ab540356cBB839Cbe05303d7705Fa # Networking # --------------------------------------------------------------- -# `2**20` (= 1048576, 1 MiB) -GOSSIP_MAX_SIZE: 1048576 +# `10 * 2**20` (= 10485760, 10 MiB) +GOSSIP_MAX_SIZE: 10485760 # `2**10` (= 1024) MAX_REQUEST_BLOCKS: 1024 # `2**8` (= 256) EPOCHS_PER_SUBNET_SUBSCRIPTION: 256 # `MIN_VALIDATOR_WITHDRAWABILITY_DELAY + CHURN_LIMIT_QUOTIENT // 2` (= 33024, ~5 months) MIN_EPOCHS_FOR_BLOCK_REQUESTS: 33024 -# `2**20` (=1048576, 1 MiB) -MAX_CHUNK_SIZE: 1048576 +# `10 * 2**20` (=10485760, 10 MiB) +MAX_CHUNK_SIZE: 10485760 # 5s TTFB_TIMEOUT: 5 # 10s diff --git a/configs/minimal.yaml b/configs/minimal.yaml index 82a813a9b..dcc834963 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -99,16 +99,16 @@ DEPOSIT_CONTRACT_ADDRESS: 0x1234567890123456789012345678901234567890 # Networking # --------------------------------------------------------------- -# `2**20` (= 1048576, 1 MiB) -GOSSIP_MAX_SIZE: 1048576 +# `10 * 2**20` (= 10485760, 10 MiB) +GOSSIP_MAX_SIZE: 10485760 # `2**10` (= 1024) MAX_REQUEST_BLOCKS: 1024 # `2**8` (= 256) EPOCHS_PER_SUBNET_SUBSCRIPTION: 256 # [customized] `MIN_VALIDATOR_WITHDRAWABILITY_DELAY + CHURN_LIMIT_QUOTIENT // 2` (= 272) MIN_EPOCHS_FOR_BLOCK_REQUESTS: 272 -# `2**20` (=1048576, 1 MiB) -MAX_CHUNK_SIZE: 1048576 +# `10 * 2**20` (=10485760, 10 MiB) +MAX_CHUNK_SIZE: 10485760 # 5s TTFB_TIMEOUT: 5 # 10s diff --git a/presets/mainnet/phase0.yaml b/presets/mainnet/phase0.yaml index 02bc96c8c..00133ba36 100644 --- a/presets/mainnet/phase0.yaml +++ b/presets/mainnet/phase0.yaml @@ -85,4 +85,4 @@ MAX_ATTESTATIONS: 128 # 2**4 (= 16) MAX_DEPOSITS: 16 # 2**4 (= 16) -MAX_VOLUNTARY_EXITS: 16 +MAX_VOLUNTARY_EXITS: 16 \ No newline at end of file diff --git a/presets/minimal/phase0.yaml b/presets/minimal/phase0.yaml index e7028f5a4..d9a6a2b6c 100644 --- a/presets/minimal/phase0.yaml +++ b/presets/minimal/phase0.yaml @@ -85,4 +85,4 @@ MAX_ATTESTATIONS: 128 # 2**4 (= 16) MAX_DEPOSITS: 16 # 2**4 (= 16) -MAX_VOLUNTARY_EXITS: 16 +MAX_VOLUNTARY_EXITS: 16 \ No newline at end of file diff --git a/specs/bellatrix/p2p-interface.md b/specs/bellatrix/p2p-interface.md index b8b3a11d6..7d80d40a8 100644 --- a/specs/bellatrix/p2p-interface.md +++ b/specs/bellatrix/p2p-interface.md @@ -14,7 +14,6 @@ Readers should understand the Phase 0 and Altair documents and use them as a bas - [Warning](#warning) - [Modifications in Bellatrix](#modifications-in-bellatrix) - - [Configuration](#configuration) - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - [Topics and messages](#topics-and-messages) - [Global topics](#global-topics) @@ -41,15 +40,6 @@ Refer to the note in the [validator guide](./validator.md) for further details. ## Modifications in Bellatrix -### Configuration - -This section outlines modifications constants that are used in this spec. - -| Name | Value | Description | -|---|---|---| -| `GOSSIP_MAX_SIZE_BELLATRIX` | `10 * 2**20` (= 10,485,760, 10 MiB) | The maximum allowed size of uncompressed gossip messages starting at Bellatrix upgrade. | -| `MAX_CHUNK_SIZE_BELLATRIX` | `10 * 2**20` (= 10,485,760, 10 MiB) | The maximum allowed size of uncompressed req/resp chunked responses starting at Bellatrix upgrade. | - ### The gossip domain: gossipsub Some gossip meshes are upgraded in Bellatrix to support upgraded types. @@ -61,11 +51,6 @@ All topics remain stable except the beacon block topic which is updated with the The specification around the creation, validation, and dissemination of messages has not changed from the Phase 0 and Altair documents unless explicitly noted here. -Starting at Bellatrix upgrade, each gossipsub [message](https://github.com/libp2p/go-libp2p-pubsub/blob/master/pb/rpc.proto#L17-L24) -has a maximum size of `GOSSIP_MAX_SIZE_BELLATRIX`. -Clients MUST reject (fail validation) messages that are over this size limit. -Likewise, clients MUST NOT emit or propagate messages larger than this limit. - The derivation of the `message-id` remains stable. The new topics along with the type of the `data` field of a gossipsub message are given in this table: @@ -130,10 +115,6 @@ down-scoring or disconnection. Request and Response remain unchanged unless explicitly noted here. -Starting at Bellatrix upgrade, -a global maximum uncompressed byte size of `MAX_CHUNK_SIZE_BELLATRIX` MUST be applied to all method response chunks -regardless of type specific bounds that *MUST* also be respected. - Bellatrix fork-digest is introduced to the `context` enum to specify Bellatrix block type. Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: @@ -171,17 +152,12 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: With the addition of `ExecutionPayload` to `BeaconBlock`s, there is a dynamic field -- `transactions` -- which can validly exceed the `GOSSIP_MAX_SIZE` limit (1 MiB) put in -place at Phase 0. At the `GAS_LIMIT` (~30M) currently seen on mainnet in 2021, a single transaction +place at Phase 0, so GOSSIP_MAX_SIZE has increased to 10 Mib on the network. +At the `GAS_LIMIT` (~30M) currently seen on mainnet in 2021, a single transaction filled entirely with data at a cost of 16 gas per byte can create a valid `ExecutionPayload` of ~2 MiB. Thus we need a size limit to at least account for current mainnet conditions. -Geth currently has a [max gossip message size](https://github.com/ethereum/go-ethereum/blob/3ce9f6d96f38712f5d6756e97b59ccc20cc403b3/eth/protocols/eth/protocol.go#L49) of 10 MiB. -To support backward compatibility with this previously defined network limit, -we adopt `GOSSIP_MAX_SIZE_BELLATRIX` of 10 MiB for maximum gossip sizes at the -point of Bellatrix and beyond. Note, that clients SHOULD still reject objects -that exceed their maximum theoretical bounds which in most cases is less than `GOSSIP_MAX_SIZE_BELLATRIX`. - Note, that due to additional size induced by the `BeaconBlock` contents (e.g. proposer signature, operations lists, etc) this does reduce the theoretical max valid `ExecutionPayload` (and `transactions` list) size as diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index de3e0e529..c0d18b08f 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -189,11 +189,11 @@ This section outlines configurations that are used in this spec. | Name | Value | Description | |---|---|---| -| `GOSSIP_MAX_SIZE` | `2**20` (= 1048576, 1 MiB) | The maximum allowed size of uncompressed gossip messages. | +| `GOSSIP_MAX_SIZE` | `10 * 2**20` (= 10485760, 10 MiB) | The maximum allowed size of uncompressed gossip messages. | | `MAX_REQUEST_BLOCKS` | `2**10` (= 1024) | Maximum number of blocks in a single request | | `EPOCHS_PER_SUBNET_SUBSCRIPTION` | `2**8` (= 256) | Number of epochs on a subnet subscription (~27 hours) | | `MIN_EPOCHS_FOR_BLOCK_REQUESTS` | `MIN_VALIDATOR_WITHDRAWABILITY_DELAY + CHURN_LIMIT_QUOTIENT // 2` (= 33024, ~5 months) | The minimum epoch range over which a node must serve blocks | -| `MAX_CHUNK_SIZE` | `2**20` (=1048576, 1 MiB) | The maximum allowed size of uncompressed req/resp chunked responses. | +| `MAX_CHUNK_SIZE` | `10 * 2**20` (=10485760, 10 MiB) | The maximum allowed size of uncompressed req/resp chunked responses. | | `TTFB_TIMEOUT` | `5` | The maximum duration in **seconds** to wait for first byte of request response (time-to-first-byte). | | `RESP_TIMEOUT` | `10` | The maximum duration in **seconds** for complete response transfer. | | `ATTESTATION_PROPAGATION_SLOT_RANGE` | `32` | The maximum number of slots during which an attestation can be propagated. | From cfd44dac56996b8195434277392686fc319b70d2 Mon Sep 17 00:00:00 2001 From: William Dowling Date: Sun, 11 Jun 2023 15:07:39 +0200 Subject: [PATCH 11/11] Update lru-dict package to latest stable release. lru-dict build fails when building under clang version 16 (https://github.com/amitdev/lru-dict/commit/6badf6376d12a2e0498f67bbc7d5c0332b96a4d7). This bumps the version to the latest stable release. Tested on MacOS M2 and Ubuntu Jammy x86-64. --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index f4d0c8719..adf24ba36 100644 --- a/setup.py +++ b/setup.py @@ -1284,7 +1284,7 @@ setup( "remerkleable==0.1.27", "trie==2.0.2", RUAMEL_YAML_VERSION, - "lru-dict==1.1.8", + "lru-dict==1.2.0", MARKO_VERSION, "py_arkworks_bls12381==0.3.4", "curdleproofs @ git+https://github.com/nalinbhardwaj/curdleproofs.pie@master#egg=curdleproofs&subdirectory=curdleproofs",