From afea8a10a5f0e1ba3b2e0035a8e40325b0df0de2 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 11:59:59 +0100 Subject: [PATCH 1/8] Rework the TODO Probably missed a few. Will try to keep up to date. --- specs/casper_sharding_v2.1.md | 40 ++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index c0d2c231c..04c7a2312 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -581,27 +581,33 @@ Finally: * Let `next_start_shard = (shard_and_committee_for_slots[-1][-1].shard_id + 1) % SHARD_COUNT` * Set `shard_and_committee_for_slots[CYCLE_LENGTH:] = get_new_shuffling(block.ancestor_hashes[0], validators, next_start_shard)` -------- +### TODO -Note: this is ~80% complete. The main sections that are missing are: +Note: This spec is ~60% complete. -* Logic for the formats of shard chains, who proposes shard blocks, etc. (in an initial release, if desired we could make crosslinks just be Merkle roots of blobs of data; in any case, one can philosophically view the whole point of the shard chains as being a coordination device for choosing what blobs of data to propose as crosslinks) -* Logic for inducting queued validators from the PoW chain -* Penalties for signing or attesting to non-canonical-chain blocks (update: may not be necessary, see https://ethresear.ch/t/attestation-committee-based-full-pos-chains/2259) -* Per-validator proofs of custody, and associated slashing conditions -* Versioning and upgrades +* [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed +* [ ] Specify the shard chain blocks, proposers, etc. +* [ ] Fully specify the registration contract on the PoW chain +* [ ] Flesh out RANDAO, including the hardening against orphaned reveals +* [ ] Add per-validator proofs of custody, including slashing conditions +* [ ] Clearly define all the terms in the glossary +* [ ] Add an appendix for BLS12-381 +* [ ] Add an appendix for the offchain signature aggregation logic +* [ ] Rework the document for readability +* [ ] Undergo peer review, security audits and formal verification -Slashing conditions may include: +**Possible changes and additions** - - Casper FFG slot equivocation [done] - Casper FFG surround [done] - Beacon chain proposal equivocation [done] - Shard chain proposal equivocation - Proof of custody secret leak - Proof of custody wrong custody bit - Proof of custody no secret reveal - RANDAO leak +* [ ] Replace Blake with a STARK-friendly hash function +* [ ] Replacing the IMD fork choice rule with LMD +* [ ] Merge `crystallized_state_root` and `active_state_root` into a single root +* [ ] Add Merklelisation of the state root(s) for light clients +* [ ] Add logic for versioning and upgrades +* [ ] Get rid of dynasties +* [ ] Add a RANDAO slashing condition for early leakage +* [ ] Reworking the `ShardAndCommittee` data structures +* [ ] Reduce the slot duration to 8 seconds +* [ ] Allow for the delayed inclusion of aggregated signatures # Appendix ## Appendix A - Hash function From 21da063d3cc0c543f79fadab2e0395c54c2b16c8 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 12:04:17 +0100 Subject: [PATCH 2/8] Update casper_sharding_v2.1.md --- specs/casper_sharding_v2.1.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index 04c7a2312..117b670f7 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -586,28 +586,29 @@ Finally: Note: This spec is ~60% complete. * [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed -* [ ] Specify the shard chain blocks, proposers, etc. +* [ ] Specify the shard chain blocks, blobs, proposers, etc. * [ ] Fully specify the registration contract on the PoW chain * [ ] Flesh out RANDAO, including the hardening against orphaned reveals * [ ] Add per-validator proofs of custody, including slashing conditions -* [ ] Clearly define all the terms in the glossary * [ ] Add an appendix for BLS12-381 -* [ ] Add an appendix for the offchain signature aggregation logic +* [ ] Add an appendix on gossip networks and the offchain signature aggregation logic +* [ ] Clearly define all the terms in the glossary * [ ] Rework the document for readability * [ ] Undergo peer review, security audits and formal verification **Possible changes and additions** * [ ] Replace Blake with a STARK-friendly hash function -* [ ] Replacing the IMD fork choice rule with LMD +* [ ] Replace the IMD fork choice rule with LMD * [ ] Merge `crystallized_state_root` and `active_state_root` into a single root * [ ] Add Merklelisation of the state root(s) for light clients * [ ] Add logic for versioning and upgrades * [ ] Get rid of dynasties * [ ] Add a RANDAO slashing condition for early leakage -* [ ] Reworking the `ShardAndCommittee` data structures +* [ ] Reworke the `ShardAndCommittee` data structures * [ ] Reduce the slot duration to 8 seconds * [ ] Allow for the delayed inclusion of aggregated signatures +* [ ] Consider separate networking-optimised serialisation formats # Appendix ## Appendix A - Hash function From 1c3b38a7496319d90920b8331b74975496ed4ed4 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 12:10:52 +0100 Subject: [PATCH 3/8] Update casper_sharding_v2.1.md --- specs/casper_sharding_v2.1.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index 117b670f7..d072a804e 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -587,17 +587,24 @@ Note: This spec is ~60% complete. * [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed * [ ] Specify the shard chain blocks, blobs, proposers, etc. +* [ ] Add a double-batched Merkle accumulator for beacon chain blocks +* [ ] Specify the various assumptions (global clock, validator honesty, validator liveness, etc.) * [ ] Fully specify the registration contract on the PoW chain * [ ] Flesh out RANDAO, including the hardening against orphaned reveals * [ ] Add per-validator proofs of custody, including slashing conditions +* [ ] Use a separate hash function for the proof of possession * [ ] Add an appendix for BLS12-381 * [ ] Add an appendix on gossip networks and the offchain signature aggregation logic -* [ ] Clearly define all the terms in the glossary +* [ ] Comprehensively and clearly define all the terms in the glossary * [ ] Rework the document for readability * [ ] Undergo peer review, security audits and formal verification **Possible changes and additions** +* [ ] Deprecate Wei and use 64-bit balances, consistent with a 64-bit EVM2.0 +* [ ] Allow for deposits larger than 32 ETH, as well as deposit top ups +* [ ] Having penalties for having a deposit below 32 ETH +* [ ] Add a `SpecialObject` to change the `withdrawal_shard_id`, `withdrawal_address` or `randao_commitment` * [ ] Replace Blake with a STARK-friendly hash function * [ ] Replace the IMD fork choice rule with LMD * [ ] Merge `crystallized_state_root` and `active_state_root` into a single root @@ -605,7 +612,7 @@ Note: This spec is ~60% complete. * [ ] Add logic for versioning and upgrades * [ ] Get rid of dynasties * [ ] Add a RANDAO slashing condition for early leakage -* [ ] Reworke the `ShardAndCommittee` data structures +* [ ] Rework the `ShardAndCommittee` data structures * [ ] Reduce the slot duration to 8 seconds * [ ] Allow for the delayed inclusion of aggregated signatures * [ ] Consider separate networking-optimised serialisation formats From ca7bb7426e4adc05f3a7ec8e1c56127caff71895 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 12:32:16 +0100 Subject: [PATCH 4/8] Update casper_sharding_v2.1.md --- specs/casper_sharding_v2.1.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index d072a804e..453537cb0 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -587,15 +587,17 @@ Note: This spec is ~60% complete. * [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed * [ ] Specify the shard chain blocks, blobs, proposers, etc. -* [ ] Add a double-batched Merkle accumulator for beacon chain blocks +* [ ] Specify the rules for forced deregistrations +* [ ] Add a double-batched Merkle accumulator for historical beacon chain blocks * [ ] Specify the various assumptions (global clock, validator honesty, validator liveness, etc.) -* [ ] Fully specify the registration contract on the PoW chain -* [ ] Flesh out RANDAO, including the hardening against orphaned reveals +* [ ] Specify in Solidity the registration contract on the PoW chain +* [ ] Specify the RANDAO logic, including the hardening against orphaned reveals * [ ] Add per-validator proofs of custody, including slashing conditions * [ ] Use a separate hash function for the proof of possession * [ ] Add an appendix for BLS12-381 * [ ] Add an appendix on gossip networks and the offchain signature aggregation logic * [ ] Comprehensively and clearly define all the terms in the glossary +* [ ] Clearly document the various edge cases, e.g. with committee sizing * [ ] Rework the document for readability * [ ] Undergo peer review, security audits and formal verification @@ -603,7 +605,7 @@ Note: This spec is ~60% complete. * [ ] Deprecate Wei and use 64-bit balances, consistent with a 64-bit EVM2.0 * [ ] Allow for deposits larger than 32 ETH, as well as deposit top ups -* [ ] Having penalties for having a deposit below 32 ETH +* [ ] Add penalties for a deposit below 32 ETH (or some other threshold) * [ ] Add a `SpecialObject` to change the `withdrawal_shard_id`, `withdrawal_address` or `randao_commitment` * [ ] Replace Blake with a STARK-friendly hash function * [ ] Replace the IMD fork choice rule with LMD From 24c8a53b5c7be0248015413b6c0f8586e79d6b67 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 14:29:56 +0100 Subject: [PATCH 5/8] Update casper_sharding_v2.1.md --- specs/casper_sharding_v2.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index 453537cb0..f7ab010bb 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -590,7 +590,7 @@ Note: This spec is ~60% complete. * [ ] Specify the rules for forced deregistrations * [ ] Add a double-batched Merkle accumulator for historical beacon chain blocks * [ ] Specify the various assumptions (global clock, validator honesty, validator liveness, etc.) -* [ ] Specify in Solidity the registration contract on the PoW chain +* [ ] Specify in Vyper the registration contract on the PoW chain * [ ] Specify the RANDAO logic, including the hardening against orphaned reveals * [ ] Add per-validator proofs of custody, including slashing conditions * [ ] Use a separate hash function for the proof of possession From 538e4e1f887e39ecdf44ee583d1ce3d1c84d1f00 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 21:35:47 +0100 Subject: [PATCH 6/8] Rename simpleserialize.md to simple-serialize.md For consistency with beacon-chain.md --- specs/{simpleserialize.md => simple-serialize.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename specs/{simpleserialize.md => simple-serialize.md} (100%) diff --git a/specs/simpleserialize.md b/specs/simple-serialize.md similarity index 100% rename from specs/simpleserialize.md rename to specs/simple-serialize.md From 2f3469161544335ad82b423d4dbb235519af72ff Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 22:00:05 +0100 Subject: [PATCH 7/8] Minor cleanups --- specs/beacon-chain.md | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/specs/beacon-chain.md b/specs/beacon-chain.md index a7de5e8cd..53e97814c 100644 --- a/specs/beacon-chain.md +++ b/specs/beacon-chain.md @@ -36,7 +36,6 @@ The primary source of load on the beacon chain are "attestations". Attestations | `SHARD_COUNT` | 2**10 (= 1,024)| shards | | `DEPOSIT_SIZE` | 2**5 (= 32) | ETH | | `MIN_COMMITTEE_SIZE` | 2**7 (= 128) | validators | -| `MAX_VALIDATOR_COUNT` | 2**22 ( = 4,194,304) | validators | | `GENESIS_TIME` | **TBD** | seconds | | `SLOT_DURATION` | 2**4 (= 16) | seconds | | `CYCLE_LENGTH` | 2**6 (= 64) | slots | ~17 minutes | @@ -48,7 +47,6 @@ The primary source of load on the beacon chain are "attestations". Attestations **Notes** -* At most `MAX_VALIDATOR_COUNT * DEPOSIT_SIZE` (~134 million ETH) can be staked. * The `SQRT_E_DROP_TIME` constant is the amount of time it takes for the quadratic leak to cut deposits of non-participating validators by ~39.4%. * The `BASE_REWARD_QUOTIENT` constant is the per-slot interest rate assuming all validators are participating, assuming total deposits of 1 ETH. It corresponds to ~3.88% annual interest assuming 10 million participating ETH. * At most `1/MAX_VALIDATOR_CHURN_QUOTIENT` of the validators can change during each dynasty. @@ -68,15 +66,9 @@ The primary source of load on the beacon chain are "attestations". Attestations ### PoW chain registration contract -The initial deployment phases of Ethereum 2.0 are implemented without consensus changes to the PoW chain. A registration contract is added to the PoW chain to deposit ETH. This contract has a `registration` function which takes the following arguments: +The initial deployment phases of Ethereum 2.0 are implemented without consensus changes to the PoW chain. A registration contract is added to the PoW chain to deposit ETH. This contract has a `registration` function which takes as arguments `pubkey`, `withdrawal_shard`, `withdrawal_address`, `randao_commitment` as defined in a `ValidatorRecord` below. A BLS `proof_of_possession` of types `bytes` is given as a final argument. -1) `pubkey` (bytes) -2) `withdrawal_shard_id` (int) -3) `withdrawal_address` (address) -4) `randao_commitment` (bytes32) -5) `bls_proof_of_possession` (bytes) - -The registration contract does minimal validation, pushing most of the registration logic to the beacon chain. In particular, the BLS proof of possession (based on the BLS12-381 curve) is not verified by the registration contract. +The registration contract emits a log with the various arguments for consumption by the beacon chain. It does not do validation, pushing the registration logic to the beacon chain. In particular, the proof of possession (based on the BLS12-381 curve) is not verified by the registration contract. ## Data Structures @@ -86,11 +78,11 @@ Beacon chain block structure: ```python fields = { - # Hash of ancestor blocks (32 items, i'th is 2**i'th ancestor or zero bytes) + # Skip list of ancestor block hashes. The i'th item is 2**i'th ancestor (or zero bytes) for i = 0, ..., 31 'ancestor_hashes': ['hash32'], - # Slot number (for the PoS mechanism) + # Slot number 'slot': 'int64', - # Randao commitment reveal + # RANDAO commitment reveal 'randao_reveal': 'hash32', # Attestations 'attestations': [AttestationRecord], @@ -363,7 +355,7 @@ def get_block_hash(active_state, curblock, slot): return active_state.recent_block_hashes[slot - earliest_slot_in_array] ``` -`get_block_hash(_, _, h)` should always return the block in the chain at slot `h`, and `get_shards_and_committees_for_slot(_, h)` should not change unless the dynasty changes. +`get_block_hash(_, _, s)` should always return the block in the chain at slot `s`, and `get_shards_and_committees_for_slot(_, s)` should not change unless the dynasty changes. We define a function to "add a link" to the validator hash chain, used when a validator is added or removed: From 9a05c79e3ecf1fafe2ceeb71bee0ff1207dc458f Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 3 Oct 2018 22:15:43 +0100 Subject: [PATCH 8/8] Update casper_sharding_v2.1.md --- specs/casper_sharding_v2.1.md | 47 ++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/specs/casper_sharding_v2.1.md b/specs/casper_sharding_v2.1.md index f7ab010bb..5c4106cbf 100644 --- a/specs/casper_sharding_v2.1.md +++ b/specs/casper_sharding_v2.1.md @@ -585,39 +585,40 @@ Finally: Note: This spec is ~60% complete. -* [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed +**Missing** + +* [ ] Specify how `crystallized_state_root` and `active_state_root` are constructed, including Merklelisation logic for light clients +* [ ] Specify the rules around acceptable values for `pow_chain_ref` * [ ] Specify the shard chain blocks, blobs, proposers, etc. * [ ] Specify the rules for forced deregistrations -* [ ] Add a double-batched Merkle accumulator for historical beacon chain blocks -* [ ] Specify the various assumptions (global clock, validator honesty, validator liveness, etc.) -* [ ] Specify in Vyper the registration contract on the PoW chain -* [ ] Specify the RANDAO logic, including the hardening against orphaned reveals -* [ ] Add per-validator proofs of custody, including slashing conditions -* [ ] Use a separate hash function for the proof of possession -* [ ] Add an appendix for BLS12-381 +* [ ] Specify the various assumptions (global clock, networking latency, validator honesty, validator liveness, etc.) +* [ ] Specify (in a separate Vyper file) the registration contract on the PoW chain +* [ ] Specify the bootstrapping logic for the beacon chain genesis (e.g. specify a minimum number validators before the genesis block) +* [ ] Specify the logic for proofs of custody, including slashing conditions +* [ ] Add an appendix about the BLS12-381 curve * [ ] Add an appendix on gossip networks and the offchain signature aggregation logic -* [ ] Comprehensively and clearly define all the terms in the glossary -* [ ] Clearly document the various edge cases, e.g. with committee sizing -* [ ] Rework the document for readability +* [ ] Add a glossary (in a separate `glossary.md`) to comprehensively and precisely define all the terms * [ ] Undergo peer review, security audits and formal verification -**Possible changes and additions** +**Possible rework/additions** -* [ ] Deprecate Wei and use 64-bit balances, consistent with a 64-bit EVM2.0 -* [ ] Allow for deposits larger than 32 ETH, as well as deposit top ups -* [ ] Add penalties for a deposit below 32 ETH (or some other threshold) -* [ ] Add a `SpecialObject` to change the `withdrawal_shard_id`, `withdrawal_address` or `randao_commitment` -* [ ] Replace Blake with a STARK-friendly hash function * [ ] Replace the IMD fork choice rule with LMD -* [ ] Merge `crystallized_state_root` and `active_state_root` into a single root -* [ ] Add Merklelisation of the state root(s) for light clients -* [ ] Add logic for versioning and upgrades +* [ ] Merklelise `crystallized_state_root` and `active_state_root` into a single root +* [ ] Replace Blake with a STARK-friendly hash function * [ ] Get rid of dynasties -* [ ] Add a RANDAO slashing condition for early leakage -* [ ] Rework the `ShardAndCommittee` data structures * [ ] Reduce the slot duration to 8 seconds * [ ] Allow for the delayed inclusion of aggregated signatures -* [ ] Consider separate networking-optimised serialisation formats +* [ ] Use a separate networking-optimised serialisation format for networking +* [ ] Harden RANDAO against orphaned reveals +* [ ] Introduce a RANDAO slashing condition for early leakage +* [ ] Use a separate hash function for the proof of possession +* [ ] Rework the `ShardAndCommittee` data structures +* [ ] Add a double-batched Merkle accumulator for historical beacon chain blocks +* [ ] Allow for deposits larger than 32 ETH, as well as deposit top-ups +* [ ] Add penalties for a deposit below 32 ETH (or some other threshold) +* [ ] Add a `SpecialObject` to (re)register +* [ ] Rework the document for readability +* [ ] Clearly document the various edge cases, e.g. with committee sizing # Appendix ## Appendix A - Hash function