Update run time to v0.9.3 (#4154)

* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

.bazelrc

@@ -16,9 +16,9 @@ run --host_force_python=PY2
 --experimental_sandbox_default_allow_network=false
 
 # Use minimal protobufs at runtime
-run --define ssz=minimal
+run --define ssz=mainnet
 test --define ssz=mainnet
-build --define ssz=minimal
+build --define ssz=mainnet
 
 # Prevent PATH changes from rebuilding when switching from IDE to command line.
 build --incompatible_strict_action_env

README.md

@@ -1,7 +1,7 @@
 # Prysm: An Ethereum 2.0 Client Written in Go
 
 [![Build status](https://badge.buildkite.com/b555891daf3614bae4284dcf365b2340cefc0089839526f096.svg?branch=master)](https://buildkite.com/prysmatic-labs/prysm)
-[![ETH2.0_Spec_Version 0.9.0](https://img.shields.io/badge/ETH2.0%20Spec%20Version-v0.9.0-blue.svg)](https://github.com/ethereum/eth2.0-specs/tree/v0.9.0)
+[![ETH2.0_Spec_Version 0.9.3](https://img.shields.io/badge/ETH2.0%20Spec%20Version-v0.9.3-blue.svg)](https://github.com/ethereum/eth2.0-specs/tree/v0.9.3)
 [![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/KSA7rPr)
 [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/prysmaticlabs/geth-sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 

WORKSPACE

@@ -149,8 +149,8 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    sha256 = "5c5b65a961b5e7251435efc9548648b45142a07993ad3e100850c240cb76e9af",
+    sha256 = "72c6ee3c20d19736b1203f364a6eb0ddee2c173073e20bee2beccd288fdc42be",
-    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.0/general.tar.gz",
+    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.4/general.tar.gz",
 )
 
 http_archive(
@@ -165,8 +165,8 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    sha256 = "3b5f0168af4331d09da52bebc26609def9d11be3e6c784ce7c3df3596617808d",
+    sha256 = "a3cc860a3679f6f62ee57b65677a9b48a65fdebb151cdcbf50f23852632845ef",
-    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.0/minimal.tar.gz",
+    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.4/minimal.tar.gz",
 )
 
 http_archive(
@@ -181,8 +181,8 @@ filegroup(
     visibility = ["//visibility:public"],
 )
     """,
-    sha256 = "f3ff68508dfe9696f23506daf0ca895cda955e30398741e00cffa33a01b0565c",
+    sha256 = "8fc1b6220973ca30fa4ddc4ed24d66b1719abadca8bedb5e06c3bd9bc0df28e9",
-    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.0/mainnet.tar.gz",
+    url = "https://github.com/ethereum/eth2.0-spec-tests/releases/download/v0.9.4/mainnet.tar.gz",
 )
 
 http_archive(
@@ -259,7 +259,7 @@ go_repository(
 
 go_repository(
     name = "com_github_prysmaticlabs_go_ssz",
-    commit = "142dfef39d12ed28360b7d2467b056b0578684f5",
+    commit = "e24db4d9e9637cf88ee9e4a779e339a1686a84ee",
     importpath = "github.com/prysmaticlabs/go-ssz",
 )
 
@@ -1255,7 +1255,7 @@ go_repository(
 
 go_repository(
     name = "com_github_prysmaticlabs_ethereumapis",
-    commit = "2a889fed542ad00e4bd3caf723f871b6a4eff63d",
+    commit = "87118fb893cc6f32b25793d819790fd3bcce3221",
     importpath = "github.com/prysmaticlabs/ethereumapis",
     patch_args = ["-p1"],
     patches = [

beacon-chain/blockchain/BUILD.bazel

@@ -22,14 +22,14 @@ go_library(
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/core/state:go_default_library",
         "//beacon-chain/db:go_default_library",
-        "//beacon-chain/operations:go_default_library",
+        "//beacon-chain/operations/attestations:go_default_library",
         "//beacon-chain/p2p:go_default_library",
         "//beacon-chain/powchain:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/params:go_default_library",
-        "//shared/runutil:go_default_library",
+        "//shared/slotutil:go_default_library",
         "//shared/traceutil:go_default_library",
         "@com_github_gogo_protobuf//proto:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
@@ -67,7 +67,6 @@ go_test(
         "//beacon-chain/core/state:go_default_library",
         "//beacon-chain/db:go_default_library",
         "//beacon-chain/db/testing:go_default_library",
-        "//beacon-chain/operations/testing:go_default_library",
         "//beacon-chain/p2p:go_default_library",
         "//beacon-chain/powchain:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",

beacon-chain/blockchain/chain_info.go

@@ -1,6 +1,7 @@
 package blockchain
 
 import (
+	"bytes"
 	"context"
 	"time"
 
@@ -29,7 +30,7 @@ type GenesisTimeFetcher interface {
 type HeadFetcher interface {
 	HeadSlot() uint64
 	HeadRoot() []byte
-	HeadBlock() *ethpb.BeaconBlock
+	HeadBlock() *ethpb.SignedBeaconBlock
 	HeadState(ctx context.Context) (*pb.BeaconState, error)
 	HeadValidatorsIndices(epoch uint64) ([]uint64, error)
 	HeadSeed(epoch uint64) ([32]byte, error)
@@ -60,6 +61,12 @@ func (s *Service) FinalizedCheckpt() *ethpb.Checkpoint {
 		return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
 	}
 
+	// If head state exists but there hasn't been a finalized check point,
+	// the check point's root should refer to genesis block root.
+	if bytes.Equal(s.headState.FinalizedCheckpoint.Root, params.BeaconConfig().ZeroHash[:]) {
+		return &ethpb.Checkpoint{Root: s.genesisRoot[:]}
+	}
+
 	return s.headState.FinalizedCheckpoint
 }
 
@@ -69,6 +76,12 @@ func (s *Service) CurrentJustifiedCheckpt() *ethpb.Checkpoint {
 		return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
 	}
 
+	// If head state exists but there hasn't been a justified check point,
+	// the check point root should refer to genesis block root.
+	if bytes.Equal(s.headState.CurrentJustifiedCheckpoint.Root, params.BeaconConfig().ZeroHash[:]) {
+		return &ethpb.Checkpoint{Root: s.genesisRoot[:]}
+	}
+
 	return s.headState.CurrentJustifiedCheckpoint
 }
 
@@ -78,6 +91,12 @@ func (s *Service) PreviousJustifiedCheckpt() *ethpb.Checkpoint {
 		return &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
 	}
 
+	// If head state exists but there hasn't been a justified check point,
+	// the check point root should refer to genesis block root.
+	if bytes.Equal(s.headState.PreviousJustifiedCheckpoint.Root, params.BeaconConfig().ZeroHash[:]) {
+		return &ethpb.Checkpoint{Root: s.genesisRoot[:]}
+	}
+
 	return s.headState.PreviousJustifiedCheckpoint
 }
 
@@ -103,11 +122,11 @@ func (s *Service) HeadRoot() []byte {
 }
 
 // HeadBlock returns the head block of the chain.
-func (s *Service) HeadBlock() *ethpb.BeaconBlock {
+func (s *Service) HeadBlock() *ethpb.SignedBeaconBlock {
 	s.headLock.RLock()
 	defer s.headLock.RUnlock()
 
-	return proto.Clone(s.headBlock).(*ethpb.BeaconBlock)
+	return proto.Clone(s.headBlock).(*ethpb.SignedBeaconBlock)
 }
 
 // HeadState returns the head state of the chain.
@@ -126,11 +145,18 @@ func (s *Service) HeadState(ctx context.Context) (*pb.BeaconState, error) {
 
 // HeadValidatorsIndices returns a list of active validator indices from the head view of a given epoch.
 func (s *Service) HeadValidatorsIndices(epoch uint64) ([]uint64, error) {
+	if s.headState == nil {
+		return []uint64{}, nil
+	}
 	return helpers.ActiveValidatorIndices(s.headState, epoch)
 }
 
 // HeadSeed returns the seed from the head view of a given epoch.
 func (s *Service) HeadSeed(epoch uint64) ([32]byte, error) {
+	if s.headState == nil {
+		return [32]byte{}, nil
+	}
+
 	return helpers.Seed(s.headState, epoch, params.BeaconConfig().DomainBeaconAttester)
 }
 

beacon-chain/blockchain/chain_info_norace_test.go

@@ -18,7 +18,7 @@ func TestHeadSlot_DataRace(t *testing.T) {
 	go func() {
 		s.saveHead(
 			context.Background(),
-			&ethpb.BeaconBlock{Slot: 777},
+			&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 777}},
 			[32]byte{},
 		)
 	}()
@@ -35,7 +35,7 @@ func TestHeadRoot_DataRace(t *testing.T) {
 	go func() {
 		s.saveHead(
 			context.Background(),
-			&ethpb.BeaconBlock{Slot: 777},
+			&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 777}},
 			[32]byte{},
 		)
 	}()
@@ -52,7 +52,7 @@ func TestHeadBlock_DataRace(t *testing.T) {
 	go func() {
 		s.saveHead(
 			context.Background(),
-			&ethpb.BeaconBlock{Slot: 777},
+			&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 777}},
 			[32]byte{},
 		)
 	}()
@@ -69,7 +69,7 @@ func TestHeadState_DataRace(t *testing.T) {
 	go func() {
 		s.saveHead(
 			context.Background(),
-			&ethpb.BeaconBlock{Slot: 777},
+			&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 777}},
 			[32]byte{},
 		)
 	}()

beacon-chain/blockchain/chain_info_test.go

@@ -20,7 +20,9 @@ var _ = GenesisTimeFetcher(&Service{})
 var _ = ForkFetcher(&Service{})
 
 func TestFinalizedCheckpt_Nil(t *testing.T) {
-	c := setupBeaconChain(t, nil)
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+	c := setupBeaconChain(t, db)
 	c.headState, _ = testutil.DeterministicGenesisState(t, 1)
 	if !bytes.Equal(c.FinalizedCheckpt().Root, params.BeaconConfig().ZeroHash[:]) {
 		t.Error("Incorrect pre chain start value")
@@ -28,7 +30,9 @@ func TestFinalizedCheckpt_Nil(t *testing.T) {
 }
 
 func TestHeadRoot_Nil(t *testing.T) {
-	c := setupBeaconChain(t, nil)
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+	c := setupBeaconChain(t, db)
 	if !bytes.Equal(c.HeadRoot(), params.BeaconConfig().ZeroHash[:]) {
 		t.Error("Incorrect pre chain start value")
 	}
@@ -47,6 +51,20 @@ func TestFinalizedCheckpt_CanRetrieve(t *testing.T) {
 	}
 }
 
+func TestFinalizedCheckpt_GenesisRootOk(t *testing.T) {
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	cp := &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
+	c := setupBeaconChain(t, db)
+	c.headState = &pb.BeaconState{FinalizedCheckpoint: cp}
+	c.genesisRoot = [32]byte{'A'}
+
+	if !bytes.Equal(c.FinalizedCheckpt().Root, c.genesisRoot[:]) {
+		t.Errorf("Got: %v, wanted: %v", c.FinalizedCheckpt().Root, c.genesisRoot[:])
+	}
+}
+
 func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
 	db := testDB.SetupDB(t)
 	defer testDB.TeardownDB(t, db)
@@ -60,6 +78,20 @@ func TestCurrentJustifiedCheckpt_CanRetrieve(t *testing.T) {
 	}
 }
 
+func TestJustifiedCheckpt_GenesisRootOk(t *testing.T) {
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	cp := &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
+	c := setupBeaconChain(t, db)
+	c.headState = &pb.BeaconState{CurrentJustifiedCheckpoint: cp}
+	c.genesisRoot = [32]byte{'B'}
+
+	if !bytes.Equal(c.CurrentJustifiedCheckpt().Root, c.genesisRoot[:]) {
+		t.Errorf("Got: %v, wanted: %v", c.CurrentJustifiedCheckpt().Root, c.genesisRoot[:])
+	}
+}
+
 func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
 	db := testDB.SetupDB(t)
 	defer testDB.TeardownDB(t, db)
@@ -73,6 +105,20 @@ func TestPreviousJustifiedCheckpt_CanRetrieve(t *testing.T) {
 	}
 }
 
+func TestPrevJustifiedCheckpt_GenesisRootOk(t *testing.T) {
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	cp := &ethpb.Checkpoint{Root: params.BeaconConfig().ZeroHash[:]}
+	c := setupBeaconChain(t, db)
+	c.headState = &pb.BeaconState{PreviousJustifiedCheckpoint: cp}
+	c.genesisRoot = [32]byte{'C'}
+
+	if !bytes.Equal(c.PreviousJustifiedCheckpt().Root, c.genesisRoot[:]) {
+		t.Errorf("Got: %v, wanted: %v", c.PreviousJustifiedCheckpt().Root, c.genesisRoot[:])
+	}
+}
+
 func TestHeadSlot_CanRetrieve(t *testing.T) {
 	c := &Service{}
 	c.headSlot = 100
@@ -91,7 +137,7 @@ func TestHeadRoot_CanRetrieve(t *testing.T) {
 }
 
 func TestHeadBlock_CanRetrieve(t *testing.T) {
-	b := &ethpb.BeaconBlock{Slot: 1}
+	b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 1}}
 	c := &Service{headBlock: b}
 	if !reflect.DeepEqual(b, c.HeadBlock()) {
 		t.Error("incorrect head block received")

beacon-chain/blockchain/forkchoice/BUILD.bazel

@@ -24,8 +24,8 @@ go_library(
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
         "//shared/featureconfig:go_default_library",
-        "//shared/hashutil:go_default_library",
         "//shared/params:go_default_library",
+        "//shared/stateutil:go_default_library",
         "//shared/traceutil:go_default_library",
         "@com_github_gogo_protobuf//proto:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
@@ -61,8 +61,8 @@ go_test(
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
         "//shared/featureconfig:go_default_library",
-        "//shared/hashutil:go_default_library",
         "//shared/params:go_default_library",
+        "//shared/stateutil:go_default_library",
         "//shared/testutil:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
         "@com_github_prysmaticlabs_go_bitfield//:go_default_library",

beacon-chain/blockchain/forkchoice/benchmark_test.go

@@ -18,7 +18,7 @@ func BenchmarkForkChoiceTree1(b *testing.B) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		b.Fatal(err)
 	}

beacon-chain/blockchain/forkchoice/lmd_ghost_yaml_test.go

@@ -14,6 +14,7 @@ import (
 	testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
+	"github.com/prysmaticlabs/prysm/shared/params"
 	"gopkg.in/yaml.v2"
 )
 
@@ -38,6 +39,8 @@ func TestGetHeadFromYaml(t *testing.T) {
 	var c *Config
 	err = yaml.Unmarshal(yamlFile, &c)
 
+	params.UseMainnetConfig()
+
 	for _, test := range c.TestCases {
 		db := testDB.SetupDB(t)
 		defer testDB.TeardownDB(t, db)
@@ -48,10 +51,10 @@ func TestGetHeadFromYaml(t *testing.T) {
 			// genesis block condition
 			if blk.ID == blk.Parent {
 				b := &ethpb.BeaconBlock{Slot: 0, ParentRoot: []byte{'g'}}
-				if err := db.SaveBlock(ctx, b); err != nil {
+				if err := db.SaveBlock(ctx, &ethpb.SignedBeaconBlock{Block: b}); err != nil {
 					t.Fatal(err)
 				}
-				root, err := ssz.SigningRoot(b)
+				root, err := ssz.HashTreeRoot(b)
 				if err != nil {
 					t.Fatal(err)
 				}
@@ -65,15 +68,18 @@ func TestGetHeadFromYaml(t *testing.T) {
 				if err != nil {
 					t.Fatal(err)
 				}
-				b := &ethpb.BeaconBlock{Slot: uint64(slot), ParentRoot: blksRoot[parentSlot]}
+				b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: uint64(slot), ParentRoot: blksRoot[parentSlot]}}
 				if err := db.SaveBlock(ctx, b); err != nil {
 					t.Fatal(err)
 				}
-				root, err := ssz.SigningRoot(b)
+				root, err := ssz.HashTreeRoot(b.Block)
 				if err != nil {
 					t.Fatal(err)
 				}
 				blksRoot[slot] = root[:]
+				if err := db.SaveState(ctx, &pb.BeaconState{}, root); err != nil {
+					t.Fatal(err)
+				}
 			}
 		}
 
@@ -100,12 +106,10 @@ func TestGetHeadFromYaml(t *testing.T) {
 
 		s := &pb.BeaconState{Validators: validators}
 
-		if err := store.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+		if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(blksRoot[0])); err != nil {
 			t.Fatal(err)
 		}
-
+		if err := store.GenesisStore(ctx, &ethpb.Checkpoint{Root: blksRoot[0]}, &ethpb.Checkpoint{Root: blksRoot[0]}); err != nil {
-		store.justifiedCheckpt.Root = blksRoot[0]
-		if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(blksRoot[0])); err != nil {
 			t.Fatal(err)
 		}
 

beacon-chain/blockchain/forkchoice/process_attestation.go

@@ -3,6 +3,7 @@ package forkchoice
 import (
 	"context"
 	"fmt"
+	"time"
 
 	"github.com/gogo/protobuf/proto"
 	"github.com/pkg/errors"
@@ -14,7 +15,7 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
-	"github.com/prysmaticlabs/prysm/shared/hashutil"
+	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/sirupsen/logrus"
 	"go.opencensus.io/trace"
 )
@@ -28,32 +29,48 @@ var ErrTargetRootNotInDB = errors.New("target root does not exist in db")
 //
 // Spec pseudocode definition:
 //   def on_attestation(store: Store, attestation: Attestation) -> None:
+//    """
+//    Run ``on_attestation`` upon receiving a new ``attestation`` from either within a block or directly on the wire.
+//
+//    An ``attestation`` that is asserted as invalid may be valid at a later time,
+//    consider scheduling it for later processing in such case.
+//    """
 //    target = attestation.data.target
 //
-//    # Cannot calculate the current shuffling if have not seen the target
+//    # Attestations must be from the current or previous epoch
-//    assert target.root in store.blocks
+//    current_epoch = compute_epoch_at_slot(get_current_slot(store))
+//    # Use GENESIS_EPOCH for previous when genesis to avoid underflow
+//    previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH
+//    assert target.epoch in [current_epoch, previous_epoch]
+//    assert target.epoch == compute_epoch_at_slot(attestation.data.slot)
 //
+//    # Attestations target be for a known block. If target block is unknown, delay consideration until the block is found
+//    assert target.root in store.blocks
 //    # Attestations cannot be from future epochs. If they are, delay consideration until the epoch arrives
 //    base_state = store.block_states[target.root].copy()
-//    assert store.time >= base_state.genesis_time + compute_start_slot_of_epoch(target.epoch) * SECONDS_PER_SLOT
+//    assert store.time >= base_state.genesis_time + compute_start_slot_at_epoch(target.epoch) * SECONDS_PER_SLOT
+//
+//    # Attestations must be for a known block. If block is unknown, delay consideration until the block is found
+//    assert attestation.data.beacon_block_root in store.blocks
+//    # Attestations must not be for blocks in the future. If not, the attestation should not be considered
+//    assert store.blocks[attestation.data.beacon_block_root].slot <= attestation.data.slot
 //
 //    # Store target checkpoint state if not yet seen
 //    if target not in store.checkpoint_states:
-//        process_slots(base_state, compute_start_slot_of_epoch(target.epoch))
+//        process_slots(base_state, compute_start_slot_at_epoch(target.epoch))
 //        store.checkpoint_states[target] = base_state
 //    target_state = store.checkpoint_states[target]
 //
 //    # Attestations can only affect the fork choice of subsequent slots.
 //    # Delay consideration in the fork choice until their slot is in the past.
-//    attestation_slot = get_attestation_data_slot(target_state, attestation.data)
+//    assert store.time >= (attestation.data.slot + 1) * SECONDS_PER_SLOT
-//    assert store.time >= (attestation_slot + 1) * SECONDS_PER_SLOT
 //
 //    # Get state at the `target` to validate attestation and calculate the committees
 //    indexed_attestation = get_indexed_attestation(target_state, attestation)
 //    assert is_valid_indexed_attestation(target_state, indexed_attestation)
 //
 //    # Update latest messages
-//    for i in indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices:
+//    for i in indexed_attestation.attesting_indices:
 //        if i not in store.latest_messages or target.epoch > store.latest_messages[i].epoch:
 //            store.latest_messages[i] = LatestMessage(epoch=target.epoch, root=attestation.data.beacon_block_root)
 func (s *Store) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
@@ -63,6 +80,10 @@ func (s *Store) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
 	tgt := proto.Clone(a.Data.Target).(*ethpb.Checkpoint)
 	tgtSlot := helpers.StartSlot(tgt.Epoch)
 
+	if helpers.SlotToEpoch(a.Data.Slot) != a.Data.Target.Epoch {
+		return fmt.Errorf("data slot is not in the same epoch as target %d != %d", helpers.SlotToEpoch(a.Data.Slot), a.Data.Target.Epoch)
+	}
+
 	// Verify beacon node has seen the target block before.
 	if !s.db.HasBlock(ctx, bytesutil.ToBytes32(tgt.Root)) {
 		return ErrTargetRootNotInDB
@@ -74,11 +95,21 @@ func (s *Store) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
 		return err
 	}
 
+	// Verify attestation target is from current epoch or previous epoch.
+	if err := s.verifyAttTargetEpoch(ctx, baseState.GenesisTime, uint64(time.Now().Unix()), tgt); err != nil {
+		return err
+	}
+
 	// Verify Attestations cannot be from future epochs.
 	if err := helpers.VerifySlotTime(baseState.GenesisTime, tgtSlot); err != nil {
 		return errors.Wrap(err, "could not verify attestation target slot")
 	}
 
+	// Verify attestation beacon block is known and not from the future.
+	if err := s.verifyBeaconBlock(ctx, a.Data); err != nil {
+		return errors.Wrap(err, "could not verify attestation beacon block")
+	}
+
 	// Store target checkpoint state if not yet seen.
 	baseState, err = s.saveCheckpointState(ctx, baseState, tgt)
 	if err != nil {
@@ -101,11 +132,6 @@ func (s *Store) OnAttestation(ctx context.Context, a *ethpb.Attestation) error {
 		return err
 	}
 
-	// Mark attestation as seen we don't update votes when it appears in block.
-	if err := s.setSeenAtt(a); err != nil {
-		return err
-	}
-
 	if err := s.db.SaveAttestation(ctx, a); err != nil {
 		return err
 	}
@@ -133,6 +159,36 @@ func (s *Store) verifyAttPreState(ctx context.Context, c *ethpb.Checkpoint) (*pb
 	return baseState, nil
 }
 
+// verifyAttTargetEpoch validates attestation is from the current or previous epoch.
+func (s *Store) verifyAttTargetEpoch(ctx context.Context, genesisTime uint64, nowTime uint64, c *ethpb.Checkpoint) error {
+	currentSlot := (nowTime - genesisTime) / params.BeaconConfig().SecondsPerSlot
+	currentEpoch := helpers.SlotToEpoch(currentSlot)
+	var prevEpoch uint64
+	// Prevents previous epoch under flow
+	if currentEpoch > 1 {
+		prevEpoch = currentEpoch - 1
+	}
+	if c.Epoch != prevEpoch && c.Epoch != currentEpoch {
+		return fmt.Errorf("target epoch %d does not match current epoch %d or prev epoch %d", c.Epoch, currentEpoch, prevEpoch)
+	}
+	return nil
+}
+
+// verifyBeaconBlock verifies beacon head block is known and not from the future.
+func (s *Store) verifyBeaconBlock(ctx context.Context, data *ethpb.AttestationData) error {
+	b, err := s.db.Block(ctx, bytesutil.ToBytes32(data.BeaconBlockRoot))
+	if err != nil {
+		return err
+	}
+	if b == nil || b.Block == nil {
+		return fmt.Errorf("beacon block %#x does not exist", bytesutil.Trunc(data.BeaconBlockRoot))
+	}
+	if b.Block.Slot > data.Slot {
+		return fmt.Errorf("could not process attestation for future block, %d > %d", b.Block.Slot, data.Slot)
+	}
+	return nil
+}
+
 // saveCheckpointState saves and returns the processed state with the associated check point.
 func (s *Store) saveCheckpointState(ctx context.Context, baseState *pb.BeaconState, c *ethpb.Checkpoint) (*pb.BeaconState, error) {
 	ctx, span := trace.StartSpan(ctx, "forkchoice.saveCheckpointState")
@@ -214,7 +270,7 @@ func (s *Store) updateAttVotes(
 	tgtRoot []byte,
 	tgtEpoch uint64) error {
 
-	indices := append(indexedAtt.CustodyBit_0Indices, indexedAtt.CustodyBit_1Indices...)
+	indices := indexedAtt.AttestingIndices
 	s.voteLock.Lock()
 	defer s.voteLock.Unlock()
 	for _, i := range indices {
@@ -229,20 +285,6 @@ func (s *Store) updateAttVotes(
 	return nil
 }
 
-// setSeenAtt sets the attestation hash in seen attestation map to true.
-func (s *Store) setSeenAtt(a *ethpb.Attestation) error {
-	s.seenAttsLock.Lock()
-	defer s.seenAttsLock.Unlock()
-
-	r, err := hashutil.HashProto(a)
-	if err != nil {
-		return err
-	}
-	s.seenAtts[r] = true
-
-	return nil
-}
-
 // aggregatedAttestation returns the aggregated attestation after checking saved one in db.
 func (s *Store) aggregatedAttestations(ctx context.Context, att *ethpb.Attestation) ([]*ethpb.Attestation, error) {
 	r, err := ssz.HashTreeRoot(att.Data)

beacon-chain/blockchain/forkchoice/process_attestation_test.go

@@ -24,31 +24,31 @@ func TestStore_OnAttestation(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	_, err := blockTree1(db)
+	_, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	BlkWithOutState := &ethpb.BeaconBlock{Slot: 0}
+	BlkWithOutState := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 0}}
 	if err := db.SaveBlock(ctx, BlkWithOutState); err != nil {
 		t.Fatal(err)
 	}
-	BlkWithOutStateRoot, _ := ssz.SigningRoot(BlkWithOutState)
+	BlkWithOutStateRoot, _ := ssz.HashTreeRoot(BlkWithOutState.Block)
 
-	BlkWithStateBadAtt := &ethpb.BeaconBlock{Slot: 1}
+	BlkWithStateBadAtt := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 1}}
 	if err := db.SaveBlock(ctx, BlkWithStateBadAtt); err != nil {
 		t.Fatal(err)
 	}
-	BlkWithStateBadAttRoot, _ := ssz.SigningRoot(BlkWithStateBadAtt)
+	BlkWithStateBadAttRoot, _ := ssz.HashTreeRoot(BlkWithStateBadAtt.Block)
 	if err := store.db.SaveState(ctx, &pb.BeaconState{}, BlkWithStateBadAttRoot); err != nil {
 		t.Fatal(err)
 	}
 
-	BlkWithValidState := &ethpb.BeaconBlock{Slot: 2}
+	BlkWithValidState := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 2}}
 	if err := db.SaveBlock(ctx, BlkWithValidState); err != nil {
 		t.Fatal(err)
 	}
-	BlkWithValidStateRoot, _ := ssz.SigningRoot(BlkWithValidState)
+	BlkWithValidStateRoot, _ := ssz.HashTreeRoot(BlkWithValidState.Block)
 	if err := store.db.SaveState(ctx, &pb.BeaconState{
 		Fork: &pb.Fork{
 			Epoch:           0,
@@ -67,6 +67,13 @@ func TestStore_OnAttestation(t *testing.T) {
 		wantErr       bool
 		wantErrString string
 	}{
+		{
+			name:          "attestation's data slot not aligned with target vote",
+			a:             &ethpb.Attestation{Data: &ethpb.AttestationData{Slot: params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{}}},
+			s:             &pb.BeaconState{},
+			wantErr:       true,
+			wantErrString: "data slot is not in the same epoch as target 1 != 0",
+		},
 		{
 			name:          "attestation's target root not in db",
 			a:             &ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Root: []byte{'A'}}}},
@@ -82,18 +89,21 @@ func TestStore_OnAttestation(t *testing.T) {
 			wantErrString: "pre state of target block 0 does not exist",
 		},
 		{
-			name: "process attestation from future epoch",
+			name: "process attestation doesn't match current epoch",
-			a: &ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Epoch: params.BeaconConfig().FarFutureEpoch,
+			a: &ethpb.Attestation{Data: &ethpb.AttestationData{Slot: 100 * params.BeaconConfig().SlotsPerEpoch, Target: &ethpb.Checkpoint{Epoch: 100,
 				Root: BlkWithStateBadAttRoot[:]}}},
 			s:             &pb.BeaconState{},
 			wantErr:       true,
-			wantErrString: "could not process slot from the future",
+			wantErrString: "does not match current epoch",
 		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			if err := store.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+			if err := store.GenesisStore(
+				ctx,
+				&ethpb.Checkpoint{Root: BlkWithValidStateRoot[:]},
+				&ethpb.Checkpoint{Root: BlkWithValidStateRoot[:]}); err != nil {
 				t.Fatal(err)
 			}
 
@@ -131,7 +141,11 @@ func TestStore_SaveCheckpointState(t *testing.T) {
 		Slashings:           make([]uint64, params.BeaconConfig().EpochsPerSlashingsVector),
 		FinalizedCheckpoint: &ethpb.Checkpoint{},
 	}
-	if err := store.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+	r := [32]byte{'g'}
+	if err := store.db.SaveState(ctx, s, r); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.GenesisStore(ctx, &ethpb.Checkpoint{Root: r[:]}, &ethpb.Checkpoint{Root: r[:]}); err != nil {
 		t.Fatal(err)
 	}
 
@@ -178,7 +192,7 @@ func TestStore_SaveCheckpointState(t *testing.T) {
 	}
 
 	s.Slot = params.BeaconConfig().SlotsPerEpoch + 1
-	if err := store.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+	if err := store.GenesisStore(ctx, &ethpb.Checkpoint{Root: r[:]}, &ethpb.Checkpoint{Root: r[:]}); err != nil {
 		t.Fatal(err)
 	}
 	cp3 := &ethpb.Checkpoint{Epoch: 1, Root: []byte{'C'}}
@@ -263,3 +277,93 @@ func TestStore_UpdateCheckpointState(t *testing.T) {
 		t.Error("Incorrectly cached base state")
 	}
 }
+
+func TestAttEpoch_MatchPrevEpoch(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+	if err := store.verifyAttTargetEpoch(
+		ctx,
+		0,
+		params.BeaconConfig().SlotsPerEpoch*params.BeaconConfig().SecondsPerSlot,
+		&ethpb.Checkpoint{}); err != nil {
+		t.Error(err)
+	}
+}
+
+func TestAttEpoch_MatchCurrentEpoch(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+	if err := store.verifyAttTargetEpoch(
+		ctx,
+		0,
+		params.BeaconConfig().SlotsPerEpoch*params.BeaconConfig().SecondsPerSlot,
+		&ethpb.Checkpoint{Epoch: 1}); err != nil {
+		t.Error(err)
+	}
+}
+
+func TestAttEpoch_NotMatch(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+	if err := store.verifyAttTargetEpoch(
+		ctx,
+		0,
+		2*params.BeaconConfig().SlotsPerEpoch*params.BeaconConfig().SecondsPerSlot,
+		&ethpb.Checkpoint{}); !strings.Contains(err.Error(),
+		"target epoch 0 does not match current epoch 2 or prev epoch 1") {
+		t.Error("Did not receive wanted error")
+	}
+}
+
+func TestVerifyBeaconBlock_NoBlock(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	s := NewForkChoiceService(ctx, db)
+	d := &ethpb.AttestationData{}
+	if err := s.verifyBeaconBlock(ctx, d); !strings.Contains(err.Error(), "beacon block  does not exist") {
+		t.Error("Did not receive the wanted error")
+	}
+}
+
+func TestVerifyBeaconBlock_futureBlock(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	s := NewForkChoiceService(ctx, db)
+	b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 2}}
+	s.db.SaveBlock(ctx, b)
+	r, _ := ssz.HashTreeRoot(b.Block)
+	d := &ethpb.AttestationData{Slot: 1, BeaconBlockRoot: r[:]}
+
+	if err := s.verifyBeaconBlock(ctx, d); !strings.Contains(err.Error(), "could not process attestation for future block") {
+		t.Error("Did not receive the wanted error")
+	}
+}
+
+func TestVerifyBeaconBlock_OK(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	s := NewForkChoiceService(ctx, db)
+	b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 2}}
+	s.db.SaveBlock(ctx, b)
+	r, _ := ssz.HashTreeRoot(b.Block)
+	d := &ethpb.AttestationData{Slot: 2, BeaconBlockRoot: r[:]}
+
+	if err := s.verifyBeaconBlock(ctx, d); err != nil {
+		t.Error("Did not receive the wanted error")
+	}
+}

beacon-chain/blockchain/forkchoice/process_block.go

@@ -5,12 +5,12 @@ import (
 	"context"
 	"encoding/hex"
 	"fmt"
+	"time"
 
 	"github.com/gogo/protobuf/proto"
 	"github.com/pkg/errors"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-ssz"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
@@ -18,7 +18,6 @@ import (
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-	"github.com/prysmaticlabs/prysm/shared/hashutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/traceutil"
 	"github.com/sirupsen/logrus"
@@ -51,15 +50,22 @@ import (
 //
 //    # Update justified checkpoint
 //    if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
-//        store.justified_checkpoint = state.current_justified_checkpoint
+//        if state.current_justified_checkpoint.epoch > store.best_justified_checkpoint.epoch:
+//            store.best_justified_checkpoint = state.current_justified_checkpoint
 //
 //    # Update finalized checkpoint
 //    if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch:
 //        store.finalized_checkpoint = state.finalized_checkpoint
-func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
+func (s *Store) OnBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "forkchoice.onBlock")
 	defer span.End()
 
+	if signed == nil || signed.Block == nil {
+		return errors.New("nil block")
+	}
+
+	b := signed.Block
+
 	// Retrieve incoming block's pre state.
 	preState, err := s.getBlockPreState(ctx, b)
 	if err != nil {
@@ -67,7 +73,7 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 	}
 	preStateValidatorCount := len(preState.Validators)
 
-	root, err := ssz.SigningRoot(b)
+	root, err := ssz.HashTreeRoot(b)
 	if err != nil {
 		return errors.Wrapf(err, "could not get signing root of block %d", b.Slot)
 	}
@@ -75,15 +81,12 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 		"slot": b.Slot,
 		"root": fmt.Sprintf("0x%s...", hex.EncodeToString(root[:])[:8]),
 	}).Info("Executing state transition on block")
-	postState, err := state.ExecuteStateTransition(ctx, preState, b)
+	postState, err := state.ExecuteStateTransition(ctx, preState, signed)
 	if err != nil {
 		return errors.Wrap(err, "could not execute state transition")
 	}
-	if err := s.updateBlockAttestationsVotes(ctx, b.Body.Attestations); err != nil {
-		return errors.Wrap(err, "could not update votes for attestations in block")
-	}
 
-	if err := s.db.SaveBlock(ctx, b); err != nil {
+	if err := s.db.SaveBlock(ctx, signed); err != nil {
 		return errors.Wrapf(err, "could not save block from slot %d", b.Slot)
 	}
 	if err := s.db.SaveState(ctx, postState, root); err != nil {
@@ -91,25 +94,20 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 	}
 
 	// Update justified check point.
-	if postState.CurrentJustifiedCheckpoint.Epoch > s.JustifiedCheckpt().Epoch {
+	if postState.CurrentJustifiedCheckpoint.Epoch > s.justifiedCheckpt.Epoch {
-		s.justifiedCheckpt = postState.CurrentJustifiedCheckpoint
+		if err := s.updateJustified(ctx, postState); err != nil {
-		if err := s.db.SaveJustifiedCheckpoint(ctx, postState.CurrentJustifiedCheckpoint); err != nil {
+			return err
-			return errors.Wrap(err, "could not save justified checkpoint")
 		}
 	}
 
 	// Update finalized check point.
 	// Prune the block cache and helper caches on every new finalized epoch.
 	if postState.FinalizedCheckpoint.Epoch > s.finalizedCheckpt.Epoch {
-		s.clearSeenAtts()
 		if err := s.db.SaveFinalizedCheckpoint(ctx, postState.FinalizedCheckpoint); err != nil {
 			return errors.Wrap(err, "could not save finalized checkpoint")
 		}
 
-		startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch) + 1
+		startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
-		if featureconfig.Get().PruneEpochBoundaryStates {
-			startSlot = helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
-		}
 		endSlot := helpers.StartSlot(s.finalizedCheckpt.Epoch)
 		if endSlot > startSlot {
 			if err := s.rmStatesOlderThanLastFinalized(ctx, startSlot, endSlot); err != nil {
@@ -132,7 +130,7 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 	}
 
 	// Epoch boundary bookkeeping such as logging epoch summaries.
-	if helpers.IsEpochStart(postState.Slot) {
+	if postState.Slot >= s.nextEpochBoundarySlot {
 		logEpochData(postState)
 		reportEpochMetrics(postState)
 
@@ -142,6 +140,8 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 				return err
 			}
 		}
+
+		s.nextEpochBoundarySlot = helpers.StartSlot(helpers.NextEpoch(postState))
 	}
 
 	return nil
@@ -151,10 +151,16 @@ func (s *Store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
 // It runs state transition on the block and without any BLS verification. The BLS verification
 // includes proposer signature, randao and attestation's aggregated signature. It also does not save
 // attestations.
-func (s *Store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.BeaconBlock) error {
+func (s *Store) OnBlockInitialSyncStateTransition(ctx context.Context, signed *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "forkchoice.onBlock")
 	defer span.End()
 
+	if signed == nil || signed.Block == nil {
+		return errors.New("nil block")
+	}
+
+	b := signed.Block
+
 	s.initSyncStateLock.Lock()
 	defer s.initSyncStateLock.Unlock()
 
@@ -167,15 +173,15 @@ func (s *Store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.
 
 	log.WithField("slot", b.Slot).Debug("Executing state transition on block")
 
-	postState, err := state.ExecuteStateTransitionNoVerify(ctx, preState, b)
+	postState, err := state.ExecuteStateTransitionNoVerify(ctx, preState, signed)
 	if err != nil {
 		return errors.Wrap(err, "could not execute state transition")
 	}
 
-	if err := s.db.SaveBlock(ctx, b); err != nil {
+	if err := s.db.SaveBlock(ctx, signed); err != nil {
 		return errors.Wrapf(err, "could not save block from slot %d", b.Slot)
 	}
-	root, err := ssz.SigningRoot(b)
+	root, err := ssz.HashTreeRoot(b)
 	if err != nil {
 		return errors.Wrapf(err, "could not get signing root of block %d", b.Slot)
 	}
@@ -189,22 +195,16 @@ func (s *Store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.
 	}
 
 	// Update justified check point.
-	if postState.CurrentJustifiedCheckpoint.Epoch > s.JustifiedCheckpt().Epoch {
+	if postState.CurrentJustifiedCheckpoint.Epoch > s.justifiedCheckpt.Epoch {
-		s.justifiedCheckpt = postState.CurrentJustifiedCheckpoint
+		if err := s.updateJustified(ctx, postState); err != nil {
-		if err := s.db.SaveJustifiedCheckpoint(ctx, postState.CurrentJustifiedCheckpoint); err != nil {
+			return err
-			return errors.Wrap(err, "could not save justified checkpoint")
 		}
 	}
 
 	// Update finalized check point.
 	// Prune the block cache and helper caches on every new finalized epoch.
 	if postState.FinalizedCheckpoint.Epoch > s.finalizedCheckpt.Epoch {
-		s.clearSeenAtts()
+		startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
-
-		startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch) + 1
-		if featureconfig.Get().PruneEpochBoundaryStates {
-			startSlot = helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
-		}
 		endSlot := helpers.StartSlot(s.finalizedCheckpt.Epoch)
 		if endSlot > startSlot {
 			if err := s.rmStatesOlderThanLastFinalized(ctx, startSlot, endSlot); err != nil {
@@ -238,8 +238,10 @@ func (s *Store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.
 	}
 
 	// Epoch boundary bookkeeping such as logging epoch summaries.
-	if helpers.IsEpochStart(postState.Slot) {
+	if postState.Slot >= s.nextEpochBoundarySlot {
 		reportEpochMetrics(postState)
+
+		s.nextEpochBoundarySlot = helpers.StartSlot(helpers.NextEpoch(postState))
 	}
 
 	return nil
@@ -276,61 +278,6 @@ func (s *Store) getBlockPreState(ctx context.Context, b *ethpb.BeaconBlock) (*pb
 	return preState, nil
 }
 
-// updateBlockAttestationsVotes checks the attestations in block and filter out the seen ones,
-// the unseen ones get passed to updateBlockAttestationVote for updating fork choice votes.
-func (s *Store) updateBlockAttestationsVotes(ctx context.Context, atts []*ethpb.Attestation) error {
-	s.seenAttsLock.Lock()
-	defer s.seenAttsLock.Unlock()
-
-	for _, att := range atts {
-		// If we have not seen the attestation yet
-		r, err := hashutil.HashProto(att)
-		if err != nil {
-			return err
-		}
-		if s.seenAtts[r] {
-			continue
-		}
-		if err := s.updateBlockAttestationVote(ctx, att); err != nil {
-			log.WithError(err).Warn("Attestation failed to update vote")
-		}
-		s.seenAtts[r] = true
-	}
-	return nil
-}
-
-// updateBlockAttestationVotes checks the attestation to update validator's latest votes.
-func (s *Store) updateBlockAttestationVote(ctx context.Context, att *ethpb.Attestation) error {
-	tgt := att.Data.Target
-	baseState, err := s.db.State(ctx, bytesutil.ToBytes32(tgt.Root))
-	if err != nil {
-		return errors.Wrap(err, "could not get state for attestation tgt root")
-	}
-	if baseState == nil {
-		return errors.New("no state found in db with attestation tgt root")
-	}
-	committee, err := helpers.BeaconCommitteeFromState(baseState, att.Data.Slot, att.Data.CommitteeIndex)
-	if err != nil {
-		return err
-	}
-	indexedAtt, err := blocks.ConvertToIndexed(ctx, att, committee)
-	if err != nil {
-		return errors.Wrap(err, "could not convert attestation to indexed attestation")
-	}
-	s.voteLock.Lock()
-	defer s.voteLock.Unlock()
-	for _, i := range append(indexedAtt.CustodyBit_0Indices, indexedAtt.CustodyBit_1Indices...) {
-		vote, ok := s.latestVoteMap[i]
-		if !ok || tgt.Epoch > vote.Epoch {
-			s.latestVoteMap[i] = &pb.ValidatorLatestVote{
-				Epoch: tgt.Epoch,
-				Root:  tgt.Root,
-			}
-		}
-	}
-	return nil
-}
-
 // verifyBlkPreState validates input block has a valid pre-state.
 func (s *Store) verifyBlkPreState(ctx context.Context, b *ethpb.BeaconBlock) (*pb.BeaconState, error) {
 	preState, err := s.db.State(ctx, bytesutil.ToBytes32(b.ParentRoot))
@@ -349,10 +296,11 @@ func (s *Store) verifyBlkDescendant(ctx context.Context, root [32]byte, slot uin
 	ctx, span := trace.StartSpan(ctx, "forkchoice.verifyBlkDescendant")
 	defer span.End()
 
-	finalizedBlk, err := s.db.Block(ctx, bytesutil.ToBytes32(s.finalizedCheckpt.Root))
+	finalizedBlkSigned, err := s.db.Block(ctx, bytesutil.ToBytes32(s.finalizedCheckpt.Root))
-	if err != nil || finalizedBlk == nil {
+	if err != nil || finalizedBlkSigned == nil || finalizedBlkSigned.Block == nil {
 		return errors.Wrap(err, "could not get finalized block")
 	}
+	finalizedBlk := finalizedBlkSigned.Block
 
 	bFinalizedRoot, err := s.ancestor(ctx, root[:], finalizedBlk.Slot)
 	if err != nil {
@@ -413,30 +361,21 @@ func (s *Store) saveNewBlockAttestations(ctx context.Context, atts []*ethpb.Atte
 	return nil
 }
 
-// clearSeenAtts clears seen attestations map, it gets called upon new finalization.
-func (s *Store) clearSeenAtts() {
-	s.seenAttsLock.Lock()
-	s.seenAttsLock.Unlock()
-	s.seenAtts = make(map[[32]byte]bool)
-}
-
 // rmStatesOlderThanLastFinalized deletes the states in db since last finalized check point.
 func (s *Store) rmStatesOlderThanLastFinalized(ctx context.Context, startSlot uint64, endSlot uint64) error {
 	ctx, span := trace.StartSpan(ctx, "forkchoice.rmStatesBySlots")
 	defer span.End()
 
 	// Make sure start slot is not a skipped slot
-	if featureconfig.Get().PruneEpochBoundaryStates {
+	for i := startSlot; i > 0; i-- {
-		for i := startSlot; i > 0; i-- {
+		filter := filters.NewFilter().SetStartSlot(i).SetEndSlot(i)
-			filter := filters.NewFilter().SetStartSlot(i).SetEndSlot(i)
+		b, err := s.db.Blocks(ctx, filter)
-			b, err := s.db.Blocks(ctx, filter)
+		if err != nil {
-			if err != nil {
+			return err
-				return err
+		}
-			}
+		if len(b) > 0 {
-			if len(b) > 0 {
+			startSlot = i
-				startSlot = i
+			break
-				break
-			}
 		}
 	}
 
@@ -480,6 +419,82 @@ func (s *Store) rmStatesOlderThanLastFinalized(ctx context.Context, startSlot ui
 	return nil
 }
 
+// shouldUpdateCurrentJustified prevents bouncing attack, by only update conflicting justified
+// checkpoints in the fork choice if in the early slots of the epoch.
+// Otherwise, delay incorporation of new justified checkpoint until next epoch boundary.
+// See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion.
+func (s *Store) shouldUpdateCurrentJustified(ctx context.Context, newJustifiedCheckpt *ethpb.Checkpoint) (bool, error) {
+	if helpers.SlotsSinceEpochStarts(s.currentSlot()) < params.BeaconConfig().SafeSlotsToUpdateJustified {
+		return true, nil
+	}
+	newJustifiedBlockSigned, err := s.db.Block(ctx, bytesutil.ToBytes32(newJustifiedCheckpt.Root))
+	if err != nil {
+		return false, err
+	}
+	if newJustifiedBlockSigned == nil || newJustifiedBlockSigned.Block == nil {
+		return false, errors.New("nil new justified block")
+	}
+	newJustifiedBlock := newJustifiedBlockSigned.Block
+	if newJustifiedBlock.Slot <= helpers.StartSlot(s.justifiedCheckpt.Epoch) {
+		return false, nil
+	}
+	justifiedBlockSigned, err := s.db.Block(ctx, bytesutil.ToBytes32(s.justifiedCheckpt.Root))
+	if err != nil {
+		return false, err
+	}
+	if justifiedBlockSigned == nil || justifiedBlockSigned.Block == nil {
+		return false, errors.New("nil justified block")
+	}
+	justifiedBlock := justifiedBlockSigned.Block
+	b, err := s.ancestor(ctx, newJustifiedCheckpt.Root, justifiedBlock.Slot)
+	if err != nil {
+		return false, err
+	}
+	if !bytes.Equal(b, s.justifiedCheckpt.Root) {
+		return false, nil
+	}
+	return true, nil
+}
+
+func (s *Store) updateJustified(ctx context.Context, state *pb.BeaconState) error {
+	if state.CurrentJustifiedCheckpoint.Epoch > s.bestJustifiedCheckpt.Epoch {
+		s.bestJustifiedCheckpt = state.CurrentJustifiedCheckpoint
+	}
+	canUpdate, err := s.shouldUpdateCurrentJustified(ctx, state.CurrentJustifiedCheckpoint)
+	if err != nil {
+		return err
+	}
+	if canUpdate {
+		s.justifiedCheckpt = state.CurrentJustifiedCheckpoint
+	}
+
+	if featureconfig.Get().InitSyncCacheState {
+		justifiedRoot := bytesutil.ToBytes32(state.CurrentJustifiedCheckpoint.Root)
+		justifiedState := s.initSyncState[justifiedRoot]
+		if err := s.db.SaveState(ctx, justifiedState, justifiedRoot); err != nil {
+			return errors.Wrap(err, "could not save justified state")
+		}
+	}
+
+	return s.db.SaveJustifiedCheckpoint(ctx, state.CurrentJustifiedCheckpoint)
+}
+
+// currentSlot returns the current slot based on time.
+func (s *Store) currentSlot() uint64 {
+	return (uint64(time.Now().Unix()) - s.genesisTime) / params.BeaconConfig().SecondsPerSlot
+}
+
+// updates justified check point in store if a better check point is known
+func (s *Store) updateJustifiedCheckpoint() {
+	// Update at epoch boundary slot only
+	if !helpers.IsEpochStart(s.currentSlot()) {
+		return
+	}
+	if s.bestJustifiedCheckpt.Epoch > s.justifiedCheckpt.Epoch {
+		s.justifiedCheckpt = s.bestJustifiedCheckpt
+	}
+}
+
 // This receives cached state in memory for initial sync only during initial sync.
 func (s *Store) cachedPreState(ctx context.Context, b *ethpb.BeaconBlock) (*pb.BeaconState, error) {
 	if featureconfig.Get().InitSyncCacheState {

beacon-chain/blockchain/forkchoice/process_block_test.go

@@ -1,10 +1,12 @@
 package forkchoice
 
 import (
+	"bytes"
 	"context"
 	"reflect"
 	"strings"
 	"testing"
+	"time"
 
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-bitfield"
@@ -15,9 +17,8 @@ import (
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-	"github.com/prysmaticlabs/prysm/shared/hashutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
-	"github.com/prysmaticlabs/prysm/shared/testutil"
+	"github.com/prysmaticlabs/prysm/shared/stateutil"
 )
 
 func TestStore_OnBlock(t *testing.T) {
@@ -27,23 +28,40 @@ func TestStore_OnBlock(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	genesisStateRoot, err := stateutil.HashTreeRootState(&pb.BeaconState{})
+	if err != nil {
+		t.Error(err)
+	}
+	genesis := blocks.NewGenesisBlock(genesisStateRoot[:])
+	if err := db.SaveBlock(ctx, genesis); err != nil {
+		t.Error(err)
+	}
+	validGenesisRoot, err := ssz.HashTreeRoot(genesis.Block)
+	if err != nil {
+		t.Error(err)
+	}
+	if err := store.db.SaveState(ctx, &pb.BeaconState{}, validGenesisRoot); err != nil {
+		t.Fatal(err)
+	}
+	roots, err := blockTree1(db, validGenesisRoot[:])
 	if err != nil {
 		t.Fatal(err)
 	}
-
+	random := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 1, ParentRoot: validGenesisRoot[:]}}
-	randomParentRoot := []byte{'a'}
+	if err := db.SaveBlock(ctx, random); err != nil {
-	if err := store.db.SaveState(ctx, &pb.BeaconState{}, bytesutil.ToBytes32(randomParentRoot)); err != nil {
+		t.Error(err)
+	}
+	randomParentRoot, err := ssz.HashTreeRoot(random.Block)
+	if err != nil {
+		t.Error(err)
+	}
+	if err := store.db.SaveState(ctx, &pb.BeaconState{}, randomParentRoot); err != nil {
 		t.Fatal(err)
 	}
 	randomParentRoot2 := roots[1]
 	if err := store.db.SaveState(ctx, &pb.BeaconState{}, bytesutil.ToBytes32(randomParentRoot2)); err != nil {
 		t.Fatal(err)
 	}
-	validGenesisRoot := []byte{'g'}
-	if err := store.db.SaveState(ctx, &pb.BeaconState{}, bytesutil.ToBytes32(validGenesisRoot)); err != nil {
-		t.Fatal(err)
-	}
 
 	tests := []struct {
 		name          string
@@ -60,13 +78,13 @@ func TestStore_OnBlock(t *testing.T) {
 		},
 		{
 			name:          "block is from the feature",
-			blk:           &ethpb.BeaconBlock{ParentRoot: randomParentRoot, Slot: params.BeaconConfig().FarFutureEpoch},
+			blk:           &ethpb.BeaconBlock{ParentRoot: randomParentRoot[:], Slot: params.BeaconConfig().FarFutureEpoch},
 			s:             &pb.BeaconState{},
 			wantErrString: "could not process slot from the future",
 		},
 		{
 			name:          "could not get finalized block",
-			blk:           &ethpb.BeaconBlock{ParentRoot: randomParentRoot},
+			blk:           &ethpb.BeaconBlock{ParentRoot: randomParentRoot[:]},
 			s:             &pb.BeaconState{},
 			wantErrString: "block from slot 0 is not a descendent of the current finalized block",
 		},
@@ -80,12 +98,12 @@ func TestStore_OnBlock(t *testing.T) {
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			if err := store.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+			if err := store.GenesisStore(ctx, &ethpb.Checkpoint{Root: validGenesisRoot[:]}, &ethpb.Checkpoint{Root: validGenesisRoot[:]}); err != nil {
 				t.Fatal(err)
 			}
 			store.finalizedCheckpt.Root = roots[0]
 
-			err := store.OnBlock(ctx, tt.blk)
+			err := store.OnBlock(ctx, &ethpb.SignedBeaconBlock{Block: tt.blk})
 			if !strings.Contains(err.Error(), tt.wantErrString) {
 				t.Errorf("Store.OnBlock() error = %v, wantErr = %v", err, tt.wantErrString)
 			}
@@ -119,101 +137,14 @@ func TestStore_SaveNewValidators(t *testing.T) {
 	}
 }
 
-func TestStore_UpdateBlockAttestationVote(t *testing.T) {
-	ctx := context.Background()
-	db := testDB.SetupDB(t)
-	defer testDB.TeardownDB(t, db)
-	params.UseMinimalConfig()
-
-	beaconState, _ := testutil.DeterministicGenesisState(t, 100)
-
-	store := NewForkChoiceService(ctx, db)
-	r := [32]byte{'A'}
-	att := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]},
-			Target: &ethpb.Checkpoint{Epoch: 0, Root: r[:]},
-		},
-		AggregationBits: []byte{255},
-		CustodyBits:     []byte{255},
-	}
-	if err := store.db.SaveState(ctx, beaconState, r); err != nil {
-		t.Fatal(err)
-	}
-
-	committee, err := helpers.BeaconCommitteeFromState(beaconState, att.Data.Slot, att.Data.CommitteeIndex)
-	if err != nil {
-		t.Error(err)
-	}
-	indices, err := blocks.ConvertToIndexed(ctx, att, committee)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	var attestedIndices []uint64
-	for _, k := range append(indices.CustodyBit_0Indices, indices.CustodyBit_1Indices...) {
-		attestedIndices = append(attestedIndices, k)
-	}
-
-	if err := store.updateBlockAttestationVote(ctx, att); err != nil {
-		t.Fatal(err)
-	}
-	for _, i := range attestedIndices {
-		v := store.latestVoteMap[i]
-		if !reflect.DeepEqual(v.Root, r[:]) {
-			t.Error("Attested roots don't match")
-		}
-	}
-}
-
-func TestStore_UpdateBlockAttestationsVote(t *testing.T) {
-	ctx := context.Background()
-	db := testDB.SetupDB(t)
-	defer testDB.TeardownDB(t, db)
-	params.UseMinimalConfig()
-
-	beaconState, _ := testutil.DeterministicGenesisState(t, 100)
-
-	store := NewForkChoiceService(ctx, db)
-	r := [32]byte{'A'}
-	atts := make([]*ethpb.Attestation, 5)
-	hashes := make([][32]byte, 5)
-	for i := 0; i < len(atts); i++ {
-		atts[i] = &ethpb.Attestation{
-			Data: &ethpb.AttestationData{
-				Source: &ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]},
-				Target: &ethpb.Checkpoint{Epoch: 0, Root: r[:]},
-			},
-			AggregationBits: []byte{255},
-			CustodyBits:     []byte{255},
-		}
-		h, _ := hashutil.HashProto(atts[i])
-		hashes[i] = h
-	}
-
-	if err := store.db.SaveState(ctx, beaconState, r); err != nil {
-		t.Fatal(err)
-	}
-
-	if err := store.updateBlockAttestationsVotes(ctx, atts); err != nil {
-		t.Fatal(err)
-	}
-
-	for _, h := range hashes {
-		if !store.seenAtts[h] {
-			t.Error("Seen attestation did not get recorded")
-		}
-	}
-}
-
 func TestStore_SavesNewBlockAttestations(t *testing.T) {
 	ctx := context.Background()
 	db := testDB.SetupDB(t)
 	defer testDB.TeardownDB(t, db)
 
 	store := NewForkChoiceService(ctx, db)
-	a1 := &ethpb.Attestation{Data: &ethpb.AttestationData{}, AggregationBits: bitfield.Bitlist{0b101}, CustodyBits: bitfield.NewBitlist(2)}
+	a1 := &ethpb.Attestation{Data: &ethpb.AttestationData{}, AggregationBits: bitfield.Bitlist{0b101}}
-	a2 := &ethpb.Attestation{Data: &ethpb.AttestationData{BeaconBlockRoot: []byte{'A'}}, AggregationBits: bitfield.Bitlist{0b110}, CustodyBits: bitfield.NewBitlist(2)}
+	a2 := &ethpb.Attestation{Data: &ethpb.AttestationData{BeaconBlockRoot: []byte{'A'}}, AggregationBits: bitfield.Bitlist{0b110}}
 	r1, _ := ssz.HashTreeRoot(a1.Data)
 	r2, _ := ssz.HashTreeRoot(a2.Data)
 
@@ -237,8 +168,8 @@ func TestStore_SavesNewBlockAttestations(t *testing.T) {
 		t.Error("did not retrieve saved attestation")
 	}
 
-	a1 = &ethpb.Attestation{Data: &ethpb.AttestationData{}, AggregationBits: bitfield.Bitlist{0b111}, CustodyBits: bitfield.NewBitlist(2)}
+	a1 = &ethpb.Attestation{Data: &ethpb.AttestationData{}, AggregationBits: bitfield.Bitlist{0b111}}
-	a2 = &ethpb.Attestation{Data: &ethpb.AttestationData{BeaconBlockRoot: []byte{'A'}}, AggregationBits: bitfield.Bitlist{0b111}, CustodyBits: bitfield.NewBitlist(2)}
+	a2 = &ethpb.Attestation{Data: &ethpb.AttestationData{BeaconBlockRoot: []byte{'A'}}, AggregationBits: bitfield.Bitlist{0b111}}
 
 	if err := store.saveNewBlockAttestations(ctx, []*ethpb.Attestation{a1, a2}); err != nil {
 		t.Fatal(err)
@@ -272,13 +203,15 @@ func TestRemoveStateSinceLastFinalized(t *testing.T) {
 
 	// Save 100 blocks in DB, each has a state.
 	numBlocks := 100
-	totalBlocks := make([]*ethpb.BeaconBlock, numBlocks)
+	totalBlocks := make([]*ethpb.SignedBeaconBlock, numBlocks)
 	blockRoots := make([][32]byte, 0)
 	for i := 0; i < len(totalBlocks); i++ {
-		totalBlocks[i] = &ethpb.BeaconBlock{
+		totalBlocks[i] = &ethpb.SignedBeaconBlock{
-			Slot: uint64(i),
+			Block: &ethpb.BeaconBlock{
+				Slot: uint64(i),
+			},
 		}
-		r, err := ssz.SigningRoot(totalBlocks[i])
+		r, err := ssz.HashTreeRoot(totalBlocks[i].Block)
 		if err != nil {
 			t.Fatal(err)
 		}
@@ -338,45 +271,147 @@ func TestRemoveStateSinceLastFinalized_EmptyStartSlot(t *testing.T) {
 	params.UseMinimalConfig()
 	defer params.UseMainnetConfig()
 
-	c := featureconfig.Get()
-	c.PruneEpochBoundaryStates = true
-	featureconfig.Init(c)
-
 	store := NewForkChoiceService(ctx, db)
+	store.genesisTime = uint64(time.Now().Unix())
 
-	// Save 5 blocks in DB, each has a state.
+	update, err := store.shouldUpdateCurrentJustified(ctx, &ethpb.Checkpoint{})
-	numBlocks := 5
-	totalBlocks := make([]*ethpb.BeaconBlock, numBlocks)
-	blockRoots := make([][32]byte, 0)
-	for i := 0; i < len(totalBlocks); i++ {
-		totalBlocks[i] = &ethpb.BeaconBlock{
-			Slot: uint64(i),
-		}
-		r, err := ssz.SigningRoot(totalBlocks[i])
-		if err != nil {
-			t.Fatal(err)
-		}
-		if err := store.db.SaveState(ctx, &pb.BeaconState{Slot: uint64(i)}, r); err != nil {
-			t.Fatal(err)
-		}
-		if err := store.db.SaveBlock(ctx, totalBlocks[i]); err != nil {
-			t.Fatal(err)
-		}
-		blockRoots = append(blockRoots, r)
-	}
-	if err := store.db.SaveHeadBlockRoot(ctx, blockRoots[0]); err != nil {
-		t.Fatal(err)
-	}
-	if err := store.rmStatesOlderThanLastFinalized(ctx, 10, 11); err != nil {
-		t.Fatal(err)
-	}
-	// Since 5-10 are skip slots, block with slot 4 should be deleted
-	s, err := store.db.State(ctx, blockRoots[4])
 	if err != nil {
 		t.Fatal(err)
 	}
-	if s != nil {
+	if !update {
-		t.Error("Did not delete state for start slot")
+		t.Error("Should be able to update justified, received false")
+	}
+
+	lastJustifiedBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{ParentRoot: []byte{'G'}}}
+	lastJustifiedRoot, _ := ssz.HashTreeRoot(lastJustifiedBlk.Block)
+	newJustifiedBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 1, ParentRoot: lastJustifiedRoot[:]}}
+	newJustifiedRoot, _ := ssz.HashTreeRoot(newJustifiedBlk.Block)
+	if err := store.db.SaveBlock(ctx, newJustifiedBlk); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveBlock(ctx, lastJustifiedBlk); err != nil {
+		t.Fatal(err)
+	}
+
+	diff := (params.BeaconConfig().SlotsPerEpoch - 1) * params.BeaconConfig().SecondsPerSlot
+	store.genesisTime = uint64(time.Now().Unix()) - diff
+	store.justifiedCheckpt = &ethpb.Checkpoint{Root: lastJustifiedRoot[:]}
+	update, err = store.shouldUpdateCurrentJustified(ctx, &ethpb.Checkpoint{Root: newJustifiedRoot[:]})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !update {
+		t.Error("Should be able to update justified, received false")
+	}
+}
+
+func TestShouldUpdateJustified_ReturnFalse(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+	params.UseMinimalConfig()
+	defer params.UseMainnetConfig()
+
+	store := NewForkChoiceService(ctx, db)
+
+	lastJustifiedBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{ParentRoot: []byte{'G'}}}
+	lastJustifiedRoot, _ := ssz.HashTreeRoot(lastJustifiedBlk.Block)
+	newJustifiedBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{ParentRoot: lastJustifiedRoot[:]}}
+	newJustifiedRoot, _ := ssz.HashTreeRoot(newJustifiedBlk.Block)
+	if err := store.db.SaveBlock(ctx, newJustifiedBlk); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveBlock(ctx, lastJustifiedBlk); err != nil {
+		t.Fatal(err)
+	}
+
+	diff := (params.BeaconConfig().SlotsPerEpoch - 1) * params.BeaconConfig().SecondsPerSlot
+	store.genesisTime = uint64(time.Now().Unix()) - diff
+	store.justifiedCheckpt = &ethpb.Checkpoint{Root: lastJustifiedRoot[:]}
+
+	update, err := store.shouldUpdateCurrentJustified(ctx, &ethpb.Checkpoint{Root: newJustifiedRoot[:]})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if update {
+		t.Error("Should not be able to update justified, received true")
+	}
+}
+
+func TestUpdateJustifiedCheckpoint_Update(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+	params.UseMinimalConfig()
+	defer params.UseMainnetConfig()
+
+	store := NewForkChoiceService(ctx, db)
+	store.genesisTime = uint64(time.Now().Unix())
+
+	store.justifiedCheckpt = &ethpb.Checkpoint{Root: []byte{'A'}}
+	store.bestJustifiedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'B'}}
+	store.updateJustifiedCheckpoint()
+
+	if !bytes.Equal(store.justifiedCheckpt.Root, []byte{'B'}) {
+		t.Error("Justified check point root did not update")
+	}
+}
+
+func TestUpdateJustifiedCheckpoint_NoUpdate(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+	params.UseMinimalConfig()
+	defer params.UseMainnetConfig()
+
+	store := NewForkChoiceService(ctx, db)
+	store.genesisTime = uint64(time.Now().Unix()) - params.BeaconConfig().SecondsPerSlot
+
+	store.justifiedCheckpt = &ethpb.Checkpoint{Root: []byte{'A'}}
+	store.bestJustifiedCheckpt = &ethpb.Checkpoint{Epoch: 1, Root: []byte{'B'}}
+	store.updateJustifiedCheckpoint()
+
+	if bytes.Equal(store.justifiedCheckpt.Root, []byte{'B'}) {
+		t.Error("Justified check point root was not suppose to update")
+
+		store := NewForkChoiceService(ctx, db)
+
+		// Save 5 blocks in DB, each has a state.
+		numBlocks := 5
+		totalBlocks := make([]*ethpb.SignedBeaconBlock, numBlocks)
+		blockRoots := make([][32]byte, 0)
+		for i := 0; i < len(totalBlocks); i++ {
+			totalBlocks[i] = &ethpb.SignedBeaconBlock{
+				Block: &ethpb.BeaconBlock{
+					Slot: uint64(i),
+				},
+			}
+			r, err := ssz.HashTreeRoot(totalBlocks[i].Block)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if err := store.db.SaveState(ctx, &pb.BeaconState{Slot: uint64(i)}, r); err != nil {
+				t.Fatal(err)
+			}
+			if err := store.db.SaveBlock(ctx, totalBlocks[i]); err != nil {
+				t.Fatal(err)
+			}
+			blockRoots = append(blockRoots, r)
+		}
+		if err := store.db.SaveHeadBlockRoot(ctx, blockRoots[0]); err != nil {
+			t.Fatal(err)
+		}
+		if err := store.rmStatesOlderThanLastFinalized(ctx, 10, 11); err != nil {
+			t.Fatal(err)
+		}
+		// Since 5-10 are skip slots, block with slot 4 should be deleted
+		s, err := store.db.State(ctx, blockRoots[4])
+		if err != nil {
+			t.Fatal(err)
+		}
+		if s != nil {
+			t.Error("Did not delete state for start slot")
+		}
 	}
 }
 
@@ -463,12 +498,12 @@ func TestSaveInitState_CanSaveDelete(t *testing.T) {
 	for i := uint64(0); i < 64; i++ {
 		b := &ethpb.BeaconBlock{Slot: i}
 		s := &pb.BeaconState{Slot: i}
-		r, _ := ssz.SigningRoot(b)
+		r, _ := ssz.HashTreeRoot(b)
 		store.initSyncState[r] = s
 	}
 
 	// Set finalized root as slot 32
-	finalizedRoot, _ := ssz.SigningRoot(&ethpb.BeaconBlock{Slot: 32})
+	finalizedRoot, _ := ssz.HashTreeRoot(&ethpb.BeaconBlock{Slot: 32})
 
 	if err := store.saveInitState(ctx, &pb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{
 		Epoch: 1, Root: finalizedRoot[:]}}); err != nil {
@@ -490,6 +525,48 @@ func TestSaveInitState_CanSaveDelete(t *testing.T) {
 	}
 }
 
+func TestUpdateJustified_CouldUpdateBest(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+	signedBlock := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{}}
+	if err := db.SaveBlock(ctx, signedBlock); err != nil {
+		t.Fatal(err)
+	}
+	r, err := ssz.HashTreeRoot(signedBlock.Block)
+	if err != nil {
+		t.Fatal(err)
+	}
+	store.justifiedCheckpt = &ethpb.Checkpoint{Root: []byte{'A'}}
+	store.bestJustifiedCheckpt = &ethpb.Checkpoint{Root: []byte{'A'}}
+	store.initSyncState[r] = &pb.BeaconState{}
+	if err := db.SaveState(ctx, &pb.BeaconState{}, r); err != nil {
+		t.Fatal(err)
+	}
+
+	// Could update
+	s := &pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Epoch: 1, Root: r[:]}}
+	if err := store.updateJustified(context.Background(), s); err != nil {
+		t.Fatal(err)
+	}
+
+	if store.bestJustifiedCheckpt.Epoch != s.CurrentJustifiedCheckpoint.Epoch {
+		t.Error("Incorrect justified epoch in store")
+	}
+
+	// Could not update
+	store.bestJustifiedCheckpt.Epoch = 2
+	if err := store.updateJustified(context.Background(), s); err != nil {
+		t.Fatal(err)
+	}
+
+	if store.bestJustifiedCheckpt.Epoch != 2 {
+		t.Error("Incorrect justified epoch in store")
+	}
+}
+
 func TestFilterBlockRoots_CanFilter(t *testing.T) {
 	ctx := context.Background()
 	db := testDB.SetupDB(t)
@@ -497,10 +574,10 @@ func TestFilterBlockRoots_CanFilter(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 	fBlock := &ethpb.BeaconBlock{}
-	fRoot, _ := ssz.SigningRoot(fBlock)
+	fRoot, _ := ssz.HashTreeRoot(fBlock)
 	hBlock := &ethpb.BeaconBlock{Slot: 1}
-	headRoot, _ := ssz.SigningRoot(hBlock)
+	headRoot, _ := ssz.HashTreeRoot(hBlock)
-	if err := store.db.SaveBlock(ctx, fBlock); err != nil {
+	if err := store.db.SaveBlock(ctx, &ethpb.SignedBeaconBlock{Block: fBlock}); err != nil {
 		t.Fatal(err)
 	}
 	if err := store.db.SaveState(ctx, &pb.BeaconState{}, fRoot); err != nil {
@@ -509,7 +586,7 @@ func TestFilterBlockRoots_CanFilter(t *testing.T) {
 	if err := store.db.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Root: fRoot[:]}); err != nil {
 		t.Fatal(err)
 	}
-	if err := store.db.SaveBlock(ctx, hBlock); err != nil {
+	if err := store.db.SaveBlock(ctx, &ethpb.SignedBeaconBlock{Block: hBlock}); err != nil {
 		t.Fatal(err)
 	}
 	if err := store.db.SaveState(ctx, &pb.BeaconState{}, headRoot); err != nil {

beacon-chain/blockchain/forkchoice/service.go

@@ -3,6 +3,8 @@ package forkchoice
 import (
 	"bytes"
 	"context"
+	"encoding/hex"
+	"fmt"
 	"sync"
 
 	"github.com/gogo/protobuf/proto"
@@ -17,7 +19,7 @@ import (
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-	"github.com/prysmaticlabs/prysm/shared/params"
+	"github.com/prysmaticlabs/prysm/shared/stateutil"
 	"go.opencensus.io/trace"
 )
 
@@ -25,8 +27,8 @@ import (
 // to beacon blocks to compute head.
 type ForkChoicer interface {
 	Head(ctx context.Context) ([]byte, error)
-	OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error
+	OnBlock(ctx context.Context, b *ethpb.SignedBeaconBlock) error
-	OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.BeaconBlock) error
+	OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.SignedBeaconBlock) error
 	OnAttestation(ctx context.Context, a *ethpb.Attestation) error
 	GenesisStore(ctx context.Context, justifiedCheckpoint *ethpb.Checkpoint, finalizedCheckpoint *ethpb.Checkpoint) error
 	FinalizedCheckpt() *ethpb.Checkpoint
@@ -35,20 +37,21 @@ type ForkChoicer interface {
 // Store represents a service struct that handles the forkchoice
 // logic of managing the full PoS beacon chain.
 type Store struct {
-	ctx                  context.Context
+	ctx                   context.Context
-	cancel               context.CancelFunc
+	cancel                context.CancelFunc
-	db                   db.Database
+	db                    db.Database
-	justifiedCheckpt     *ethpb.Checkpoint
+	justifiedCheckpt      *ethpb.Checkpoint
-	finalizedCheckpt     *ethpb.Checkpoint
+	finalizedCheckpt      *ethpb.Checkpoint
-	prevFinalizedCheckpt *ethpb.Checkpoint
+	prevFinalizedCheckpt  *ethpb.Checkpoint
-	checkpointState      *cache.CheckpointStateCache
+	checkpointState       *cache.CheckpointStateCache
-	checkpointStateLock  sync.Mutex
+	checkpointStateLock   sync.Mutex
-	seenAtts             map[[32]byte]bool
+	genesisTime           uint64
-	seenAttsLock         sync.Mutex
+	bestJustifiedCheckpt  *ethpb.Checkpoint
-	latestVoteMap        map[uint64]*pb.ValidatorLatestVote
+	latestVoteMap         map[uint64]*pb.ValidatorLatestVote
-	voteLock             sync.RWMutex
+	voteLock              sync.RWMutex
-	initSyncState        map[[32]byte]*pb.BeaconState
+	initSyncState         map[[32]byte]*pb.BeaconState
-	initSyncStateLock    sync.RWMutex
+	initSyncStateLock     sync.RWMutex
+	nextEpochBoundarySlot uint64
 }
 
 // NewForkChoiceService instantiates a new service instance that will
@@ -61,7 +64,6 @@ func NewForkChoiceService(ctx context.Context, db db.Database) *Store {
 		db:              db,
 		checkpointState: cache.NewCheckpointStateCache(),
 		latestVoteMap:   make(map[uint64]*pb.ValidatorLatestVote),
-		seenAtts:        make(map[[32]byte]bool),
 		initSyncState:   make(map[[32]byte]*pb.BeaconState),
 	}
 }
@@ -89,6 +91,7 @@ func (s *Store) GenesisStore(
 	finalizedCheckpoint *ethpb.Checkpoint) error {
 
 	s.justifiedCheckpt = proto.Clone(justifiedCheckpoint).(*ethpb.Checkpoint)
+	s.bestJustifiedCheckpt = proto.Clone(justifiedCheckpoint).(*ethpb.Checkpoint)
 	s.finalizedCheckpt = proto.Clone(finalizedCheckpoint).(*ethpb.Checkpoint)
 	s.prevFinalizedCheckpt = proto.Clone(finalizedCheckpoint).(*ethpb.Checkpoint)
 
@@ -104,6 +107,7 @@ func (s *Store) GenesisStore(
 		return errors.Wrap(err, "could not save genesis state in check point cache")
 	}
 
+	s.genesisTime = justifiedState.GenesisTime
 	if err := s.cacheGenesisState(ctx); err != nil {
 		return errors.Wrap(err, "could not cache initial sync state")
 	}
@@ -121,12 +125,12 @@ func (s *Store) cacheGenesisState(ctx context.Context) error {
 	if err != nil {
 		return err
 	}
-	stateRoot, err := ssz.HashTreeRoot(genesisState)
+	stateRoot, err := stateutil.HashTreeRootState(genesisState)
 	if err != nil {
 		return errors.Wrap(err, "could not tree hash genesis state")
 	}
 	genesisBlk := blocks.NewGenesisBlock(stateRoot[:])
-	genesisBlkRoot, err := ssz.SigningRoot(genesisBlk)
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get genesis block root")
 	}
@@ -150,10 +154,19 @@ func (s *Store) ancestor(ctx context.Context, root []byte, slot uint64) ([]byte,
 	ctx, span := trace.StartSpan(ctx, "forkchoice.ancestor")
 	defer span.End()
 
-	b, err := s.db.Block(ctx, bytesutil.ToBytes32(root))
+	// Stop recursive ancestry lookup if context is cancelled.
+	if ctx.Err() != nil {
+		return nil, ctx.Err()
+	}
+
+	signed, err := s.db.Block(ctx, bytesutil.ToBytes32(root))
 	if err != nil {
 		return nil, errors.Wrap(err, "could not get ancestor block")
 	}
+	if signed == nil || signed.Block == nil {
+		return nil, errors.New("nil block")
+	}
+	b := signed.Block
 
 	// If we dont have the ancestor in the DB, simply return nil so rest of fork choice
 	// operation can proceed. This is not an error condition.
@@ -197,10 +210,14 @@ func (s *Store) latestAttestingBalance(ctx context.Context, root []byte) (uint64
 		return 0, errors.Wrap(err, "could not get active indices for last justified checkpoint")
 	}
 
-	wantedBlk, err := s.db.Block(ctx, bytesutil.ToBytes32(root))
+	wantedBlkSigned, err := s.db.Block(ctx, bytesutil.ToBytes32(root))
 	if err != nil {
 		return 0, errors.Wrap(err, "could not get target block")
 	}
+	if wantedBlkSigned == nil || wantedBlkSigned.Block == nil {
+		return 0, errors.New("nil wanted block")
+	}
+	wantedBlk := wantedBlkSigned.Block
 
 	balances := uint64(0)
 	s.voteLock.RLock()
@@ -225,14 +242,16 @@ func (s *Store) latestAttestingBalance(ctx context.Context, root []byte) (uint64
 // Head returns the head of the beacon chain.
 //
 // Spec pseudocode definition:
-//   def get_head(store: Store) -> Hash:
+//   def get_head(store: Store) -> Root:
+//    # Get filtered block tree that only includes viable branches
+//    blocks = get_filtered_block_tree(store)
 //    # Execute the LMD-GHOST fork choice
 //    head = store.justified_checkpoint.root
-//    justified_slot = compute_start_slot_of_epoch(store.justified_checkpoint.epoch)
+//    justified_slot = compute_start_slot_at_epoch(store.justified_checkpoint.epoch)
 //    while True:
 //        children = [
-//            root for root in store.blocks.keys()
+//            root for root in blocks.keys()
-//            if store.blocks[root].parent_root == head and store.blocks[root].slot > justified_slot
+//            if blocks[root].parent_root == head and blocks[root].slot > justified_slot
 //        ]
 //        if len(children) == 0:
 //            return head
@@ -243,13 +262,18 @@ func (s *Store) Head(ctx context.Context) ([]byte, error) {
 	defer span.End()
 
 	head := s.JustifiedCheckpt().Root
+	filteredBlocks, err := s.getFilterBlockTree(ctx)
+	if err != nil {
+		return nil, err
+	}
 
+	justifiedSlot := helpers.StartSlot(s.justifiedCheckpt.Epoch)
 	for {
-		startSlot := s.JustifiedCheckpt().Epoch * params.BeaconConfig().SlotsPerEpoch
+		children := make([][32]byte, 0, len(filteredBlocks))
-		filter := filters.NewFilter().SetParentRoot(head).SetStartSlot(startSlot)
+		for root, block := range filteredBlocks {
-		children, err := s.db.BlockRoots(ctx, filter)
+			if bytes.Equal(block.ParentRoot, head) && block.Slot > justifiedSlot {
-		if err != nil {
+				children = append(children, root)
-			return nil, errors.Wrap(err, "could not retrieve children info")
+			}
 		}
 
 		if len(children) == 0 {
@@ -280,6 +304,124 @@ func (s *Store) Head(ctx context.Context) ([]byte, error) {
 	}
 }
 
+// getFilterBlockTree retrieves a filtered block tree from store, it only returns branches
+// whose leaf state's justified and finalized info agrees with what's in the store.
+// Rationale: https://notes.ethereum.org/Fj-gVkOSTpOyUx-zkWjuwg?view
+//
+// Spec pseudocode definition:
+//   def get_filtered_block_tree(store: Store) -> Dict[Root, BeaconBlock]:
+//    """
+//    Retrieve a filtered block true from ``store``, only returning branches
+//    whose leaf state's justified/finalized info agrees with that in ``store``.
+//    """
+//    base = store.justified_checkpoint.root
+//    blocks: Dict[Root, BeaconBlock] = {}
+//    filter_block_tree(store, base, blocks)
+//    return blocks
+func (s *Store) getFilterBlockTree(ctx context.Context) (map[[32]byte]*ethpb.BeaconBlock, error) {
+	ctx, span := trace.StartSpan(ctx, "forkchoice.getFilterBlockTree")
+	defer span.End()
+
+	baseRoot := bytesutil.ToBytes32(s.justifiedCheckpt.Root)
+	filteredBlocks := make(map[[32]byte]*ethpb.BeaconBlock)
+	if _, err := s.filterBlockTree(ctx, baseRoot, filteredBlocks); err != nil {
+		return nil, err
+	}
+
+	return filteredBlocks, nil
+}
+
+// filterBlockTree filters for branches that see latest finalized and justified info as correct on-chain
+// before running Head.
+//
+// Spec pseudocode definition:
+//   def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconBlock]) -> bool:
+//    block = store.blocks[block_root]
+//    children = [
+//        root for root in store.blocks.keys()
+//        if store.blocks[root].parent_root == block_root
+//    ]
+//    # If any children branches contain expected finalized/justified checkpoints,
+//    # add to filtered block-tree and signal viability to parent.
+//    if any(children):
+//        filter_block_tree_result = [filter_block_tree(store, child, blocks) for child in children]
+//        if any(filter_block_tree_result):
+//            blocks[block_root] = block
+//            return True
+//        return False
+//    # If leaf block, check finalized/justified checkpoints as matching latest.
+//    head_state = store.block_states[block_root]
+//    correct_justified = (
+//        store.justified_checkpoint.epoch == GENESIS_EPOCH
+//        or head_state.current_justified_checkpoint == store.justified_checkpoint
+//    )
+//    correct_finalized = (
+//        store.finalized_checkpoint.epoch == GENESIS_EPOCH
+//        or head_state.finalized_checkpoint == store.finalized_checkpoint
+//    )
+//    # If expected finalized/justified, add to viable block-tree and signal viability to parent.
+//    if correct_justified and correct_finalized:
+//        blocks[block_root] = block
+//        return True
+//    # Otherwise, branch not viable
+//    return False
+func (s *Store) filterBlockTree(ctx context.Context, blockRoot [32]byte, filteredBlocks map[[32]byte]*ethpb.BeaconBlock) (bool, error) {
+	ctx, span := trace.StartSpan(ctx, "forkchoice.filterBlockTree")
+	defer span.End()
+	signed, err := s.db.Block(ctx, blockRoot)
+	if err != nil {
+		return false, err
+	}
+	if signed == nil || signed.Block == nil {
+		return false, errors.New("nil block")
+	}
+	block := signed.Block
+
+	filter := filters.NewFilter().SetParentRoot(blockRoot[:])
+	childrenRoots, err := s.db.BlockRoots(ctx, filter)
+	if err != nil {
+		return false, err
+	}
+
+	if len(childrenRoots) != 0 {
+		var filtered bool
+		for _, childRoot := range childrenRoots {
+			didFilter, err := s.filterBlockTree(ctx, childRoot, filteredBlocks)
+			if err != nil {
+				return false, err
+			}
+			if didFilter {
+				filtered = true
+			}
+		}
+		if filtered {
+			filteredBlocks[blockRoot] = block
+			return true, nil
+		}
+		return false, nil
+	}
+
+	headState, err := s.db.State(ctx, blockRoot)
+	if err != nil {
+		return false, err
+	}
+
+	if headState == nil {
+		return false, fmt.Errorf("no state matching block root %v", hex.EncodeToString(blockRoot[:]))
+	}
+
+	correctJustified := s.justifiedCheckpt.Epoch == 0 ||
+		proto.Equal(s.justifiedCheckpt, headState.CurrentJustifiedCheckpoint)
+	correctFinalized := s.finalizedCheckpt.Epoch == 0 ||
+		proto.Equal(s.finalizedCheckpt, headState.FinalizedCheckpoint)
+	if correctJustified && correctFinalized {
+		filteredBlocks[blockRoot] = block
+		return true, nil
+	}
+
+	return false, nil
+}
+
 // JustifiedCheckpt returns the latest justified check point from fork choice store.
 func (s *Store) JustifiedCheckpt() *ethpb.Checkpoint {
 	return proto.Clone(s.justifiedCheckpt).(*ethpb.Checkpoint)

beacon-chain/blockchain/forkchoice/service_test.go

@@ -16,6 +16,7 @@ import (
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
+	"github.com/prysmaticlabs/prysm/shared/stateutil"
 )
 
 func TestStore_GenesisStoreOk(t *testing.T) {
@@ -27,18 +28,21 @@ func TestStore_GenesisStoreOk(t *testing.T) {
 
 	genesisTime := time.Unix(9999, 0)
 	genesisState := &pb.BeaconState{GenesisTime: uint64(genesisTime.Unix())}
-	genesisStateRoot, err := ssz.HashTreeRoot(genesisState)
+	genesisStateRoot, err := stateutil.HashTreeRootState(genesisState)
 	if err != nil {
 		t.Fatal(err)
 	}
 	genesisBlk := blocks.NewGenesisBlock(genesisStateRoot[:])
-	genesisBlkRoot, err := ssz.SigningRoot(genesisBlk)
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
 	if err := db.SaveState(ctx, genesisState, genesisBlkRoot); err != nil {
 		t.Fatal(err)
 	}
+	if err := db.SaveGenesisBlockRoot(ctx, genesisBlkRoot); err != nil {
+		t.Fatal(err)
+	}
 
 	checkPoint := &ethpb.Checkpoint{Root: genesisBlkRoot[:]}
 	if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
@@ -68,7 +72,7 @@ func TestStore_AncestorOk(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -108,7 +112,7 @@ func TestStore_AncestorNotPartOfTheChain(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -139,7 +143,7 @@ func TestStore_LatestAttestingBalance(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -150,18 +154,21 @@ func TestStore_LatestAttestingBalance(t *testing.T) {
 	}
 
 	s := &pb.BeaconState{Validators: validators}
-	stateRoot, err := ssz.HashTreeRoot(s)
+	stateRoot, err := stateutil.HashTreeRootState(s)
 	if err != nil {
 		t.Fatal(err)
 	}
 	b := blocks.NewGenesisBlock(stateRoot[:])
-	blkRoot, err := ssz.SigningRoot(b)
+	blkRoot, err := ssz.HashTreeRoot(b.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
 	if err := db.SaveState(ctx, s, blkRoot); err != nil {
 		t.Fatal(err)
 	}
+	if err := db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
+		t.Fatal(err)
+	}
 
 	checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
 	if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
@@ -211,7 +218,7 @@ func TestStore_ChildrenBlocksFromParentRoot(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -242,7 +249,7 @@ func TestStore_GetHead(t *testing.T) {
 
 	store := NewForkChoiceService(ctx, db)
 
-	roots, err := blockTree1(db)
+	roots, err := blockTree1(db, []byte{'g'})
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -253,15 +260,21 @@ func TestStore_GetHead(t *testing.T) {
 	}
 
 	s := &pb.BeaconState{Validators: validators}
-	stateRoot, err := ssz.HashTreeRoot(s)
+	stateRoot, err := stateutil.HashTreeRootState(s)
 	if err != nil {
 		t.Fatal(err)
 	}
 	b := blocks.NewGenesisBlock(stateRoot[:])
-	blkRoot, err := ssz.SigningRoot(b)
+	blkRoot, err := ssz.HashTreeRoot(b.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
+	if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
+		t.Fatal(err)
+	}
 
 	checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
 
@@ -340,7 +353,7 @@ func TestCacheGenesisState_Correct(t *testing.T) {
 	featureconfig.Init(config)
 
 	b := &ethpb.BeaconBlock{Slot: 1}
-	r, _ := ssz.SigningRoot(b)
+	r, _ := ssz.HashTreeRoot(b)
 	s := &pb.BeaconState{GenesisTime: 99}
 
 	store.db.SaveState(ctx, s, r)
@@ -356,3 +369,149 @@ func TestCacheGenesisState_Correct(t *testing.T) {
 		}
 	}
 }
+
+func TestStore_GetFilterBlockTree_CorrectLeaf(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+
+	roots, err := blockTree1(db, []byte{'g'})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	s := &pb.BeaconState{}
+	stateRoot, err := stateutil.HashTreeRootState(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	b := blocks.NewGenesisBlock(stateRoot[:])
+	blkRoot, err := ssz.HashTreeRoot(b.Block)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
+		t.Fatal(err)
+	}
+
+	checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
+
+	if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[0])); err != nil {
+		t.Fatal(err)
+	}
+	store.justifiedCheckpt.Root = roots[0]
+	if err := store.checkpointState.AddCheckpointState(&cache.CheckpointState{
+		Checkpoint: store.justifiedCheckpt,
+		State:      s,
+	}); err != nil {
+		t.Fatal(err)
+	}
+
+	tree, err := store.getFilterBlockTree(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	wanted := make(map[[32]byte]*ethpb.BeaconBlock)
+	for _, root := range roots {
+		root32 := bytesutil.ToBytes32(root)
+		b, _ := store.db.Block(ctx, root32)
+		if b != nil {
+			wanted[root32] = b.Block
+		}
+	}
+	if !reflect.DeepEqual(tree, wanted) {
+		t.Error("Did not filter tree correctly")
+	}
+}
+
+func TestStore_GetFilterBlockTree_IncorrectLeaf(t *testing.T) {
+	ctx := context.Background()
+	db := testDB.SetupDB(t)
+	defer testDB.TeardownDB(t, db)
+
+	store := NewForkChoiceService(ctx, db)
+
+	roots, err := blockTree1(db, []byte{'g'})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	s := &pb.BeaconState{}
+	stateRoot, err := stateutil.HashTreeRootState(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	b := blocks.NewGenesisBlock(stateRoot[:])
+	blkRoot, err := ssz.HashTreeRoot(b.Block)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
+		t.Fatal(err)
+	}
+
+	checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
+
+	if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[0])); err != nil {
+		t.Fatal(err)
+	}
+	store.justifiedCheckpt.Root = roots[0]
+	if err := store.checkpointState.AddCheckpointState(&cache.CheckpointState{
+		Checkpoint: store.justifiedCheckpt,
+		State:      s,
+	}); err != nil {
+		t.Fatal(err)
+	}
+	// Filter for incorrect leaves for 1, 7 and 8
+	store.db.SaveState(ctx, &pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{}}, bytesutil.ToBytes32(roots[1]))
+	store.db.SaveState(ctx, &pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{}}, bytesutil.ToBytes32(roots[7]))
+	store.db.SaveState(ctx, &pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{}}, bytesutil.ToBytes32(roots[8]))
+	store.justifiedCheckpt.Epoch = 1
+	tree, err := store.getFilterBlockTree(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(tree) != 0 {
+		t.Error("filtered tree should be 0 length")
+	}
+
+	// Set leave 1 as correct
+	store.db.SaveState(ctx, &pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Epoch: 1, Root: store.justifiedCheckpt.Root}}, bytesutil.ToBytes32(roots[1]))
+	tree, err = store.getFilterBlockTree(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	wanted := make(map[[32]byte]*ethpb.BeaconBlock)
+	root32 := bytesutil.ToBytes32(roots[0])
+	b, err = store.db.Block(ctx, root32)
+	if err != nil {
+		t.Fatal(err)
+	}
+	wanted[root32] = b.Block
+	root32 = bytesutil.ToBytes32(roots[1])
+	b, err = store.db.Block(ctx, root32)
+	if err != nil {
+		t.Fatal(err)
+	}
+	wanted[root32] = b.Block
+
+	if !reflect.DeepEqual(tree, wanted) {
+		t.Error("Did not filter tree correctly")
+	}
+}

beacon-chain/blockchain/forkchoice/tree_test.go

@@ -15,31 +15,40 @@ import (
 // B0           /- B5 - B7
 //    \- B3 - B4 - B6 - B8
 // (B1, and B3 are all from the same slots)
-func blockTree1(db db.Database) ([][]byte, error) {
+func blockTree1(db db.Database, genesisRoot []byte) ([][]byte, error) {
-	b0 := &ethpb.BeaconBlock{Slot: 0, ParentRoot: []byte{'g'}}
+	b0 := &ethpb.BeaconBlock{Slot: 0, ParentRoot: genesisRoot}
-	r0, _ := ssz.SigningRoot(b0)
+	r0, _ := ssz.HashTreeRoot(b0)
 	b1 := &ethpb.BeaconBlock{Slot: 1, ParentRoot: r0[:]}
-	r1, _ := ssz.SigningRoot(b1)
+	r1, _ := ssz.HashTreeRoot(b1)
 	b3 := &ethpb.BeaconBlock{Slot: 3, ParentRoot: r0[:]}
-	r3, _ := ssz.SigningRoot(b3)
+	r3, _ := ssz.HashTreeRoot(b3)
 	b4 := &ethpb.BeaconBlock{Slot: 4, ParentRoot: r3[:]}
-	r4, _ := ssz.SigningRoot(b4)
+	r4, _ := ssz.HashTreeRoot(b4)
 	b5 := &ethpb.BeaconBlock{Slot: 5, ParentRoot: r4[:]}
-	r5, _ := ssz.SigningRoot(b5)
+	r5, _ := ssz.HashTreeRoot(b5)
 	b6 := &ethpb.BeaconBlock{Slot: 6, ParentRoot: r4[:]}
-	r6, _ := ssz.SigningRoot(b6)
+	r6, _ := ssz.HashTreeRoot(b6)
 	b7 := &ethpb.BeaconBlock{Slot: 7, ParentRoot: r5[:]}
-	r7, _ := ssz.SigningRoot(b7)
+	r7, _ := ssz.HashTreeRoot(b7)
 	b8 := &ethpb.BeaconBlock{Slot: 8, ParentRoot: r6[:]}
-	r8, _ := ssz.SigningRoot(b8)
+	r8, _ := ssz.HashTreeRoot(b8)
 	for _, b := range []*ethpb.BeaconBlock{b0, b1, b3, b4, b5, b6, b7, b8} {
-		if err := db.SaveBlock(context.Background(), b); err != nil {
+		if err := db.SaveBlock(context.Background(), &ethpb.SignedBeaconBlock{Block: b}); err != nil {
 			return nil, err
 		}
 		if err := db.SaveState(context.Background(), &pb.BeaconState{}, bytesutil.ToBytes32(b.ParentRoot)); err != nil {
 			return nil, err
 		}
 	}
+	if err := db.SaveState(context.Background(), &pb.BeaconState{}, r1); err != nil {
+		return nil, err
+	}
+	if err := db.SaveState(context.Background(), &pb.BeaconState{}, r7); err != nil {
+		return nil, err
+	}
+	if err := db.SaveState(context.Background(), &pb.BeaconState{}, r8); err != nil {
+		return nil, err
+	}
 	return [][]byte{r0[:], r1[:], nil, r3[:], r4[:], r5[:], r6[:], r7[:], r8[:]}, nil
 }
 
@@ -72,39 +81,39 @@ func blockTree1(db db.Database) ([][]byte, error) {
 //}
 func blockTree2(db db.Database) ([][]byte, error) {
 	b0 := &ethpb.BeaconBlock{Slot: 0, ParentRoot: []byte{'g'}}
-	r0, _ := ssz.SigningRoot(b0)
+	r0, _ := ssz.HashTreeRoot(b0)
 	b1 := &ethpb.BeaconBlock{Slot: 1, ParentRoot: r0[:]}
-	r1, _ := ssz.SigningRoot(b1)
+	r1, _ := ssz.HashTreeRoot(b1)
 	b2 := &ethpb.BeaconBlock{Slot: 2, ParentRoot: r0[:]}
-	r2, _ := ssz.SigningRoot(b2)
+	r2, _ := ssz.HashTreeRoot(b2)
 	b3 := &ethpb.BeaconBlock{Slot: 3, ParentRoot: r1[:]}
-	r3, _ := ssz.SigningRoot(b3)
+	r3, _ := ssz.HashTreeRoot(b3)
 	b4 := &ethpb.BeaconBlock{Slot: 4, ParentRoot: r1[:]}
-	r4, _ := ssz.SigningRoot(b4)
+	r4, _ := ssz.HashTreeRoot(b4)
 	b5 := &ethpb.BeaconBlock{Slot: 5, ParentRoot: r2[:]}
-	r5, _ := ssz.SigningRoot(b5)
+	r5, _ := ssz.HashTreeRoot(b5)
 	b6 := &ethpb.BeaconBlock{Slot: 6, ParentRoot: r2[:]}
-	r6, _ := ssz.SigningRoot(b6)
+	r6, _ := ssz.HashTreeRoot(b6)
 	b7 := &ethpb.BeaconBlock{Slot: 7, ParentRoot: r3[:]}
-	r7, _ := ssz.SigningRoot(b7)
+	r7, _ := ssz.HashTreeRoot(b7)
 	b8 := &ethpb.BeaconBlock{Slot: 8, ParentRoot: r3[:]}
-	r8, _ := ssz.SigningRoot(b8)
+	r8, _ := ssz.HashTreeRoot(b8)
 	b9 := &ethpb.BeaconBlock{Slot: 9, ParentRoot: r3[:]}
-	r9, _ := ssz.SigningRoot(b9)
+	r9, _ := ssz.HashTreeRoot(b9)
 	b10 := &ethpb.BeaconBlock{Slot: 10, ParentRoot: r3[:]}
-	r10, _ := ssz.SigningRoot(b10)
+	r10, _ := ssz.HashTreeRoot(b10)
 	b11 := &ethpb.BeaconBlock{Slot: 11, ParentRoot: r4[:]}
-	r11, _ := ssz.SigningRoot(b11)
+	r11, _ := ssz.HashTreeRoot(b11)
 	b12 := &ethpb.BeaconBlock{Slot: 12, ParentRoot: r6[:]}
-	r12, _ := ssz.SigningRoot(b12)
+	r12, _ := ssz.HashTreeRoot(b12)
 	b13 := &ethpb.BeaconBlock{Slot: 13, ParentRoot: r6[:]}
-	r13, _ := ssz.SigningRoot(b13)
+	r13, _ := ssz.HashTreeRoot(b13)
 	b14 := &ethpb.BeaconBlock{Slot: 14, ParentRoot: r7[:]}
-	r14, _ := ssz.SigningRoot(b14)
+	r14, _ := ssz.HashTreeRoot(b14)
 	b15 := &ethpb.BeaconBlock{Slot: 15, ParentRoot: r7[:]}
-	r15, _ := ssz.SigningRoot(b15)
+	r15, _ := ssz.HashTreeRoot(b15)
 	for _, b := range []*ethpb.BeaconBlock{b0, b1, b2, b3, b4, b5, b6, b7, b8, b9, b10, b11, b12, b13, b14, b15} {
-		if err := db.SaveBlock(context.Background(), b); err != nil {
+		if err := db.SaveBlock(context.Background(), &ethpb.SignedBeaconBlock{Block: b}); err != nil {
 			return nil, err
 		}
 		if err := db.SaveState(context.Background(), &pb.BeaconState{}, bytesutil.ToBytes32(b.ParentRoot)); err != nil {
@@ -121,19 +130,19 @@ func blockTree3(db db.Database) ([][]byte, error) {
 	roots := make([][]byte, 0, blkCount)
 	blks := make([]*ethpb.BeaconBlock, 0, blkCount)
 	b0 := &ethpb.BeaconBlock{Slot: 0, ParentRoot: []byte{'g'}}
-	r0, _ := ssz.SigningRoot(b0)
+	r0, _ := ssz.HashTreeRoot(b0)
 	roots = append(roots, r0[:])
 	blks = append(blks, b0)
 
 	for i := 1; i < blkCount; i++ {
 		b := &ethpb.BeaconBlock{Slot: uint64(i), ParentRoot: roots[len(roots)-1]}
-		r, _ := ssz.SigningRoot(b)
+		r, _ := ssz.HashTreeRoot(b)
 		roots = append(roots, r[:])
 		blks = append(blks, b)
 	}
 
 	for _, b := range blks {
-		if err := db.SaveBlock(context.Background(), b); err != nil {
+		if err := db.SaveBlock(context.Background(), &ethpb.SignedBeaconBlock{Block: b}); err != nil {
 			return nil, err
 		}
 		if err := db.SaveState(context.Background(), &pb.BeaconState{}, bytesutil.ToBytes32(b.ParentRoot)); err != nil {

beacon-chain/blockchain/receive_attestation.go

@@ -4,13 +4,13 @@ import (
 	"bytes"
 	"context"
 	"fmt"
-	"time"
 
 	"github.com/pkg/errors"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
-	"github.com/prysmaticlabs/prysm/shared/runutil"
+	"github.com/prysmaticlabs/prysm/shared/slotutil"
 	"github.com/sirupsen/logrus"
 	"go.opencensus.io/trace"
 )
@@ -41,11 +41,14 @@ func (s *Service) ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Att
 	}
 	// Only save head if it's different than the current head.
 	if !bytes.Equal(headRoot, s.HeadRoot()) {
-		headBlk, err := s.beaconDB.Block(ctx, bytesutil.ToBytes32(headRoot))
+		signed, err := s.beaconDB.Block(ctx, bytesutil.ToBytes32(headRoot))
 		if err != nil {
 			return errors.Wrap(err, "could not compute state from block head")
 		}
-		if err := s.saveHead(ctx, headBlk, bytesutil.ToBytes32(headRoot)); err != nil {
+		if signed == nil || signed.Block == nil {
+			return errors.New("nil head block")
+		}
+		if err := s.saveHead(ctx, signed, bytesutil.ToBytes32(headRoot)); err != nil {
 			return errors.Wrap(err, "could not save head")
 		}
 	}
@@ -56,21 +59,31 @@ func (s *Service) ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Att
 
 // This processes attestations from the attestation pool to account for validator votes and fork choice.
 func (s *Service) processAttestation() {
-	period := time.Duration(params.BeaconConfig().SecondsPerSlot) * time.Second
+	// Wait for state to be initialized.
-	ctx := context.Background()
+	stateChannel := make(chan *feed.Event, 1)
-	runutil.RunEvery(s.ctx, period, func() {
+	stateSub := s.stateNotifier.StateFeed().Subscribe(stateChannel)
-		atts, err := s.opsPoolService.AttestationPoolForForkchoice(ctx)
+	<-stateChannel
-		if err != nil {
+	stateSub.Unsubscribe()
-			log.WithError(err).Error("Could not retrieve attestation from pool")
-			return
-		}
 
-		for _, a := range atts {
+	st := slotutil.GetSlotTicker(s.genesisTime, params.BeaconConfig().SecondsPerSlot)
-			if err := s.ReceiveAttestationNoPubsub(ctx, a); err != nil {
+	for {
-				log.WithFields(logrus.Fields{
+		select {
-					"targetRoot": fmt.Sprintf("%#x", a.Data.Target.Root),
+		case <-s.ctx.Done():
-				}).WithError(err).Error("Could not receive attestation in chain service")
+			return
+		case <-st.C():
+			ctx := context.Background()
+			atts := s.attPool.ForkchoiceAttestations()
+			for _, a := range atts {
+				if err := s.attPool.DeleteForkchoiceAttestation(a); err != nil {
+					log.WithError(err).Error("Could not delete fork choice attestation in pool")
+				}
+
+				if err := s.ReceiveAttestationNoPubsub(ctx, a); err != nil {
+					log.WithFields(logrus.Fields{
+						"targetRoot": fmt.Sprintf("%#x", a.Data.Target.Root),
+					}).WithError(err).Error("Could not receive attestation in chain service")
+				}
 			}
 		}
-	})
+	}
 }

beacon-chain/blockchain/receive_attestation_test.go

@@ -19,14 +19,14 @@ func TestReceiveAttestationNoPubsub_ProcessCorrectly(t *testing.T) {
 	ctx := context.Background()
 
 	chainService := setupBeaconChain(t, db)
-	r, _ := ssz.SigningRoot(&ethpb.BeaconBlock{})
+	r, _ := ssz.HashTreeRoot(&ethpb.BeaconBlock{})
 	chainService.forkChoiceStore = &store{headRoot: r[:]}
 
-	b := &ethpb.BeaconBlock{}
+	b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{}}
 	if err := chainService.beaconDB.SaveBlock(ctx, b); err != nil {
 		t.Fatal(err)
 	}
-	root, err := ssz.SigningRoot(b)
+	root, err := ssz.HashTreeRoot(b.Block)
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/blockchain/receive_block.go

@@ -20,10 +20,10 @@ import (
 
 // BlockReceiver interface defines the methods of chain service receive and processing new blocks.
 type BlockReceiver interface {
-	ReceiveBlock(ctx context.Context, block *ethpb.BeaconBlock) error
+	ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlock) error
-	ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconBlock) error
+	ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.SignedBeaconBlock) error
-	ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.BeaconBlock) error
+	ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.SignedBeaconBlock) error
-	ReceiveBlockNoVerify(ctx context.Context, block *ethpb.BeaconBlock) error
+	ReceiveBlockNoVerify(ctx context.Context, block *ethpb.SignedBeaconBlock) error
 }
 
 // ReceiveBlock is a function that defines the operations that are preformed on
@@ -32,11 +32,11 @@ type BlockReceiver interface {
 //   2. Validate block, apply state transition and update check points
 //   3. Apply fork choice to the processed block
 //   4. Save latest head info
-func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveBlock")
 	defer span.End()
 
-	root, err := ssz.SigningRoot(block)
+	root, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get signing root on received block")
 	}
@@ -62,10 +62,10 @@ func (s *Service) ReceiveBlock(ctx context.Context, block *ethpb.BeaconBlock) er
 //   1. Validate block, apply state transition and update check points
 //   2. Apply fork choice to the processed block
 //   3. Save latest head info
-func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveBlockNoPubsub")
 	defer span.End()
-	blockCopy := proto.Clone(block).(*ethpb.BeaconBlock)
+	blockCopy := proto.Clone(block).(*ethpb.SignedBeaconBlock)
 
 	// Apply state transition on the new block.
 	if err := s.forkChoiceStore.OnBlock(ctx, blockCopy); err != nil {
@@ -73,7 +73,7 @@ func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconB
 		traceutil.AnnotateError(span, err)
 		return err
 	}
-	root, err := ssz.SigningRoot(blockCopy)
+	root, err := ssz.HashTreeRoot(blockCopy.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get signing root on received block")
 	}
@@ -83,23 +83,21 @@ func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconB
 	if err != nil {
 		return errors.Wrap(err, "could not get head from fork choice service")
 	}
-	headBlk, err := s.beaconDB.Block(ctx, bytesutil.ToBytes32(headRoot))
+	signedHeadBlock, err := s.beaconDB.Block(ctx, bytesutil.ToBytes32(headRoot))
 	if err != nil {
 		return errors.Wrap(err, "could not compute state from block head")
 	}
+	if signedHeadBlock == nil || signedHeadBlock.Block == nil {
+		return errors.New("nil head block")
+	}
 
 	// Only save head if it's different than the current head.
 	if !bytes.Equal(headRoot, s.HeadRoot()) {
-		if err := s.saveHead(ctx, headBlk, bytesutil.ToBytes32(headRoot)); err != nil {
+		if err := s.saveHead(ctx, signedHeadBlock, bytesutil.ToBytes32(headRoot)); err != nil {
 			return errors.Wrap(err, "could not save head")
 		}
 	}
 
-	// Remove block's contained deposits, attestations, and other operations from persistent storage.
-	if err := s.cleanupBlockOperations(ctx, blockCopy); err != nil {
-		return errors.Wrap(err, "could not clean up block deposits, attestations, and other operations")
-	}
-
 	// Send notification of the processed block to the state feed.
 	s.stateNotifier.StateFeed().Send(&feed.Event{
 		Type: statefeed.BlockProcessed,
@@ -109,14 +107,20 @@ func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconB
 		},
 	})
 
+	// Add attestations from the block to the pool for fork choice.
+	if err := s.attPool.SaveBlockAttestations(blockCopy.Block.Body.Attestations); err != nil {
+		log.Errorf("Could not save attestation for fork choice: %v", err)
+		return nil
+	}
+
 	// Reports on block and fork choice metrics.
-	s.reportSlotMetrics(blockCopy.Slot)
+	s.reportSlotMetrics(blockCopy.Block.Slot)
 
 	// Log if block is a competing block.
-	isCompetingBlock(root[:], blockCopy.Slot, headRoot, headBlk.Slot)
+	isCompetingBlock(root[:], blockCopy.Block.Slot, headRoot, signedHeadBlock.Block.Slot)
 
 	// Log state transition data.
-	logStateTransitionData(blockCopy, root[:])
+	logStateTransitionData(blockCopy.Block, root[:])
 
 	processedBlkNoPubsub.Inc()
 
@@ -127,10 +131,10 @@ func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconB
 // that are preformed blocks that is received from initial sync service. The operations consists of:
 //   1. Validate block, apply state transition and update check points
 //   2. Save latest head info
-func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveBlockNoForkchoice")
 	defer span.End()
-	blockCopy := proto.Clone(block).(*ethpb.BeaconBlock)
+	blockCopy := proto.Clone(block).(*ethpb.SignedBeaconBlock)
 
 	// Apply state transition on the incoming newly received block.
 	if err := s.forkChoiceStore.OnBlock(ctx, blockCopy); err != nil {
@@ -138,7 +142,7 @@ func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *eth
 		traceutil.AnnotateError(span, err)
 		return err
 	}
-	root, err := ssz.SigningRoot(blockCopy)
+	root, err := ssz.HashTreeRoot(blockCopy.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get signing root on received block")
 	}
@@ -149,11 +153,6 @@ func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *eth
 		}
 	}
 
-	// Remove block's contained deposits, attestations, and other operations from persistent storage.
-	if err := s.cleanupBlockOperations(ctx, blockCopy); err != nil {
-		return errors.Wrap(err, "could not clean up block deposits, attestations, and other operations")
-	}
-
 	// Send notification of the processed block to the state feed.
 	s.stateNotifier.StateFeed().Send(&feed.Event{
 		Type: statefeed.BlockProcessed,
@@ -164,10 +163,10 @@ func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *eth
 	})
 
 	// Reports on block and fork choice metrics.
-	s.reportSlotMetrics(blockCopy.Slot)
+	s.reportSlotMetrics(blockCopy.Block.Slot)
 
 	// Log state transition data.
-	logStateTransitionData(blockCopy, root[:])
+	logStateTransitionData(blockCopy.Block, root[:])
 
 	processedBlkNoPubsubForkchoice.Inc()
 	return nil
@@ -176,16 +175,16 @@ func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *eth
 // ReceiveBlockNoVerify runs state transition on a input block without verifying the block's BLS contents.
 // Depends on the security model, this is the "minimal" work a node can do to sync the chain.
 // It simulates light client behavior and assumes 100% trust with the syncing peer.
-func (s *Service) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (s *Service) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.blockchain.ReceiveBlockNoVerify")
 	defer span.End()
-	blockCopy := proto.Clone(block).(*ethpb.BeaconBlock)
+	blockCopy := proto.Clone(block).(*ethpb.SignedBeaconBlock)
 
 	// Apply state transition on the incoming newly received blockCopy without verifying its BLS contents.
 	if err := s.forkChoiceStore.OnBlockInitialSyncStateTransition(ctx, blockCopy); err != nil {
 		return errors.Wrap(err, "could not process blockCopy from fork choice service")
 	}
-	root, err := ssz.SigningRoot(blockCopy)
+	root, err := ssz.HashTreeRoot(blockCopy.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get signing root on received blockCopy")
 	}
@@ -218,35 +217,18 @@ func (s *Service) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.BeaconB
 	})
 
 	// Reports on blockCopy and fork choice metrics.
-	s.reportSlotMetrics(blockCopy.Slot)
+	s.reportSlotMetrics(blockCopy.Block.Slot)
 
 	// Log state transition data.
 	log.WithFields(logrus.Fields{
-		"slot":         blockCopy.Slot,
+		"slot":         blockCopy.Block.Slot,
-		"attestations": len(blockCopy.Body.Attestations),
+		"attestations": len(blockCopy.Block.Body.Attestations),
-		"deposits":     len(blockCopy.Body.Deposits),
+		"deposits":     len(blockCopy.Block.Body.Deposits),
 	}).Debug("Finished applying state transition")
 
 	return nil
 }
 
-// cleanupBlockOperations processes and cleans up any block operations relevant to the beacon node
-// such as attestations, exits, and deposits. We update the latest seen attestation by validator
-// in the local node's runtime, cleanup and remove pending deposits which have been included in the block
-// from our node's local cache, and process validator exits and more.
-func (s *Service) cleanupBlockOperations(ctx context.Context, block *ethpb.BeaconBlock) error {
-	// Forward processed block to operation pool to remove individual operation from DB.
-	if s.opsPoolService.IncomingProcessedBlockFeed().Send(block) == 0 {
-		log.Error("Sent processed block to no subscribers")
-	}
-
-	// Remove pending deposits from the deposit queue.
-	for _, dep := range block.Body.Deposits {
-		s.depositCache.RemovePendingDeposit(ctx, dep)
-	}
-	return nil
-}
-
 // This checks if the block is from a competing chain, emits warning and updates metrics.
 func isCompetingBlock(root []byte, slot uint64, headRoot []byte, headSlot uint64) {
 	if !bytes.Equal(root[:], headRoot) {

beacon-chain/blockchain/receive_block_test.go

@@ -8,10 +8,12 @@ import (
 
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-ssz"
+	b "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
+	"github.com/prysmaticlabs/prysm/shared/stateutil"
 	"github.com/prysmaticlabs/prysm/shared/testutil"
 	logTest "github.com/sirupsen/logrus/hooks/test"
 )
@@ -30,10 +32,13 @@ func TestReceiveBlock_ProcessCorrectly(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	genesisBlkRoot, err := ssz.SigningRoot(genesis)
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesis.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
+	if err := db.SaveState(ctx, beaconState, genesisBlkRoot); err != nil {
+		t.Fatal(err)
+	}
 	cp := &ethpb.Checkpoint{Root: genesisBlkRoot[:]}
 	if err := chainService.forkChoiceStore.GenesisStore(ctx, cp, cp); err != nil {
 		t.Fatal(err)
@@ -69,11 +74,11 @@ func TestReceiveReceiveBlockNoPubsub_CanSaveHeadInfo(t *testing.T) {
 
 	chainService := setupBeaconChain(t, db)
 
-	headBlk := &ethpb.BeaconBlock{Slot: 100}
+	headBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 100}}
 	if err := db.SaveBlock(ctx, headBlk); err != nil {
 		t.Fatal(err)
 	}
-	r, err := ssz.SigningRoot(headBlk)
+	r, err := ssz.HashTreeRoot(headBlk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -83,9 +88,12 @@ func TestReceiveReceiveBlockNoPubsub_CanSaveHeadInfo(t *testing.T) {
 	}
 	chainService.forkChoiceStore = &store{headRoot: r[:]}
 
-	if err := chainService.ReceiveBlockNoPubsub(ctx, &ethpb.BeaconBlock{
+	if err := chainService.ReceiveBlockNoPubsub(ctx, &ethpb.SignedBeaconBlock{
-		Slot: 1,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{}}); err != nil {
+			Slot: 1,
+			Body: &ethpb.BeaconBlockBody{},
+		},
+	}); err != nil {
 		t.Fatal(err)
 	}
 
@@ -108,14 +116,17 @@ func TestReceiveReceiveBlockNoPubsub_SameHead(t *testing.T) {
 
 	chainService := setupBeaconChain(t, db)
 
-	headBlk := &ethpb.BeaconBlock{}
+	headBlk := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{}}
 	if err := db.SaveBlock(ctx, headBlk); err != nil {
 		t.Fatal(err)
 	}
-	newBlk := &ethpb.BeaconBlock{
+	newBlk := &ethpb.SignedBeaconBlock{
-		Slot: 1,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{}}
+			Slot: 1,
-	newRoot, _ := ssz.SigningRoot(newBlk)
+			Body: &ethpb.BeaconBlockBody{},
+		},
+	}
+	newRoot, _ := ssz.HashTreeRoot(newBlk.Block)
 	if err := db.SaveBlock(ctx, newBlk); err != nil {
 		t.Fatal(err)
 	}
@@ -144,7 +155,20 @@ func TestReceiveBlockNoPubsubForkchoice_ProcessCorrectly(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	if err := chainService.forkChoiceStore.GenesisStore(ctx, &ethpb.Checkpoint{}, &ethpb.Checkpoint{}); err != nil {
+	stateRoot, err := stateutil.HashTreeRootState(beaconState)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	genesis := b.NewGenesisBlock(stateRoot[:])
+	parentRoot, err := ssz.HashTreeRoot(genesis.Block)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := db.SaveState(ctx, beaconState, parentRoot); err != nil {
+		t.Fatal(err)
+	}
+	if err := chainService.forkChoiceStore.GenesisStore(ctx, &ethpb.Checkpoint{Root: parentRoot[:]}, &ethpb.Checkpoint{Root: parentRoot[:]}); err != nil {
 		t.Fatal(err)
 	}
 
@@ -156,7 +180,7 @@ func TestReceiveBlockNoPubsubForkchoice_ProcessCorrectly(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	if err := db.SaveState(ctx, beaconState, bytesutil.ToBytes32(block.ParentRoot)); err != nil {
+	if err := db.SaveState(ctx, beaconState, bytesutil.ToBytes32(block.Block.ParentRoot)); err != nil {
 		t.Fatal(err)
 	}
 

beacon-chain/blockchain/service.go

@@ -21,7 +21,7 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db"
-	"github.com/prysmaticlabs/prysm/beacon-chain/operations"
+	"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
 	"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
 	"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
@@ -39,17 +39,18 @@ type Service struct {
 	beaconDB          db.Database
 	depositCache      *depositcache.DepositCache
 	chainStartFetcher powchain.ChainStartFetcher
-	opsPoolService    operations.OperationFeeds
+	attPool           attestations.Pool
 	forkChoiceStore   forkchoice.ForkChoicer
 	genesisTime       time.Time
 	p2p               p2p.Broadcaster
 	maxRoutines       int64
 	headSlot          uint64
-	headBlock         *ethpb.BeaconBlock
+	headBlock         *ethpb.SignedBeaconBlock
 	headState         *pb.BeaconState
 	canonicalRoots    map[uint64][]byte
 	headLock          sync.RWMutex
 	stateNotifier     statefeed.Notifier
+	genesisRoot       [32]byte
 }
 
 // Config options for the service.
@@ -58,7 +59,7 @@ type Config struct {
 	ChainStartFetcher powchain.ChainStartFetcher
 	BeaconDB          db.Database
 	DepositCache      *depositcache.DepositCache
-	OpsPoolService    operations.OperationFeeds
+	AttPool           attestations.Pool
 	P2p               p2p.Broadcaster
 	MaxRoutines       int64
 	StateNotifier     statefeed.Notifier
@@ -75,7 +76,7 @@ func NewService(ctx context.Context, cfg *Config) (*Service, error) {
 		beaconDB:          cfg.BeaconDB,
 		depositCache:      cfg.DepositCache,
 		chainStartFetcher: cfg.ChainStartFetcher,
-		opsPoolService:    cfg.OpsPoolService,
+		attPool:           cfg.AttPool,
 		forkChoiceStore:   store,
 		p2p:               cfg.P2p,
 		canonicalRoots:    make(map[uint64][]byte),
@@ -168,8 +169,8 @@ func (s *Service) Start() {
 // processChainStartTime initializes a series of deposits from the ChainStart deposits in the eth1
 // deposit contract, initializes the beacon chain's state, and kicks off the beacon chain.
 func (s *Service) processChainStartTime(ctx context.Context, genesisTime time.Time) {
-	initialDeposits := s.chainStartFetcher.ChainStartDeposits()
+	preGenesisState := s.chainStartFetcher.PreGenesisState()
-	if err := s.initializeBeaconChain(ctx, genesisTime, initialDeposits, s.chainStartFetcher.ChainStartEth1Data()); err != nil {
+	if err := s.initializeBeaconChain(ctx, genesisTime, preGenesisState, s.chainStartFetcher.ChainStartEth1Data()); err != nil {
 		log.Fatalf("Could not initialize beacon chain: %v", err)
 	}
 	s.stateNotifier.StateFeed().Send(&feed.Event{
@@ -186,7 +187,7 @@ func (s *Service) processChainStartTime(ctx context.Context, genesisTime time.Ti
 func (s *Service) initializeBeaconChain(
 	ctx context.Context,
 	genesisTime time.Time,
-	deposits []*ethpb.Deposit,
+	preGenesisState *pb.BeaconState,
 	eth1data *ethpb.Eth1Data) error {
 	_, span := trace.StartSpan(context.Background(), "beacon-chain.Service.initializeBeaconChain")
 	defer span.End()
@@ -194,7 +195,7 @@ func (s *Service) initializeBeaconChain(
 	s.genesisTime = genesisTime
 	unixTime := uint64(genesisTime.Unix())
 
-	genesisState, err := state.GenesisBeaconState(deposits, unixTime, eth1data)
+	genesisState, err := state.OptimizedGenesisBeaconState(unixTime, preGenesisState, eth1data)
 	if err != nil {
 		return errors.Wrap(err, "could not initialize genesis state")
 	}
@@ -229,18 +230,22 @@ func (s *Service) Status() error {
 }
 
 // This gets called to update canonical root mapping.
-func (s *Service) saveHead(ctx context.Context, b *ethpb.BeaconBlock, r [32]byte) error {
+func (s *Service) saveHead(ctx context.Context, signed *ethpb.SignedBeaconBlock, r [32]byte) error {
 	s.headLock.Lock()
 	defer s.headLock.Unlock()
 
-	s.headSlot = b.Slot
+	if signed == nil || signed.Block == nil {
+		return errors.New("cannot save nil head block")
+	}
 
-	s.canonicalRoots[b.Slot] = r[:]
+	s.headSlot = signed.Block.Slot
+
+	s.canonicalRoots[signed.Block.Slot] = r[:]
 
 	if err := s.beaconDB.SaveHeadBlockRoot(ctx, r); err != nil {
 		return errors.Wrap(err, "could not save head root in DB")
 	}
-	s.headBlock = b
+	s.headBlock = signed
 
 	headState, err := s.beaconDB.State(ctx, r)
 	if err != nil {
@@ -249,7 +254,7 @@ func (s *Service) saveHead(ctx context.Context, b *ethpb.BeaconBlock, r [32]byte
 	s.headState = headState
 
 	log.WithFields(logrus.Fields{
-		"slot":     b.Slot,
+		"slot":     signed.Block.Slot,
 		"headRoot": fmt.Sprintf("%#x", r),
 	}).Debug("Saved new head info")
 	return nil
@@ -258,13 +263,13 @@ func (s *Service) saveHead(ctx context.Context, b *ethpb.BeaconBlock, r [32]byte
 // This gets called to update canonical root mapping. It does not save head block
 // root in DB. With the inception of inital-sync-cache-state flag, it uses finalized
 // check point as anchors to resume sync therefore head is no longer needed to be saved on per slot basis.
-func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.BeaconBlock, r [32]byte) error {
+func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.SignedBeaconBlock, r [32]byte) error {
 	s.headLock.Lock()
 	defer s.headLock.Unlock()
 
-	s.headSlot = b.Slot
+	s.headSlot = b.Block.Slot
 
-	s.canonicalRoots[b.Slot] = r[:]
+	s.canonicalRoots[b.Block.Slot] = r[:]
 
 	s.headBlock = b
 
@@ -275,7 +280,7 @@ func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.BeaconBlock, r [32]
 	s.headState = headState
 
 	log.WithFields(logrus.Fields{
-		"slot":     b.Slot,
+		"slot":     b.Block.Slot,
 		"headRoot": fmt.Sprintf("%#x", r),
 	}).Debug("Saved new head info")
 	return nil
@@ -301,7 +306,7 @@ func (s *Service) saveGenesisData(ctx context.Context, genesisState *pb.BeaconSt
 		return errors.Wrap(err, "could not tree hash genesis state")
 	}
 	genesisBlk := blocks.NewGenesisBlock(stateRoot[:])
-	genesisBlkRoot, err := ssz.SigningRoot(genesisBlk)
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get genesis block root")
 	}
@@ -327,6 +332,7 @@ func (s *Service) saveGenesisData(ctx context.Context, genesisState *pb.BeaconSt
 		return errors.Wrap(err, "Could not start fork choice service: %v")
 	}
 
+	s.genesisRoot = genesisBlkRoot
 	s.headBlock = genesisBlk
 	s.headState = genesisState
 	s.canonicalRoots[genesisState.Slot] = genesisBlkRoot[:]
@@ -339,6 +345,19 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
 	s.headLock.Lock()
 	defer s.headLock.Unlock()
 
+	genesisBlock, err := s.beaconDB.GenesisBlock(ctx)
+	if err != nil {
+		return errors.Wrap(err, "could not get genesis block from db")
+	}
+	if genesisBlock == nil {
+		return errors.New("no genesis block in db")
+	}
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlock.Block)
+	if err != nil {
+		return errors.Wrap(err, "could not get signing root of genesis block")
+	}
+	s.genesisRoot = genesisBlkRoot
+
 	finalized, err := s.beaconDB.FinalizedCheckpoint(ctx)
 	if err != nil {
 		return errors.Wrap(err, "could not get finalized checkpoint from db")
@@ -357,7 +376,9 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
 		return errors.Wrap(err, "could not get finalized block from db")
 	}
 
-	s.headSlot = s.headBlock.Slot
+	if s.headBlock != nil && s.headBlock.Block != nil {
+		s.headSlot = s.headBlock.Block.Slot
+	}
 	s.canonicalRoots[s.headSlot] = finalized.Root
 
 	return nil

beacon-chain/blockchain/service_norace_test.go

@@ -25,13 +25,13 @@ func TestChainService_SaveHead_DataRace(t *testing.T) {
 	go func() {
 		s.saveHead(
 			context.Background(),
-			&ethpb.BeaconBlock{Slot: 777},
+			&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 777}},
 			[32]byte{},
 		)
 	}()
 	s.saveHead(
 		context.Background(),
-		&ethpb.BeaconBlock{Slot: 888},
+		&ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 888}},
 		[32]byte{},
 	)
 }

beacon-chain/blockchain/service_test.go

@@ -18,9 +18,10 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/feed"
 	statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db"
 	testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
-	ops "github.com/prysmaticlabs/prysm/beacon-chain/operations/testing"
+	"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
 	"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
 	"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
@@ -41,11 +42,11 @@ type store struct {
 	headRoot []byte
 }
 
-func (s *store) OnBlock(ctx context.Context, b *ethpb.BeaconBlock) error {
+func (s *store) OnBlock(ctx context.Context, b *ethpb.SignedBeaconBlock) error {
 	return nil
 }
 
-func (s *store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.BeaconBlock) error {
+func (s *store) OnBlockInitialSyncStateTransition(ctx context.Context, b *ethpb.SignedBeaconBlock) error {
 	return nil
 }
 
@@ -88,24 +89,13 @@ func (mb *mockBroadcaster) Broadcast(_ context.Context, _ proto.Message) error {
 
 var _ = p2p.Broadcaster(&mockBroadcaster{})
 
-func setupGenesisBlock(t *testing.T, cs *Service) ([32]byte, *ethpb.BeaconBlock) {
-	genesis := b.NewGenesisBlock([]byte{})
-	if err := cs.beaconDB.SaveBlock(context.Background(), genesis); err != nil {
-		t.Fatalf("could not save block to db: %v", err)
-	}
-	parentHash, err := ssz.SigningRoot(genesis)
-	if err != nil {
-		t.Fatalf("unable to get tree hash root of canonical head: %v", err)
-	}
-	return parentHash, genesis
-}
-
 func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 	endpoint := "ws://127.0.0.1"
 	ctx := context.Background()
 	var web3Service *powchain.Service
 	var err error
 	web3Service, err = powchain.NewService(ctx, &powchain.Web3ServiceConfig{
+		BeaconDB:        beaconDB,
 		ETH1Endpoint:    endpoint,
 		DepositContract: common.Address{},
 	})
@@ -118,9 +108,9 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 		BeaconDB:          beaconDB,
 		DepositCache:      depositcache.NewDepositCache(),
 		ChainStartFetcher: web3Service,
-		OpsPoolService:    &ops.Operations{},
 		P2p:               &mockBroadcaster{},
 		StateNotifier:     &mockBeaconNode{},
+		AttPool:           attestations.NewPool(),
 	}
 	if err != nil {
 		t.Fatalf("could not register blockchain service: %v", err)
@@ -129,6 +119,7 @@ func setupBeaconChain(t *testing.T, beaconDB db.Database) *Service {
 	if err != nil {
 		t.Fatalf("unable to setup chain service: %v", err)
 	}
+	chainService.genesisTime = time.Unix(1, 0) // non-zero time
 
 	return chainService
 }
@@ -144,7 +135,7 @@ func TestChainStartStop_Uninitialized(t *testing.T) {
 	stateSub := chainService.stateNotifier.StateFeed().Subscribe(stateSubChannel)
 
 	// Test the chain start state notifier.
-	genesisTime := time.Unix(0, 0)
+	genesisTime := time.Unix(1, 0)
 	chainService.Start()
 	event := &feed.Event{
 		Type: statefeed.ChainStarted,
@@ -197,7 +188,7 @@ func TestChainStartStop_Initialized(t *testing.T) {
 	chainService := setupBeaconChain(t, db)
 
 	genesisBlk := b.NewGenesisBlock([]byte{})
-	blkRoot, err := ssz.SigningRoot(genesisBlk)
+	blkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -237,11 +228,28 @@ func TestChainService_InitializeBeaconChain(t *testing.T) {
 	ctx := context.Background()
 
 	bc := setupBeaconChain(t, db)
+	var err error
 
 	// Set up 10 deposits pre chain start for validators to register
 	count := uint64(10)
 	deposits, _, _ := testutil.DeterministicDepositsAndKeys(count)
-	if err := bc.initializeBeaconChain(ctx, time.Unix(0, 0), deposits, &ethpb.Eth1Data{}); err != nil {
+	trie, _, err := testutil.DepositTrieFromDeposits(deposits)
+	if err != nil {
+		t.Fatal(err)
+	}
+	hashTreeRoot := trie.HashTreeRoot()
+	genState := state.EmptyGenesisState()
+	genState.Eth1Data = &ethpb.Eth1Data{
+		DepositRoot:  hashTreeRoot[:],
+		DepositCount: uint64(len(deposits)),
+	}
+	genState, err = b.ProcessDeposits(ctx, genState, &ethpb.BeaconBlockBody{Deposits: deposits})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := bc.initializeBeaconChain(ctx, time.Unix(0, 0), genState, &ethpb.Eth1Data{
+		DepositRoot: hashTreeRoot[:],
+	}); err != nil {
 		t.Fatal(err)
 	}
 
@@ -273,7 +281,7 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
 	ctx := context.Background()
 
 	genesis := b.NewGenesisBlock([]byte{})
-	genesisRoot, err := ssz.SigningRoot(genesis)
+	genesisRoot, err := ssz.HashTreeRoot(genesis.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -285,9 +293,9 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
 	}
 
 	finalizedSlot := params.BeaconConfig().SlotsPerEpoch*2 + 1
-	headBlock := &ethpb.BeaconBlock{Slot: finalizedSlot, ParentRoot: genesisRoot[:]}
+	headBlock := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: finalizedSlot, ParentRoot: genesisRoot[:]}}
 	headState := &pb.BeaconState{Slot: finalizedSlot}
-	headRoot, _ := ssz.SigningRoot(headBlock)
+	headRoot, _ := ssz.HashTreeRoot(headBlock.Block)
 	if err := db.SaveState(ctx, headState, headRoot); err != nil {
 		t.Fatal(err)
 	}
@@ -317,12 +325,15 @@ func TestChainService_InitializeChainInfo(t *testing.T) {
 	if !reflect.DeepEqual(s, headState) {
 		t.Error("head state incorrect")
 	}
-	if headBlock.Slot != c.HeadSlot() {
+	if headBlock.Block.Slot != c.HeadSlot() {
 		t.Error("head slot incorrect")
 	}
 	if !bytes.Equal(headRoot[:], c.HeadRoot()) {
 		t.Error("head slot incorrect")
 	}
+	if c.genesisRoot != genesisRoot {
+		t.Error("genesis block root incorrect")
+	}
 }
 
 func TestChainService_SaveHeadNoDB(t *testing.T) {
@@ -333,8 +344,8 @@ func TestChainService_SaveHeadNoDB(t *testing.T) {
 		beaconDB:       db,
 		canonicalRoots: make(map[uint64][]byte),
 	}
-	b := &ethpb.BeaconBlock{Slot: 1}
+	b := &ethpb.SignedBeaconBlock{Block: &ethpb.BeaconBlock{Slot: 1}}
-	r, _ := ssz.SigningRoot(b)
+	r, _ := ssz.HashTreeRoot(b)
 	if err := s.saveHeadNoDB(ctx, b, r); err != nil {
 		t.Fatal(err)
 	}

beacon-chain/blockchain/testing/BUILD.bazel

@@ -7,6 +7,7 @@ go_library(
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing",
     visibility = ["//beacon-chain:__subpackages__"],
     deps = [
+        "//beacon-chain/core/feed/operation:go_default_library",
         "//beacon-chain/core/feed/state:go_default_library",
         "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/db:go_default_library",

beacon-chain/blockchain/testing/mock.go

@@ -8,6 +8,7 @@ import (
 	"github.com/pkg/errors"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-ssz"
+	opfeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/operation"
 	statefeed "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db"
@@ -21,15 +22,16 @@ import (
 type ChainService struct {
 	State                       *pb.BeaconState
 	Root                        []byte
-	Block                       *ethpb.BeaconBlock
+	Block                       *ethpb.SignedBeaconBlock
 	FinalizedCheckPoint         *ethpb.Checkpoint
 	CurrentJustifiedCheckPoint  *ethpb.Checkpoint
 	PreviousJustifiedCheckPoint *ethpb.Checkpoint
-	BlocksReceived              []*ethpb.BeaconBlock
+	BlocksReceived              []*ethpb.SignedBeaconBlock
 	Genesis                     time.Time
 	Fork                        *pb.Fork
 	DB                          db.Database
 	stateNotifier               statefeed.Notifier
+	opNotifier                  opfeed.Notifier
 }
 
 // StateNotifier mocks the same method in the chain service.
@@ -53,32 +55,53 @@ func (msn *MockStateNotifier) StateFeed() *event.Feed {
 	return msn.feed
 }
 
+// OperationNotifier mocks the same method in the chain service.
+func (ms *ChainService) OperationNotifier() opfeed.Notifier {
+	if ms.opNotifier == nil {
+		ms.opNotifier = &MockOperationNotifier{}
+	}
+	return ms.opNotifier
+}
+
+// MockOperationNotifier mocks the operation notifier.
+type MockOperationNotifier struct {
+	feed *event.Feed
+}
+
+// OperationFeed returns an operation feed.
+func (mon *MockOperationNotifier) OperationFeed() *event.Feed {
+	if mon.feed == nil {
+		mon.feed = new(event.Feed)
+	}
+	return mon.feed
+}
+
 // ReceiveBlock mocks ReceiveBlock method in chain service.
-func (ms *ChainService) ReceiveBlock(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (ms *ChainService) ReceiveBlock(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	return nil
 }
 
 // ReceiveBlockNoVerify mocks ReceiveBlockNoVerify method in chain service.
-func (ms *ChainService) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (ms *ChainService) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	return nil
 }
 
 // ReceiveBlockNoPubsub mocks ReceiveBlockNoPubsub method in chain service.
-func (ms *ChainService) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (ms *ChainService) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	return nil
 }
 
 // ReceiveBlockNoPubsubForkchoice mocks ReceiveBlockNoPubsubForkchoice method in chain service.
-func (ms *ChainService) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (ms *ChainService) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *ethpb.SignedBeaconBlock) error {
 	if ms.State == nil {
 		ms.State = &pb.BeaconState{}
 	}
-	if !bytes.Equal(ms.Root, block.ParentRoot) {
+	if !bytes.Equal(ms.Root, block.Block.ParentRoot) {
-		return errors.Errorf("wanted %#x but got %#x", ms.Root, block.ParentRoot)
+		return errors.Errorf("wanted %#x but got %#x", ms.Root, block.Block.ParentRoot)
 	}
-	ms.State.Slot = block.Slot
+	ms.State.Slot = block.Block.Slot
 	ms.BlocksReceived = append(ms.BlocksReceived, block)
-	signingRoot, err := ssz.SigningRoot(block)
+	signingRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		return err
 	}
@@ -86,7 +109,7 @@ func (ms *ChainService) ReceiveBlockNoPubsubForkchoice(ctx context.Context, bloc
 		if err := ms.DB.SaveBlock(ctx, block); err != nil {
 			return err
 		}
-		logrus.Infof("Saved block with root: %#x at slot %d", signingRoot, block.Slot)
+		logrus.Infof("Saved block with root: %#x at slot %d", signingRoot, block.Block.Slot)
 	}
 	ms.Root = signingRoot[:]
 	ms.Block = block
@@ -95,6 +118,9 @@ func (ms *ChainService) ReceiveBlockNoPubsubForkchoice(ctx context.Context, bloc
 
 // HeadSlot mocks HeadSlot method in chain service.
 func (ms *ChainService) HeadSlot() uint64 {
+	if ms.State == nil {
+		return 0
+	}
 	return ms.State.Slot
 
 }
@@ -106,7 +132,7 @@ func (ms *ChainService) HeadRoot() []byte {
 }
 
 // HeadBlock mocks HeadBlock method in chain service.
-func (ms *ChainService) HeadBlock() *ethpb.BeaconBlock {
+func (ms *ChainService) HeadBlock() *ethpb.SignedBeaconBlock {
 	return ms.Block
 }
 
@@ -147,6 +173,9 @@ func (ms *ChainService) ReceiveAttestationNoPubsub(context.Context, *ethpb.Attes
 
 // HeadValidatorsIndices mocks the same method in the chain service.
 func (ms *ChainService) HeadValidatorsIndices(epoch uint64) ([]uint64, error) {
+	if ms.State == nil {
+		return []uint64{}, nil
+	}
 	return helpers.ActiveValidatorIndices(ms.State, epoch)
 }
 

beacon-chain/cache/BUILD.bazel

@@ -13,7 +13,6 @@ go_library(
     visibility = ["//beacon-chain:__subpackages__"],
     deps = [
         "//proto/beacon/p2p/v1:go_default_library",
-        "//proto/beacon/rpc/v1:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/hashutil:go_default_library",
         "//shared/params:go_default_library",
@@ -40,7 +39,6 @@ go_test(
     race = "on",
     deps = [
         "//proto/beacon/p2p/v1:go_default_library",
-        "//proto/beacon/rpc/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/hashutil:go_default_library",

beacon-chain/cache/attestation_data.go

@@ -11,7 +11,6 @@ import (
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	pb "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 	"k8s.io/client-go/tools/cache"
 )
@@ -59,7 +58,7 @@ func NewAttestationCache() *AttestationCache {
 
 // Get waits for any in progress calculation to complete before returning a
 // cached response, if any.
-func (c *AttestationCache) Get(ctx context.Context, req *pb.AttestationRequest) (*ethpb.AttestationData, error) {
+func (c *AttestationCache) Get(ctx context.Context, req *ethpb.AttestationDataRequest) (*ethpb.AttestationData, error) {
 	if !featureconfig.Get().EnableAttestationCache {
 		// Return a miss result if cache is not enabled.
 		attestationCacheMiss.Inc()
@@ -113,7 +112,7 @@ func (c *AttestationCache) Get(ctx context.Context, req *pb.AttestationRequest)
 
 // MarkInProgress a request so that any other similar requests will block on
 // Get until MarkNotInProgress is called.
-func (c *AttestationCache) MarkInProgress(req *pb.AttestationRequest) error {
+func (c *AttestationCache) MarkInProgress(req *ethpb.AttestationDataRequest) error {
 	if !featureconfig.Get().EnableAttestationCache {
 		return nil
 	}
@@ -135,7 +134,7 @@ func (c *AttestationCache) MarkInProgress(req *pb.AttestationRequest) error {
 
 // MarkNotInProgress will release the lock on a given request. This should be
 // called after put.
-func (c *AttestationCache) MarkNotInProgress(req *pb.AttestationRequest) error {
+func (c *AttestationCache) MarkNotInProgress(req *ethpb.AttestationDataRequest) error {
 	if !featureconfig.Get().EnableAttestationCache {
 		return nil
 	}
@@ -151,7 +150,7 @@ func (c *AttestationCache) MarkNotInProgress(req *pb.AttestationRequest) error {
 }
 
 // Put the response in the cache.
-func (c *AttestationCache) Put(ctx context.Context, req *pb.AttestationRequest, res *ethpb.AttestationData) error {
+func (c *AttestationCache) Put(ctx context.Context, req *ethpb.AttestationDataRequest, res *ethpb.AttestationData) error {
 	if !featureconfig.Get().EnableAttestationCache {
 		return nil
 	}
@@ -180,11 +179,11 @@ func wrapperToKey(i interface{}) (string, error) {
 	return reqToKey(w.req)
 }
 
-func reqToKey(req *pb.AttestationRequest) (string, error) {
+func reqToKey(req *ethpb.AttestationDataRequest) (string, error) {
 	return fmt.Sprintf("%d-%d", req.CommitteeIndex, req.Slot), nil
 }
 
 type attestationReqResWrapper struct {
-	req *pb.AttestationRequest
+	req *ethpb.AttestationDataRequest
 	res *ethpb.AttestationData
 }

beacon-chain/cache/attestation_data_test.go

@@ -7,14 +7,13 @@ import (
 	"github.com/gogo/protobuf/proto"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/prysm/beacon-chain/cache"
-	pb "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1"
 )
 
 func TestAttestationCache_RoundTrip(t *testing.T) {
 	ctx := context.Background()
 	c := cache.NewAttestationCache()
 
-	req := &pb.AttestationRequest{
+	req := &ethpb.AttestationDataRequest{
 		CommitteeIndex: 0,
 		Slot:           1,
 	}

beacon-chain/cache/depositcache/BUILD.bazel

@@ -9,6 +9,8 @@ go_library(
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/cache/depositcache",
     visibility = ["//beacon-chain:__subpackages__"],
     deps = [
+        "//proto/beacon/db:go_default_library",
+        "//shared/bytesutil:go_default_library",
         "//shared/hashutil:go_default_library",
         "@com_github_prometheus_client_golang//prometheus:go_default_library",
         "@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
@@ -26,6 +28,7 @@ go_test(
     ],
     embed = [":go_default_library"],
     deps = [
+        "//proto/beacon/db:go_default_library",
         "//shared/bytesutil:go_default_library",
         "@com_github_gogo_protobuf//proto:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",

beacon-chain/cache/depositcache/deposits_cache.go

@@ -11,6 +11,8 @@ import (
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
+	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	log "github.com/sirupsen/logrus"
 	"go.opencensus.io/trace"
 )
@@ -33,28 +35,19 @@ type DepositFetcher interface {
 // stores all the deposit related data that is required by the beacon-node.
 type DepositCache struct {
 	// Beacon chain deposits in memory.
-	pendingDeposits       []*DepositContainer
+	pendingDeposits       []*dbpb.DepositContainer
-	deposits              []*DepositContainer
+	deposits              []*dbpb.DepositContainer
 	depositsLock          sync.RWMutex
 	chainStartDeposits    []*ethpb.Deposit
 	chainstartPubkeys     map[string]bool
 	chainstartPubkeysLock sync.RWMutex
 }
 
-// DepositContainer object for holding the deposit and a reference to the block in
-// which the deposit transaction was included in the proof of work chain.
-type DepositContainer struct {
-	Deposit     *ethpb.Deposit
-	Block       *big.Int
-	Index       int
-	depositRoot [32]byte
-}
-
 // NewDepositCache instantiates a new deposit cache
 func NewDepositCache() *DepositCache {
 	return &DepositCache{
-		pendingDeposits:    []*DepositContainer{},
+		pendingDeposits:    []*dbpb.DepositContainer{},
-		deposits:           []*DepositContainer{},
+		deposits:           []*dbpb.DepositContainer{},
 		chainstartPubkeys:  make(map[string]bool),
 		chainStartDeposits: make([]*ethpb.Deposit, 0),
 	}
@@ -62,10 +55,10 @@ func NewDepositCache() *DepositCache {
 
 // InsertDeposit into the database. If deposit or block number are nil
 // then this method does nothing.
-func (dc *DepositCache) InsertDeposit(ctx context.Context, d *ethpb.Deposit, blockNum *big.Int, index int, depositRoot [32]byte) {
+func (dc *DepositCache) InsertDeposit(ctx context.Context, d *ethpb.Deposit, blockNum uint64, index int64, depositRoot [32]byte) {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.InsertDeposit")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertDeposit")
 	defer span.End()
-	if d == nil || blockNum == nil {
+	if d == nil {
 		log.WithFields(log.Fields{
 			"block":        blockNum,
 			"deposit":      d,
@@ -78,14 +71,36 @@ func (dc *DepositCache) InsertDeposit(ctx context.Context, d *ethpb.Deposit, blo
 	defer dc.depositsLock.Unlock()
 	// keep the slice sorted on insertion in order to avoid costly sorting on retrival.
 	heightIdx := sort.Search(len(dc.deposits), func(i int) bool { return dc.deposits[i].Index >= index })
-	newDeposits := append([]*DepositContainer{{Deposit: d, Block: blockNum, depositRoot: depositRoot, Index: index}}, dc.deposits[heightIdx:]...)
+	newDeposits := append([]*dbpb.DepositContainer{{Deposit: d, Eth1BlockHeight: blockNum, DepositRoot: depositRoot[:], Index: index}}, dc.deposits[heightIdx:]...)
 	dc.deposits = append(dc.deposits[:heightIdx], newDeposits...)
 	historicalDepositsCount.Inc()
 }
 
+// InsertDepositContainers inserts a set of deposit containers into our deposit cache.
+func (dc *DepositCache) InsertDepositContainers(ctx context.Context, ctrs []*dbpb.DepositContainer) {
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertDepositContainers")
+	defer span.End()
+	dc.depositsLock.Lock()
+	defer dc.depositsLock.Unlock()
+
+	sort.SliceStable(ctrs, func(i int, j int) bool { return ctrs[i].Index < ctrs[j].Index })
+	dc.deposits = ctrs
+	historicalDepositsCount.Add(float64(len(ctrs)))
+}
+
+// AllDepositContainers returns a list of deposits all historical deposit containers until the given block number.
+func (dc *DepositCache) AllDepositContainers(ctx context.Context) []*dbpb.DepositContainer {
+	ctx, span := trace.StartSpan(ctx, "BeaconDB.AllDepositContainers")
+	defer span.End()
+	dc.depositsLock.RLock()
+	defer dc.depositsLock.RUnlock()
+
+	return dc.deposits
+}
+
 // MarkPubkeyForChainstart sets the pubkey deposit status to true.
 func (dc *DepositCache) MarkPubkeyForChainstart(ctx context.Context, pubkey string) {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.MarkPubkeyForChainstart")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.MarkPubkeyForChainstart")
 	defer span.End()
 	dc.chainstartPubkeysLock.Lock()
 	defer dc.chainstartPubkeysLock.Unlock()
@@ -94,7 +109,7 @@ func (dc *DepositCache) MarkPubkeyForChainstart(ctx context.Context, pubkey stri
 
 // PubkeyInChainstart returns bool for whether the pubkey passed in has deposited.
 func (dc *DepositCache) PubkeyInChainstart(ctx context.Context, pubkey string) bool {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.PubkeyInChainstart")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.PubkeyInChainstart")
 	defer span.End()
 	dc.chainstartPubkeysLock.Lock()
 	defer dc.chainstartPubkeysLock.Unlock()
@@ -108,14 +123,14 @@ func (dc *DepositCache) PubkeyInChainstart(ctx context.Context, pubkey string) b
 // AllDeposits returns a list of deposits all historical deposits until the given block number
 // (inclusive). If no block is specified then this method returns all historical deposits.
 func (dc *DepositCache) AllDeposits(ctx context.Context, beforeBlk *big.Int) []*ethpb.Deposit {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.AllDeposits")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.AllDeposits")
 	defer span.End()
 	dc.depositsLock.RLock()
 	defer dc.depositsLock.RUnlock()
 
 	var deposits []*ethpb.Deposit
 	for _, ctnr := range dc.deposits {
-		if beforeBlk == nil || beforeBlk.Cmp(ctnr.Block) > -1 {
+		if beforeBlk == nil || beforeBlk.Uint64() >= ctnr.Eth1BlockHeight {
 			deposits = append(deposits, ctnr.Deposit)
 		}
 	}
@@ -125,23 +140,23 @@ func (dc *DepositCache) AllDeposits(ctx context.Context, beforeBlk *big.Int) []*
 // DepositsNumberAndRootAtHeight returns number of deposits made prior to blockheight and the
 // root that corresponds to the latest deposit at that blockheight.
 func (dc *DepositCache) DepositsNumberAndRootAtHeight(ctx context.Context, blockHeight *big.Int) (uint64, [32]byte) {
-	ctx, span := trace.StartSpan(ctx, "Beacondb.DepositsNumberAndRootAtHeight")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.DepositsNumberAndRootAtHeight")
 	defer span.End()
 	dc.depositsLock.RLock()
 	defer dc.depositsLock.RUnlock()
-	heightIdx := sort.Search(len(dc.deposits), func(i int) bool { return dc.deposits[i].Block.Cmp(blockHeight) > 0 })
+	heightIdx := sort.Search(len(dc.deposits), func(i int) bool { return dc.deposits[i].Eth1BlockHeight > blockHeight.Uint64() })
 	// send the deposit root of the empty trie, if eth1follow distance is greater than the time of the earliest
 	// deposit.
 	if heightIdx == 0 {
 		return 0, [32]byte{}
 	}
-	return uint64(heightIdx), dc.deposits[heightIdx-1].depositRoot
+	return uint64(heightIdx), bytesutil.ToBytes32(dc.deposits[heightIdx-1].DepositRoot)
 }
 
 // DepositByPubkey looks through historical deposits and finds one which contains
 // a certain public key within its deposit data.
 func (dc *DepositCache) DepositByPubkey(ctx context.Context, pubKey []byte) (*ethpb.Deposit, *big.Int) {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.DepositByPubkey")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.DepositByPubkey")
 	defer span.End()
 	dc.depositsLock.RLock()
 	defer dc.depositsLock.RUnlock()
@@ -151,7 +166,7 @@ func (dc *DepositCache) DepositByPubkey(ctx context.Context, pubKey []byte) (*et
 	for _, ctnr := range dc.deposits {
 		if bytes.Equal(ctnr.Deposit.Data.PublicKey, pubKey) {
 			deposit = ctnr.Deposit
-			blockNum = ctnr.Block
+			blockNum = big.NewInt(int64(ctnr.Eth1BlockHeight))
 			break
 		}
 	}

beacon-chain/cache/depositcache/deposits_test.go

@@ -7,6 +7,7 @@ import (
 	"testing"
 
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	logTest "github.com/sirupsen/logrus/hooks/test"
 )
@@ -19,21 +20,7 @@ func TestBeaconDB_InsertDeposit_LogsOnNilDepositInsertion(t *testing.T) {
 	hook := logTest.NewGlobal()
 	dc := DepositCache{}
 
-	dc.InsertDeposit(context.Background(), nil, big.NewInt(1), 0, [32]byte{})
+	dc.InsertDeposit(context.Background(), nil, 1, 0, [32]byte{})
-
-	if len(dc.deposits) != 0 {
-		t.Fatal("Number of deposits changed")
-	}
-	if hook.LastEntry().Message != nilDepositErr {
-		t.Errorf("Did not log correct message, wanted \"Ignoring nil deposit insertion\", got \"%s\"", hook.LastEntry().Message)
-	}
-}
-
-func TestBeaconDB_InsertDeposit_LogsOnNilBlockNumberInsertion(t *testing.T) {
-	hook := logTest.NewGlobal()
-	dc := DepositCache{}
-
-	dc.InsertDeposit(context.Background(), &ethpb.Deposit{}, nil, 0, [32]byte{})
 
 	if len(dc.deposits) != 0 {
 		t.Fatal("Number of deposits changed")
@@ -47,27 +34,27 @@ func TestBeaconDB_InsertDeposit_MaintainsSortedOrderByIndex(t *testing.T) {
 	dc := DepositCache{}
 
 	insertions := []struct {
-		blkNum  *big.Int
+		blkNum  uint64
 		deposit *ethpb.Deposit
-		index   int
+		index   int64
 	}{
 		{
-			blkNum:  big.NewInt(0),
+			blkNum:  0,
 			deposit: &ethpb.Deposit{},
 			index:   0,
 		},
 		{
-			blkNum:  big.NewInt(0),
+			blkNum:  0,
 			deposit: &ethpb.Deposit{},
 			index:   3,
 		},
 		{
-			blkNum:  big.NewInt(0),
+			blkNum:  0,
 			deposit: &ethpb.Deposit{},
 			index:   1,
 		},
 		{
-			blkNum:  big.NewInt(0),
+			blkNum:  0,
 			deposit: &ethpb.Deposit{},
 			index:   4,
 		},
@@ -77,7 +64,7 @@ func TestBeaconDB_InsertDeposit_MaintainsSortedOrderByIndex(t *testing.T) {
 		dc.InsertDeposit(context.Background(), ins.deposit, ins.blkNum, ins.index, [32]byte{})
 	}
 
-	expectedIndices := []int{0, 1, 3, 4}
+	expectedIndices := []int64{0, 1, 3, 4}
 	for i, ei := range expectedIndices {
 		if dc.deposits[i].Index != ei {
 			t.Errorf("dc.deposits[%d].Index = %d, wanted %d", i, dc.deposits[i].Index, ei)
@@ -88,34 +75,34 @@ func TestBeaconDB_InsertDeposit_MaintainsSortedOrderByIndex(t *testing.T) {
 func TestBeaconDB_AllDeposits_ReturnsAllDeposits(t *testing.T) {
 	dc := DepositCache{}
 
-	deposits := []*DepositContainer{
+	deposits := []*dbpb.DepositContainer{
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 	}
 	dc.deposits = deposits
@@ -129,34 +116,34 @@ func TestBeaconDB_AllDeposits_ReturnsAllDeposits(t *testing.T) {
 func TestBeaconDB_AllDeposits_FiltersDepositUpToAndIncludingBlockNumber(t *testing.T) {
 	dc := DepositCache{}
 
-	deposits := []*DepositContainer{
+	deposits := []*dbpb.DepositContainer{
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 	}
 	dc.deposits = deposits
@@ -171,35 +158,35 @@ func TestBeaconDB_AllDeposits_FiltersDepositUpToAndIncludingBlockNumber(t *testi
 func TestBeaconDB_DepositsNumberAndRootAtHeight_ReturnsAppropriateCountAndRoot(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.deposits = []*DepositContainer{
+	dc.deposits = []*dbpb.DepositContainer{
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(11),
+			Eth1BlockHeight: 10,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:       big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit:     &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
-			depositRoot: bytesutil.ToBytes32([]byte("root")),
+			DepositRoot:     []byte("root"),
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 		{
-			Block:   big.NewInt(12),
+			Eth1BlockHeight: 12,
-			Deposit: &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
 		},
 	}
 
@@ -216,16 +203,16 @@ func TestBeaconDB_DepositsNumberAndRootAtHeight_ReturnsAppropriateCountAndRoot(t
 func TestBeaconDB_DepositsNumberAndRootAtHeight_ReturnsEmptyTrieIfBlockHeightLessThanOldestDeposit(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.deposits = []*DepositContainer{
+	dc.deposits = []*dbpb.DepositContainer{
 		{
-			Block:       big.NewInt(10),
+			Eth1BlockHeight: 10,
-			Deposit:     &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
-			depositRoot: bytesutil.ToBytes32([]byte("root")),
+			DepositRoot:     []byte("root"),
 		},
 		{
-			Block:       big.NewInt(11),
+			Eth1BlockHeight: 11,
-			Deposit:     &ethpb.Deposit{},
+			Deposit:         &ethpb.Deposit{},
-			depositRoot: bytesutil.ToBytes32([]byte("root")),
+			DepositRoot:     []byte("root"),
 		},
 	}
 
@@ -242,9 +229,9 @@ func TestBeaconDB_DepositsNumberAndRootAtHeight_ReturnsEmptyTrieIfBlockHeightLes
 func TestBeaconDB_DepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.deposits = []*DepositContainer{
+	dc.deposits = []*dbpb.DepositContainer{
 		{
-			Block: big.NewInt(9),
+			Eth1BlockHeight: 9,
 			Deposit: &ethpb.Deposit{
 				Data: &ethpb.Deposit_Data{
 					PublicKey: []byte("pk0"),
@@ -252,7 +239,7 @@ func TestBeaconDB_DepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
 			},
 		},
 		{
-			Block: big.NewInt(10),
+			Eth1BlockHeight: 10,
 			Deposit: &ethpb.Deposit{
 				Data: &ethpb.Deposit_Data{
 					PublicKey: []byte("pk1"),
@@ -260,7 +247,7 @@ func TestBeaconDB_DepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
 			},
 		},
 		{
-			Block: big.NewInt(11),
+			Eth1BlockHeight: 11,
 			Deposit: &ethpb.Deposit{
 				Data: &ethpb.Deposit_Data{
 					PublicKey: []byte("pk1"),
@@ -268,7 +255,7 @@ func TestBeaconDB_DepositByPubkey_ReturnsFirstMatchingDeposit(t *testing.T) {
 			},
 		},
 		{
-			Block: big.NewInt(12),
+			Eth1BlockHeight: 12,
 			Deposit: &ethpb.Deposit{
 				Data: &ethpb.Deposit_Data{
 					PublicKey: []byte("pk2"),

beacon-chain/cache/depositcache/pending_deposits.go

@@ -5,6 +5,7 @@ import (
 	"math/big"
 	"sort"
 
+	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
@@ -23,15 +24,15 @@ var (
 // PendingDepositsFetcher specifically outlines a struct that can retrieve deposits
 // which have not yet been included in the chain.
 type PendingDepositsFetcher interface {
-	PendingContainers(ctx context.Context, beforeBlk *big.Int) []*DepositContainer
+	PendingContainers(ctx context.Context, beforeBlk *big.Int) []*dbpb.DepositContainer
 }
 
 // InsertPendingDeposit into the database. If deposit or block number are nil
 // then this method does nothing.
-func (dc *DepositCache) InsertPendingDeposit(ctx context.Context, d *ethpb.Deposit, blockNum *big.Int, index int, depositRoot [32]byte) {
+func (dc *DepositCache) InsertPendingDeposit(ctx context.Context, d *ethpb.Deposit, blockNum uint64, index int64, depositRoot [32]byte) {
-	ctx, span := trace.StartSpan(ctx, "BeaconDB.InsertPendingDeposit")
+	ctx, span := trace.StartSpan(ctx, "DepositsCache.InsertPendingDeposit")
 	defer span.End()
-	if d == nil || blockNum == nil {
+	if d == nil {
 		log.WithFields(log.Fields{
 			"block":   blockNum,
 			"deposit": d,
@@ -40,7 +41,8 @@ func (dc *DepositCache) InsertPendingDeposit(ctx context.Context, d *ethpb.Depos
 	}
 	dc.depositsLock.Lock()
 	defer dc.depositsLock.Unlock()
-	dc.pendingDeposits = append(dc.pendingDeposits, &DepositContainer{Deposit: d, Block: blockNum, Index: index, depositRoot: depositRoot})
+	dc.pendingDeposits = append(dc.pendingDeposits,
+		&dbpb.DepositContainer{Deposit: d, Eth1BlockHeight: blockNum, Index: index, DepositRoot: depositRoot[:]})
 	pendingDepositsCount.Inc()
 	span.AddAttributes(trace.Int64Attribute("count", int64(len(dc.pendingDeposits))))
 }
@@ -54,9 +56,9 @@ func (dc *DepositCache) PendingDeposits(ctx context.Context, beforeBlk *big.Int)
 	dc.depositsLock.RLock()
 	defer dc.depositsLock.RUnlock()
 
-	var depositCntrs []*DepositContainer
+	var depositCntrs []*dbpb.DepositContainer
 	for _, ctnr := range dc.pendingDeposits {
-		if beforeBlk == nil || beforeBlk.Cmp(ctnr.Block) > -1 {
+		if beforeBlk == nil || beforeBlk.Uint64() >= ctnr.Eth1BlockHeight {
 			depositCntrs = append(depositCntrs, ctnr)
 		}
 	}
@@ -77,15 +79,15 @@ func (dc *DepositCache) PendingDeposits(ctx context.Context, beforeBlk *big.Int)
 
 // PendingContainers returns a list of deposit containers until the given block number
 // (inclusive).
-func (dc *DepositCache) PendingContainers(ctx context.Context, beforeBlk *big.Int) []*DepositContainer {
+func (dc *DepositCache) PendingContainers(ctx context.Context, beforeBlk *big.Int) []*dbpb.DepositContainer {
 	ctx, span := trace.StartSpan(ctx, "DepositsCache.PendingDeposits")
 	defer span.End()
 	dc.depositsLock.RLock()
 	defer dc.depositsLock.RUnlock()
 
-	var depositCntrs []*DepositContainer
+	var depositCntrs []*dbpb.DepositContainer
 	for _, ctnr := range dc.pendingDeposits {
-		if beforeBlk == nil || beforeBlk.Cmp(ctnr.Block) > -1 {
+		if beforeBlk == nil || beforeBlk.Uint64() >= ctnr.Eth1BlockHeight {
 			depositCntrs = append(depositCntrs, ctnr)
 		}
 	}
@@ -151,9 +153,9 @@ func (dc *DepositCache) PrunePendingDeposits(ctx context.Context, merkleTreeInde
 	dc.depositsLock.Lock()
 	defer dc.depositsLock.Unlock()
 
-	var cleanDeposits []*DepositContainer
+	var cleanDeposits []*dbpb.DepositContainer
 	for _, dp := range dc.pendingDeposits {
-		if dp.Index >= merkleTreeIndex {
+		if dp.Index >= int64(merkleTreeIndex) {
 			cleanDeposits = append(cleanDeposits, dp)
 		}
 	}

beacon-chain/cache/depositcache/pending_deposits_test.go

@@ -6,6 +6,7 @@ import (
 	"reflect"
 	"testing"
 
+	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
 	"github.com/gogo/protobuf/proto"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 )
@@ -14,7 +15,7 @@ var _ = PendingDepositsFetcher(&DepositCache{})
 
 func TestInsertPendingDeposit_OK(t *testing.T) {
 	dc := DepositCache{}
-	dc.InsertPendingDeposit(context.Background(), &ethpb.Deposit{}, big.NewInt(111), 100, [32]byte{})
+	dc.InsertPendingDeposit(context.Background(), &ethpb.Deposit{}, 111, 100, [32]byte{})
 
 	if len(dc.pendingDeposits) != 1 {
 		t.Error("Deposit not inserted")
@@ -23,7 +24,7 @@ func TestInsertPendingDeposit_OK(t *testing.T) {
 
 func TestInsertPendingDeposit_ignoresNilDeposit(t *testing.T) {
 	dc := DepositCache{}
-	dc.InsertPendingDeposit(context.Background(), nil /*deposit*/, nil /*blockNum*/, 0, [32]byte{})
+	dc.InsertPendingDeposit(context.Background(), nil /*deposit*/, 0 /*blockNum*/, 0, [32]byte{})
 
 	if len(dc.pendingDeposits) > 0 {
 		t.Error("Unexpected deposit insertion")
@@ -34,7 +35,7 @@ func TestRemovePendingDeposit_OK(t *testing.T) {
 	db := DepositCache{}
 	depToRemove := &ethpb.Deposit{Proof: [][]byte{[]byte("A")}}
 	otherDep := &ethpb.Deposit{Proof: [][]byte{[]byte("B")}}
-	db.pendingDeposits = []*DepositContainer{
+	db.pendingDeposits = []*dbpb.DepositContainer{
 		{Deposit: depToRemove, Index: 1},
 		{Deposit: otherDep, Index: 5},
 	}
@@ -47,7 +48,7 @@ func TestRemovePendingDeposit_OK(t *testing.T) {
 
 func TestRemovePendingDeposit_IgnoresNilDeposit(t *testing.T) {
 	dc := DepositCache{}
-	dc.pendingDeposits = []*DepositContainer{{Deposit: &ethpb.Deposit{}}}
+	dc.pendingDeposits = []*dbpb.DepositContainer{{Deposit: &ethpb.Deposit{}}}
 	dc.RemovePendingDeposit(context.Background(), nil /*deposit*/)
 	if len(dc.pendingDeposits) != 1 {
 		t.Errorf("Deposit unexpectedly removed")
@@ -57,7 +58,7 @@ func TestRemovePendingDeposit_IgnoresNilDeposit(t *testing.T) {
 func TestPendingDeposit_RoundTrip(t *testing.T) {
 	dc := DepositCache{}
 	dep := &ethpb.Deposit{Proof: [][]byte{[]byte("A")}}
-	dc.InsertPendingDeposit(context.Background(), dep, big.NewInt(111), 100, [32]byte{})
+	dc.InsertPendingDeposit(context.Background(), dep, 111, 100, [32]byte{})
 	dc.RemovePendingDeposit(context.Background(), dep)
 	if len(dc.pendingDeposits) != 0 {
 		t.Error("Failed to insert & delete a pending deposit")
@@ -67,10 +68,10 @@ func TestPendingDeposit_RoundTrip(t *testing.T) {
 func TestPendingDeposits_OK(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.pendingDeposits = []*DepositContainer{
+	dc.pendingDeposits = []*dbpb.DepositContainer{
-		{Block: big.NewInt(2), Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("A")}}},
+		{Eth1BlockHeight: 2, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("A")}}},
-		{Block: big.NewInt(4), Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("B")}}},
+		{Eth1BlockHeight: 4, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("B")}}},
-		{Block: big.NewInt(6), Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("c")}}},
+		{Eth1BlockHeight: 6, Deposit: &ethpb.Deposit{Proof: [][]byte{[]byte("c")}}},
 	}
 
 	deposits := dc.PendingDeposits(context.Background(), big.NewInt(4))
@@ -92,25 +93,24 @@ func TestPendingDeposits_OK(t *testing.T) {
 func TestPrunePendingDeposits_ZeroMerkleIndex(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.pendingDeposits = []*DepositContainer{
+	dc.pendingDeposits = []*dbpb.DepositContainer{
-		{Block: big.NewInt(2), Index: 2},
+		{Eth1BlockHeight: 2, Index: 2},
-		{Block: big.NewInt(4), Index: 4},
+		{Eth1BlockHeight: 4, Index: 4},
-		{Block: big.NewInt(6), Index: 6},
+		{Eth1BlockHeight: 6, Index: 6},
-		{Block: big.NewInt(8), Index: 8},
+		{Eth1BlockHeight: 8, Index: 8},
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
 
 	dc.PrunePendingDeposits(context.Background(), 0)
-	expected := []*DepositContainer{
+	expected := []*dbpb.DepositContainer{
-		{Block: big.NewInt(2), Index: 2},
+		{Eth1BlockHeight: 2, Index: 2},
-		{Block: big.NewInt(4), Index: 4},
+		{Eth1BlockHeight: 4, Index: 4},
-		{Block: big.NewInt(6), Index: 6},
+		{Eth1BlockHeight: 6, Index: 6},
-		{Block: big.NewInt(8), Index: 8},
+		{Eth1BlockHeight: 8, Index: 8},
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
-
 	if !reflect.DeepEqual(dc.pendingDeposits, expected) {
 		t.Errorf("Unexpected deposits. got=%+v want=%+v", dc.pendingDeposits, expected)
 	}
@@ -119,40 +119,40 @@ func TestPrunePendingDeposits_ZeroMerkleIndex(t *testing.T) {
 func TestPrunePendingDeposits_OK(t *testing.T) {
 	dc := DepositCache{}
 
-	dc.pendingDeposits = []*DepositContainer{
+	dc.pendingDeposits = []*dbpb.DepositContainer{
-		{Block: big.NewInt(2), Index: 2},
+		{Eth1BlockHeight: 2, Index: 2},
-		{Block: big.NewInt(4), Index: 4},
+		{Eth1BlockHeight: 4, Index: 4},
-		{Block: big.NewInt(6), Index: 6},
+		{Eth1BlockHeight: 6, Index: 6},
-		{Block: big.NewInt(8), Index: 8},
+		{Eth1BlockHeight: 8, Index: 8},
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
 
 	dc.PrunePendingDeposits(context.Background(), 6)
-	expected := []*DepositContainer{
+	expected := []*dbpb.DepositContainer{
-		{Block: big.NewInt(6), Index: 6},
+		{Eth1BlockHeight: 6, Index: 6},
-		{Block: big.NewInt(8), Index: 8},
+		{Eth1BlockHeight: 8, Index: 8},
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
 
 	if !reflect.DeepEqual(dc.pendingDeposits, expected) {
 		t.Errorf("Unexpected deposits. got=%+v want=%+v", dc.pendingDeposits, expected)
 	}
 
-	dc.pendingDeposits = []*DepositContainer{
+	dc.pendingDeposits = []*dbpb.DepositContainer{
-		{Block: big.NewInt(2), Index: 2},
+		{Eth1BlockHeight: 2, Index: 2},
-		{Block: big.NewInt(4), Index: 4},
+		{Eth1BlockHeight: 4, Index: 4},
-		{Block: big.NewInt(6), Index: 6},
+		{Eth1BlockHeight: 6, Index: 6},
-		{Block: big.NewInt(8), Index: 8},
+		{Eth1BlockHeight: 8, Index: 8},
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
 
 	dc.PrunePendingDeposits(context.Background(), 10)
-	expected = []*DepositContainer{
+	expected = []*dbpb.DepositContainer{
-		{Block: big.NewInt(10), Index: 10},
+		{Eth1BlockHeight: 10, Index: 10},
-		{Block: big.NewInt(12), Index: 12},
+		{Eth1BlockHeight: 12, Index: 12},
 	}
 
 	if !reflect.DeepEqual(dc.pendingDeposits, expected) {

beacon-chain/core/blocks/BUILD.bazel

@@ -21,6 +21,7 @@ go_library(
         "//shared/bytesutil:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/hashutil:go_default_library",
+        "//shared/mathutil:go_default_library",
         "//shared/params:go_default_library",
         "//shared/sliceutil:go_default_library",
         "//shared/trieutil:go_default_library",

beacon-chain/core/blocks/block.go

@@ -9,13 +9,15 @@ import (
 )
 
 // NewGenesisBlock returns the canonical, genesis block for the beacon chain protocol.
-func NewGenesisBlock(stateRoot []byte) *ethpb.BeaconBlock {
+func NewGenesisBlock(stateRoot []byte) *ethpb.SignedBeaconBlock {
 	zeroHash := params.BeaconConfig().ZeroHash[:]
 	genBlock := &ethpb.BeaconBlock{
 		ParentRoot: zeroHash,
 		StateRoot:  stateRoot,
 		Body:       &ethpb.BeaconBlockBody{},
-		Signature:  params.BeaconConfig().EmptySignature[:],
 	}
-	return genBlock
+	return &ethpb.SignedBeaconBlock{
+		Block:     genBlock,
+		Signature: params.BeaconConfig().EmptySignature[:],
+	}
 }

beacon-chain/core/blocks/block_operations.go

@@ -4,8 +4,8 @@ import (
 	"bytes"
 	"context"
 	"encoding/binary"
-
 	"fmt"
+	"reflect"
 	"sort"
 
 	"github.com/gogo/protobuf/proto"
@@ -21,6 +21,7 @@ import (
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 	"github.com/prysmaticlabs/prysm/shared/hashutil"
+	"github.com/prysmaticlabs/prysm/shared/mathutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/sliceutil"
 	"github.com/prysmaticlabs/prysm/shared/trieutil"
@@ -37,6 +38,26 @@ var eth1DataCache = cache.NewEth1DataVoteCache()
 var ErrSigFailedToVerify = errors.New("signature did not verify")
 
 func verifySigningRoot(obj interface{}, pub []byte, signature []byte, domain uint64) error {
+	publicKey, err := bls.PublicKeyFromBytes(pub)
+	if err != nil {
+		return errors.Wrap(err, "could not convert bytes to public key")
+	}
+	sig, err := bls.SignatureFromBytes(signature)
+	if err != nil {
+		return errors.Wrap(err, "could not convert bytes to signature")
+	}
+	root, err := ssz.HashTreeRoot(obj)
+	if err != nil {
+		return errors.Wrap(err, "could not get signing root")
+	}
+	if !sig.Verify(root[:], publicKey, domain) {
+		return ErrSigFailedToVerify
+	}
+	return nil
+}
+
+// Deprecated: This method uses deprecated ssz.SigningRoot.
+func verifyDepositDataSigningRoot(obj *ethpb.Deposit_Data, pub []byte, signature []byte, domain uint64) error {
 	publicKey, err := bls.PublicKeyFromBytes(pub)
 	if err != nil {
 		return errors.Wrap(err, "could not convert bytes to public key")
@@ -161,9 +182,9 @@ func Eth1DataHasEnoughSupport(beaconState *pb.BeaconState, data *ethpb.Eth1Data)
 //    assert bls_verify(proposer.pubkey, signing_root(block), block.signature, get_domain(state, DOMAIN_BEACON_PROPOSER))
 func ProcessBlockHeader(
 	beaconState *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	block *ethpb.SignedBeaconBlock,
 ) (*pb.BeaconState, error) {
-	beaconState, err := ProcessBlockHeaderNoVerify(beaconState, block)
+	beaconState, err := ProcessBlockHeaderNoVerify(beaconState, block.Block)
 	if err != nil {
 		return nil, err
 	}
@@ -173,14 +194,11 @@ func ProcessBlockHeader(
 		return nil, err
 	}
 	proposer := beaconState.Validators[idx]
-	if proposer.Slashed {
-		return nil, fmt.Errorf("proposer at index %d was previously slashed", idx)
-	}
 
 	// Verify proposer signature.
 	currentEpoch := helpers.CurrentEpoch(beaconState)
 	domain := helpers.Domain(beaconState.Fork, currentEpoch, params.BeaconConfig().DomainBeaconProposer)
-	if err := verifySigningRoot(block, proposer.PublicKey, block.Signature, domain); err != nil {
+	if err := verifySigningRoot(block.Block, proposer.PublicKey, block.Signature, domain); err != nil {
 		return nil, ErrSigFailedToVerify
 	}
 
@@ -214,11 +232,14 @@ func ProcessBlockHeaderNoVerify(
 	beaconState *pb.BeaconState,
 	block *ethpb.BeaconBlock,
 ) (*pb.BeaconState, error) {
+	if block == nil {
+		return nil, errors.New("nil block")
+	}
 	if beaconState.Slot != block.Slot {
 		return nil, fmt.Errorf("state slot: %d is different then block slot: %d", beaconState.Slot, block.Slot)
 	}
 
-	parentRoot, err := ssz.SigningRoot(beaconState.LatestBlockHeader)
+	parentRoot, err := ssz.HashTreeRoot(beaconState.LatestBlockHeader)
 	if err != nil {
 		return nil, err
 	}
@@ -228,17 +249,24 @@ func ProcessBlockHeaderNoVerify(
 			block.ParentRoot, parentRoot)
 	}
 
+	idx, err := helpers.BeaconProposerIndex(beaconState)
+	if err != nil {
+		return nil, err
+	}
+	proposer := beaconState.Validators[idx]
+	if proposer.Slashed {
+		return nil, fmt.Errorf("proposer at index %d was previously slashed", idx)
+	}
+
 	bodyRoot, err := ssz.HashTreeRoot(block.Body)
 	if err != nil {
 		return nil, err
 	}
-	emptySig := make([]byte, 96)
 	beaconState.LatestBlockHeader = &ethpb.BeaconBlockHeader{
 		Slot:       block.Slot,
 		ParentRoot: block.ParentRoot,
 		StateRoot:  params.BeaconConfig().ZeroHash[:],
 		BodyRoot:   bodyRoot[:],
-		Signature:  emptySig,
 	}
 	return beaconState, nil
 }
@@ -362,8 +390,8 @@ func VerifyProposerSlashing(
 ) error {
 	proposer := beaconState.Validators[slashing.ProposerIndex]
 
-	if slashing.Header_1.Slot != slashing.Header_2.Slot {
+	if slashing.Header_1.Header.Slot != slashing.Header_2.Header.Slot {
-		return fmt.Errorf("mismatched header slots, received %d == %d", slashing.Header_1.Slot, slashing.Header_2.Slot)
+		return fmt.Errorf("mismatched header slots, received %d == %d", slashing.Header_1.Header.Slot, slashing.Header_2.Header.Slot)
 	}
 	if proto.Equal(slashing.Header_1, slashing.Header_2) {
 		return errors.New("expected slashing headers to differ")
@@ -372,10 +400,10 @@ func VerifyProposerSlashing(
 		return fmt.Errorf("validator with key %#x is not slashable", proposer.PublicKey)
 	}
 	// Using headerEpoch1 here because both of the headers should have the same epoch.
-	domain := helpers.Domain(beaconState.Fork, helpers.StartSlot(slashing.Header_1.Slot), params.BeaconConfig().DomainBeaconProposer)
+	domain := helpers.Domain(beaconState.Fork, helpers.StartSlot(slashing.Header_1.Header.Slot), params.BeaconConfig().DomainBeaconProposer)
-	headers := append([]*ethpb.BeaconBlockHeader{slashing.Header_1}, slashing.Header_2)
+	headers := []*ethpb.SignedBeaconBlockHeader{slashing.Header_1, slashing.Header_2}
 	for _, header := range headers {
-		if err := verifySigningRoot(header, proposer.PublicKey, header.Signature, domain); err != nil {
+		if err := verifySigningRoot(header.Header, proposer.PublicKey, header.Signature, domain); err != nil {
 			return errors.Wrap(err, "could not verify beacon block header")
 		}
 	}
@@ -388,19 +416,15 @@ func VerifyProposerSlashing(
 //
 // Spec pseudocode definition:
 //   def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSlashing) -> None:
-//    """
-//    Process ``AttesterSlashing`` operation.
-//    """
 //    attestation_1 = attester_slashing.attestation_1
 //    attestation_2 = attester_slashing.attestation_2
 //    assert is_slashable_attestation_data(attestation_1.data, attestation_2.data)
-//    validate_indexed_attestation(state, attestation_1)
+//    assert is_valid_indexed_attestation(state, attestation_1)
-//    validate_indexed_attestation(state, attestation_2)
+//    assert is_valid_indexed_attestation(state, attestation_2)
 //
 //    slashed_any = False
-//    attesting_indices_1 = attestation_1.custody_bit_0_indices + attestation_1.custody_bit_1_indices
+//    indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices)
-//    attesting_indices_2 = attestation_2.custody_bit_0_indices + attestation_2.custody_bit_1_indices
+//    for index in sorted(indices):
-//    for index in sorted(set(attesting_indices_1).intersection(attesting_indices_2)):
 //        if is_slashable_validator(state.validators[index], get_current_epoch(state)):
 //            slash_validator(state, index)
 //            slashed_any = True
@@ -472,10 +496,8 @@ func IsSlashableAttestationData(data1 *ethpb.AttestationData, data2 *ethpb.Attes
 }
 
 func slashableAttesterIndices(slashing *ethpb.AttesterSlashing) []uint64 {
-	att1 := slashing.Attestation_1
+	indices1 := slashing.Attestation_1.AttestingIndices
-	att2 := slashing.Attestation_1
+	indices2 := slashing.Attestation_1.AttestingIndices
-	indices1 := append(att1.CustodyBit_0Indices, att1.CustodyBit_1Indices...)
-	indices2 := append(att2.CustodyBit_0Indices, att2.CustodyBit_1Indices...)
 	return sliceutil.IntersectionUint64(indices1, indices2)
 }
 
@@ -513,10 +535,11 @@ func ProcessAttestationsNoVerify(ctx context.Context, beaconState *pb.BeaconStat
 //    data = attestation.data
 //    assert data.index < get_committee_count_at_slot(state, data.slot)
 //    assert data.target.epoch in (get_previous_epoch(state), get_current_epoch(state))
+//    assert data.target.epoch == compute_epoch_at_slot(data.slot)
 //    assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH
 //
 //    committee = get_beacon_committee(state, data.slot, data.index)
-//    assert len(attestation.aggregation_bits) == len(attestation.custody_bits) == len(committee)
+//    assert len(attestation.aggregation_bits) == len(committee)
 //
 //    pending_attestation = PendingAttestation(
 //        data=data,
@@ -561,6 +584,9 @@ func ProcessAttestationNoVerify(ctx context.Context, beaconState *pb.BeaconState
 			helpers.CurrentEpoch(beaconState),
 		)
 	}
+	if helpers.SlotToEpoch(data.Slot) != data.Target.Epoch {
+		return nil, fmt.Errorf("data slot is not in the same epoch as target %d != %d", helpers.SlotToEpoch(data.Slot), data.Target.Epoch)
+	}
 
 	s := att.Data.Slot
 	minInclusionCheck := s+params.BeaconConfig().MinAttestationInclusionDelay <= beaconState.Slot
@@ -636,13 +662,9 @@ func ProcessAttestationNoVerify(ctx context.Context, beaconState *pb.BeaconState
 //    Return the indexed attestation corresponding to ``attestation``.
 //    """
 //    attesting_indices = get_attesting_indices(state, attestation.data, attestation.aggregation_bits)
-//    custody_bit_1_indices = get_attesting_indices(state, attestation.data, attestation.custody_bits)
-//    assert custody_bit_1_indices.issubset(attesting_indices)
-//    custody_bit_0_indices = attesting_indices.difference(custody_bit_1_indices)
 //
 //    return IndexedAttestation(
-//        custody_bit_0_indices=sorted(custody_bit_0_indices),
+//        attesting_indices=sorted(attesting_indices),
-//        custody_bit_1_indices=sorted(custody_bit_1_indices),
 //        data=attestation.data,
 //        signature=attestation.signature,
 //    )
@@ -655,35 +677,13 @@ func ConvertToIndexed(ctx context.Context, attestation *ethpb.Attestation, commi
 		return nil, errors.Wrap(err, "could not get attesting indices")
 	}
 
-	cb1i, err := helpers.AttestingIndices(attestation.CustodyBits, committee)
+	sort.Slice(attIndices, func(i, j int) bool {
-	if err != nil {
+		return attIndices[i] < attIndices[j]
-		return nil, err
-	}
-	if !sliceutil.SubsetUint64(cb1i, attIndices) {
-		return nil, fmt.Errorf("%v is not a subset of %v", cb1i, attIndices)
-	}
-	cb1Map := make(map[uint64]bool)
-	for _, idx := range cb1i {
-		cb1Map[idx] = true
-	}
-	cb0i := []uint64{}
-	for _, idx := range attIndices {
-		if !cb1Map[idx] {
-			cb0i = append(cb0i, idx)
-		}
-	}
-	sort.Slice(cb0i, func(i, j int) bool {
-		return cb0i[i] < cb0i[j]
-	})
-
-	sort.Slice(cb1i, func(i, j int) bool {
-		return cb1i[i] < cb1i[j]
 	})
 	inAtt := &ethpb.IndexedAttestation{
-		Data:                attestation.Data,
+		Data:             attestation.Data,
-		Signature:           attestation.Signature,
+		Signature:        attestation.Signature,
-		CustodyBit_0Indices: cb0i,
+		AttestingIndices: attIndices,
-		CustodyBit_1Indices: cb1i,
 	}
 	return inAtt, nil
 }
@@ -695,33 +695,19 @@ func ConvertToIndexed(ctx context.Context, attestation *ethpb.Attestation, commi
 //    """
 //    Check if ``indexed_attestation`` has valid indices and signature.
 //    """
-//    bit_0_indices = indexed_attestation.custody_bit_0_indices
+//    indices = indexed_attestation.attesting_indices
-//    bit_1_indices = indexed_attestation.custody_bit_1_indices
 //
-//    # Verify no index has custody bit equal to 1 [to be removed in phase 1]
-//    if not len(bit_1_indices) == 0:
-//        return False
 //    # Verify max number of indices
-//    if not len(bit_0_indices) + len(bit_1_indices) <= MAX_VALIDATORS_PER_COMMITTEE:
+//    if not len(indices) <= MAX_VALIDATORS_PER_COMMITTEE:
-//        return False
-//    # Verify index sets are disjoint
-//    if not len(set(bit_0_indices).intersection(bit_1_indices)) == 0:
-//        return False
-//    # Verify indices are sorted
-//    if not (bit_0_indices == sorted(bit_0_indices) and bit_1_indices == sorted(bit_1_indices)):
 //        return False
+//    # Verify indices are sorted and unique
+//        if not indices == sorted(set(indices)):
 //    # Verify aggregate signature
-//    if not bls_verify_multiple(
+//    if not bls_verify(
-//        pubkeys=[
+//        pubkey=bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]),
-//            bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_0_indices]),
+//        message_hash=hash_tree_root(indexed_attestation.data),
-//            bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_1_indices]),
-//        ],
-//        message_hashes=[
-//            hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b0)),
-//            hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b1)),
-//        ],
 //        signature=indexed_attestation.signature,
-//        domain=get_domain(state, DOMAIN_ATTESTATION, indexed_attestation.data.target.epoch),
+//        domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch),
 //    ):
 //        return False
 //    return True
@@ -729,87 +715,48 @@ func VerifyIndexedAttestation(ctx context.Context, beaconState *pb.BeaconState,
 	ctx, span := trace.StartSpan(ctx, "core.VerifyIndexedAttestation")
 	defer span.End()
 
-	custodyBit0Indices := indexedAtt.CustodyBit_0Indices
+	indices := indexedAtt.AttestingIndices
-	custodyBit1Indices := indexedAtt.CustodyBit_1Indices
 
-	// To be removed in phase 1
+	if uint64(len(indices)) > params.BeaconConfig().MaxValidatorsPerCommittee {
-	if len(custodyBit1Indices) != 0 {
+		return fmt.Errorf("validator indices count exceeds MAX_VALIDATORS_PER_COMMITTEE, %d > %d", len(indices), params.BeaconConfig().MaxValidatorsPerCommittee)
-		return fmt.Errorf("expected no bit 1 indices, received %v", len(custodyBit1Indices))
 	}
 
-	maxIndices := params.BeaconConfig().MaxValidatorsPerCommittee
+	set := make(map[uint64]bool)
-	totalIndicesLength := uint64(len(custodyBit0Indices) + len(custodyBit1Indices))
+	setIndices := make([]uint64, 0, len(indices))
-	if totalIndicesLength > maxIndices {
+	for _, i := range indices {
-		return fmt.Errorf("over max number of allowed indices per attestation: %d", totalIndicesLength)
+		if ok := set[i]; ok {
+			continue
+		}
+		setIndices = append(setIndices, i)
+		set[i] = true
 	}
-	custodyBitIntersection := sliceutil.IntersectionUint64(custodyBit0Indices, custodyBit1Indices)
+	sort.SliceStable(setIndices, func(i, j int) bool {
-	if len(custodyBitIntersection) != 0 {
+		return setIndices[i] < setIndices[j]
-		return fmt.Errorf("expected disjoint indices intersection, received %v", custodyBitIntersection)
-	}
-
-	custodyBit0IndicesIsSorted := sort.SliceIsSorted(custodyBit0Indices, func(i, j int) bool {
-		return custodyBit0Indices[i] < custodyBit0Indices[j]
 	})
-
+	if !reflect.DeepEqual(setIndices, indices) {
-	if !custodyBit0IndicesIsSorted {
+		return errors.New("attesting indices is not uniquely sorted")
-		return fmt.Errorf("custody Bit0 indices are not sorted, got %v", custodyBit0Indices)
-	}
-
-	custodyBit1IndicesIsSorted := sort.SliceIsSorted(custodyBit1Indices, func(i, j int) bool {
-		return custodyBit1Indices[i] < custodyBit1Indices[j]
-	})
-
-	if !custodyBit1IndicesIsSorted {
-		return fmt.Errorf("custody Bit1 indices are not sorted, got %v", custodyBit1Indices)
 	}
 
 	domain := helpers.Domain(beaconState.Fork, indexedAtt.Data.Target.Epoch, params.BeaconConfig().DomainBeaconAttester)
-	var pubkeys []*bls.PublicKey
+	var pubkey *bls.PublicKey
-	if len(custodyBit0Indices) > 0 {
+	var err error
-		pubkey, err := bls.PublicKeyFromBytes(beaconState.Validators[custodyBit0Indices[0]].PublicKey)
+	if len(indices) > 0 {
+		pubkey, err = bls.PublicKeyFromBytes(beaconState.Validators[indices[0]].PublicKey)
 		if err != nil {
 			return errors.Wrap(err, "could not deserialize validator public key")
 		}
-		for _, i := range custodyBit0Indices[1:] {
+		for _, i := range indices[1:] {
 			pk, err := bls.PublicKeyFromBytes(beaconState.Validators[i].PublicKey)
 			if err != nil {
 				return errors.Wrap(err, "could not deserialize validator public key")
 			}
 			pubkey.Aggregate(pk)
 		}
-		pubkeys = append(pubkeys, pubkey)
-	}
-	if len(custodyBit1Indices) > 0 {
-		pubkey, err := bls.PublicKeyFromBytes(beaconState.Validators[custodyBit1Indices[0]].PublicKey)
-		if err != nil {
-			return errors.Wrap(err, "could not deserialize validator public key")
-		}
-		for _, i := range custodyBit1Indices[1:] {
-			pk, err := bls.PublicKeyFromBytes(beaconState.Validators[i].PublicKey)
-			if err != nil {
-				return errors.Wrap(err, "could not deserialize validator public key")
-			}
-			pubkey.Aggregate(pk)
-		}
-		pubkeys = append(pubkeys, pubkey)
 	}
 
-	var msgs [][32]byte
+	messageHash, err := ssz.HashTreeRoot(indexedAtt.Data)
-	cus0 := &pb.AttestationDataAndCustodyBit{Data: indexedAtt.Data, CustodyBit: false}
+	if err != nil {
-	cus1 := &pb.AttestationDataAndCustodyBit{Data: indexedAtt.Data, CustodyBit: true}
+		return errors.Wrap(err, "could not tree hash att data")
-	if len(custodyBit0Indices) > 0 {
-		cus0Root, err := ssz.HashTreeRoot(cus0)
-		if err != nil {
-			return errors.Wrap(err, "could not tree hash att data and custody bit 0")
-		}
-		msgs = append(msgs, cus0Root)
-	}
-	if len(custodyBit1Indices) > 0 {
-		cus1Root, err := ssz.HashTreeRoot(cus1)
-		if err != nil {
-			return errors.Wrap(err, "could not tree hash att data and custody bit 1")
-		}
-		msgs = append(msgs, cus1Root)
 	}
 
 	sig, err := bls.SignatureFromBytes(indexedAtt.Signature)
@@ -817,9 +764,8 @@ func VerifyIndexedAttestation(ctx context.Context, beaconState *pb.BeaconState,
 		return errors.Wrap(err, "could not convert bytes to signature")
 	}
 
-	hasVotes := len(custodyBit0Indices) > 0 || len(custodyBit1Indices) > 0
+	voted := len(indices) > 0
-
+	if voted && !sig.Verify(messageHash[:], pubkey, domain) {
-	if hasVotes && !sig.VerifyAggregate(pubkeys, msgs, domain) {
 		return ErrSigFailedToVerify
 	}
 	return nil
@@ -860,6 +806,29 @@ func ProcessDeposits(ctx context.Context, beaconState *pb.BeaconState, body *eth
 	return beaconState, nil
 }
 
+// ProcessPreGenesisDeposit processes a deposit for the beacon state before chainstart.
+func ProcessPreGenesisDeposit(ctx context.Context, beaconState *pb.BeaconState,
+	deposit *ethpb.Deposit, validatorIndices map[[48]byte]int) (*pb.BeaconState, error) {
+	var err error
+	beaconState, err = ProcessDeposit(beaconState, deposit, validatorIndices)
+	if err != nil {
+		return nil, errors.Wrap(err, "could not process deposit")
+	}
+	pubkey := deposit.Data.PublicKey
+	index, ok := validatorIndices[bytesutil.ToBytes48(pubkey)]
+	if !ok {
+		return beaconState, nil
+	}
+	balance := beaconState.Balances[index]
+	beaconState.Validators[index].EffectiveBalance = mathutil.Min(balance-balance%params.BeaconConfig().EffectiveBalanceIncrement, params.BeaconConfig().MaxEffectiveBalance)
+	if beaconState.Validators[index].EffectiveBalance ==
+		params.BeaconConfig().MaxEffectiveBalance {
+		beaconState.Validators[index].ActivationEligibilityEpoch = 0
+		beaconState.Validators[index].ActivationEpoch = 0
+	}
+	return beaconState, nil
+}
+
 // ProcessDeposit takes in a deposit object and inserts it
 // into the registry as a new validator or balance change.
 //
@@ -914,7 +883,7 @@ func ProcessDeposit(beaconState *pb.BeaconState, deposit *ethpb.Deposit, valInde
 	if !ok {
 		domain := bls.ComputeDomain(params.BeaconConfig().DomainDeposit)
 		depositSig := deposit.Data.Signature
-		if err := verifySigningRoot(deposit.Data, pubKey, depositSig, domain); err != nil {
+		if err := verifyDepositDataSigningRoot(deposit.Data, pubKey, depositSig, domain); err != nil {
 			// Ignore this error as in the spec pseudo code.
 			log.Errorf("Skipping deposit: could not verify deposit data signature: %v", err)
 			return beaconState, nil
@@ -934,6 +903,7 @@ func ProcessDeposit(beaconState *pb.BeaconState, deposit *ethpb.Deposit, valInde
 			EffectiveBalance:           effectiveBalance,
 		})
 		beaconState.Balances = append(beaconState.Balances, amount)
+		valIndexMap[bytesutil.ToBytes48(pubKey)] = len(beaconState.Validators) - 1
 	} else {
 		beaconState = helpers.IncreaseBalance(beaconState, uint64(index), amount)
 	}
@@ -993,7 +963,7 @@ func ProcessVoluntaryExits(ctx context.Context, beaconState *pb.BeaconState, bod
 		if err := VerifyExit(beaconState, exit); err != nil {
 			return nil, errors.Wrapf(err, "could not verify exit %d", idx)
 		}
-		beaconState, err = v.InitiateValidatorExit(beaconState, exit.ValidatorIndex)
+		beaconState, err = v.InitiateValidatorExit(beaconState, exit.Exit.ValidatorIndex)
 		if err != nil {
 			return nil, err
 		}
@@ -1011,7 +981,7 @@ func ProcessVoluntaryExitsNoVerify(
 	exits := body.VoluntaryExits
 
 	for idx, exit := range exits {
-		beaconState, err = v.InitiateValidatorExit(beaconState, exit.ValidatorIndex)
+		beaconState, err = v.InitiateValidatorExit(beaconState, exit.Exit.ValidatorIndex)
 		if err != nil {
 			return nil, errors.Wrapf(err, "failed to process voluntary exit at index %d", idx)
 		}
@@ -1038,7 +1008,12 @@ func ProcessVoluntaryExitsNoVerify(
 //    # Verify signature
 //    domain = get_domain(state, DOMAIN_VOLUNTARY_EXIT, exit.epoch)
 //    assert bls_verify(validator.pubkey, signing_root(exit), exit.signature, domain)
-func VerifyExit(beaconState *pb.BeaconState, exit *ethpb.VoluntaryExit) error {
+func VerifyExit(beaconState *pb.BeaconState, signed *ethpb.SignedVoluntaryExit) error {
+	if signed == nil || signed.Exit == nil {
+		return errors.New("nil exit")
+	}
+
+	exit := signed.Exit
 	if int(exit.ValidatorIndex) >= len(beaconState.Validators) {
 		return fmt.Errorf("validator index out of bound %d > %d", exit.ValidatorIndex, len(beaconState.Validators))
 	}
@@ -1066,7 +1041,7 @@ func VerifyExit(beaconState *pb.BeaconState, exit *ethpb.VoluntaryExit) error {
 		)
 	}
 	domain := helpers.Domain(beaconState.Fork, exit.Epoch, params.BeaconConfig().DomainVoluntaryExit)
-	if err := verifySigningRoot(exit, validator.PublicKey, exit.Signature, domain); err != nil {
+	if err := verifySigningRoot(exit, validator.PublicKey, signed.Signature, domain); err != nil {
 		return ErrSigFailedToVerify
 	}
 	return nil

beacon-chain/core/blocks/block_operations_fuzz_test.go

@@ -26,7 +26,7 @@ func TestFuzzProcessAttestation_10000(t *testing.T) {
 func TestFuzzProcessBlockHeader_10000(t *testing.T) {
 	fuzzer := fuzz.NewWithSeed(0)
 	state := &ethereum_beacon_p2p_v1.BeaconState{}
-	block := &eth.BeaconBlock{}
+	block := &eth.SignedBeaconBlock{}
 
 	for i := 0; i < 10000; i++ {
 		fuzzer.Fuzz(state)

beacon-chain/core/blocks/block_operations_test.go

@@ -34,7 +34,7 @@ func TestProcessBlockHeader_WrongProposerSig(t *testing.T) {
 	beaconState, privKeys := testutil.DeterministicGenesisState(t, 100)
 	beaconState.LatestBlockHeader = &ethpb.BeaconBlockHeader{Slot: 9}
 
-	lbhsr, err := ssz.SigningRoot(beaconState.LatestBlockHeader)
+	lbhsr, err := ssz.HashTreeRoot(beaconState.LatestBlockHeader)
 	if err != nil {
 		t.Error(err)
 	}
@@ -44,14 +44,16 @@ func TestProcessBlockHeader_WrongProposerSig(t *testing.T) {
 		t.Error(err)
 	}
 
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 0,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: 0,
-			RandaoReveal: []byte{'A', 'B', 'C'},
+			Body: &ethpb.BeaconBlockBody{
+				RandaoReveal: []byte{'A', 'B', 'C'},
+			},
+			ParentRoot: lbhsr[:],
 		},
-		ParentRoot: lbhsr[:],
 	}
-	signingRoot, err := ssz.SigningRoot(block)
+	signingRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatalf("Failed to get signing root of block: %v", err)
 	}
@@ -96,13 +98,15 @@ func TestProcessBlockHeader_DifferentSlots(t *testing.T) {
 	priv := bls.RandKey()
 	blockSig := priv.Sign([]byte("hello"), dt)
 	validators[5896].PublicKey = priv.PublicKey().Marshal()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 1,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: 1,
-			RandaoReveal: []byte{'A', 'B', 'C'},
+			Body: &ethpb.BeaconBlockBody{
+				RandaoReveal: []byte{'A', 'B', 'C'},
+			},
+			ParentRoot: lbhsr[:],
 		},
-		ParentRoot: lbhsr[:],
+		Signature: blockSig.Marshal(),
-		Signature:  blockSig.Marshal(),
 	}
 
 	_, err = blocks.ProcessBlockHeader(state, block)
@@ -137,13 +141,15 @@ func TestProcessBlockHeader_PreviousBlockRootNotSignedRoot(t *testing.T) {
 	priv := bls.RandKey()
 	blockSig := priv.Sign([]byte("hello"), dt)
 	validators[5896].PublicKey = priv.PublicKey().Marshal()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 0,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: 0,
-			RandaoReveal: []byte{'A', 'B', 'C'},
+			Body: &ethpb.BeaconBlockBody{
+				RandaoReveal: []byte{'A', 'B', 'C'},
+			},
+			ParentRoot: []byte{'A'},
 		},
-		ParentRoot: []byte{'A'},
+		Signature: blockSig.Marshal(),
-		Signature:  blockSig.Marshal(),
 	}
 
 	_, err := blocks.ProcessBlockHeader(state, block)
@@ -173,7 +179,7 @@ func TestProcessBlockHeader_SlashedProposer(t *testing.T) {
 		RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector),
 	}
 
-	parentRoot, err := ssz.SigningRoot(state.LatestBlockHeader)
+	parentRoot, err := ssz.HashTreeRoot(state.LatestBlockHeader)
 	if err != nil {
 		t.Error(err)
 	}
@@ -182,13 +188,15 @@ func TestProcessBlockHeader_SlashedProposer(t *testing.T) {
 	priv := bls.RandKey()
 	blockSig := priv.Sign([]byte("hello"), dt)
 	validators[12683].PublicKey = priv.PublicKey().Marshal()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 0,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: 0,
-			RandaoReveal: []byte{'A', 'B', 'C'},
+			Body: &ethpb.BeaconBlockBody{
+				RandaoReveal: []byte{'A', 'B', 'C'},
+			},
+			ParentRoot: parentRoot[:],
 		},
-		ParentRoot: parentRoot[:],
+		Signature: blockSig.Marshal(),
-		Signature:  blockSig.Marshal(),
 	}
 
 	_, err = blocks.ProcessBlockHeader(state, block)
@@ -218,27 +226,29 @@ func TestProcessBlockHeader_OK(t *testing.T) {
 		RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector),
 	}
 
-	latestBlockSignedRoot, err := ssz.SigningRoot(state.LatestBlockHeader)
+	latestBlockSignedRoot, err := ssz.HashTreeRoot(state.LatestBlockHeader)
 	if err != nil {
 		t.Error(err)
 	}
 	currentEpoch := helpers.CurrentEpoch(state)
 	dt := helpers.Domain(state.Fork, currentEpoch, params.BeaconConfig().DomainBeaconProposer)
 	priv := bls.RandKey()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 0,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: 0,
-			RandaoReveal: []byte{'A', 'B', 'C'},
+			Body: &ethpb.BeaconBlockBody{
+				RandaoReveal: []byte{'A', 'B', 'C'},
+			},
+			ParentRoot: latestBlockSignedRoot[:],
 		},
-		ParentRoot: latestBlockSignedRoot[:],
 	}
-	signingRoot, err := ssz.SigningRoot(block)
+	signingRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatalf("Failed to get signing root of block: %v", err)
 	}
 	blockSig := priv.Sign(signingRoot[:], dt)
 	block.Signature = blockSig.Marshal()[:]
-	bodyRoot, err := ssz.HashTreeRoot(block.Body)
+	bodyRoot, err := ssz.HashTreeRoot(block.Block.Body)
 	if err != nil {
 		t.Fatalf("Failed to hash block bytes got: %v", err)
 	}
@@ -255,17 +265,15 @@ func TestProcessBlockHeader_OK(t *testing.T) {
 		t.Fatalf("Failed to process block header got: %v", err)
 	}
 	var zeroHash [32]byte
-	var zeroSig [96]byte
 	nsh := newState.LatestBlockHeader
 	expected := &ethpb.BeaconBlockHeader{
-		Slot:       block.Slot,
+		Slot:       block.Block.Slot,
 		ParentRoot: latestBlockSignedRoot[:],
 		BodyRoot:   bodyRoot[:],
 		StateRoot:  zeroHash[:],
-		Signature:  zeroSig[:],
 	}
 	if !proto.Equal(nsh, expected) {
-		t.Errorf("Expected %v, received %vk9k", expected, nsh)
+		t.Errorf("Expected %v, received %v", expected, nsh)
 	}
 }
 
@@ -370,11 +378,15 @@ func TestProcessProposerSlashings_UnmatchedHeaderSlots(t *testing.T) {
 	slashings := []*ethpb.ProposerSlashing{
 		{
 			ProposerIndex: 1,
-			Header_1: &ethpb.BeaconBlockHeader{
+			Header_1: &ethpb.SignedBeaconBlockHeader{
-				Slot: params.BeaconConfig().SlotsPerEpoch + 1,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: params.BeaconConfig().SlotsPerEpoch + 1,
+				},
 			},
-			Header_2: &ethpb.BeaconBlockHeader{
+			Header_2: &ethpb.SignedBeaconBlockHeader{
-				Slot: 0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 			},
 		},
 	}
@@ -400,11 +412,15 @@ func TestProcessProposerSlashings_SameHeaders(t *testing.T) {
 	slashings := []*ethpb.ProposerSlashing{
 		{
 			ProposerIndex: 1,
-			Header_1: &ethpb.BeaconBlockHeader{
+			Header_1: &ethpb.SignedBeaconBlockHeader{
-				Slot: 0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 			},
-			Header_2: &ethpb.BeaconBlockHeader{
+			Header_2: &ethpb.SignedBeaconBlockHeader{
-				Slot: 0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 			},
 		},
 	}
@@ -437,12 +453,16 @@ func TestProcessProposerSlashings_ValidatorNotSlashable(t *testing.T) {
 	slashings := []*ethpb.ProposerSlashing{
 		{
 			ProposerIndex: 0,
-			Header_1: &ethpb.BeaconBlockHeader{
+			Header_1: &ethpb.SignedBeaconBlockHeader{
-				Slot:      0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 				Signature: []byte("A"),
 			},
-			Header_2: &ethpb.BeaconBlockHeader{
+			Header_2: &ethpb.SignedBeaconBlockHeader{
-				Slot:      0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 				Signature: []byte("B"),
 			},
 		},
@@ -474,21 +494,25 @@ func TestProcessProposerSlashings_AppliesCorrectStatus(t *testing.T) {
 	proposerIdx := uint64(1)
 
 	domain := helpers.Domain(beaconState.Fork, 0, params.BeaconConfig().DomainBeaconProposer)
-	header1 := &ethpb.BeaconBlockHeader{
+	header1 := &ethpb.SignedBeaconBlockHeader{
-		Slot:      0,
+		Header: &ethpb.BeaconBlockHeader{
-		StateRoot: []byte("A"),
+			Slot:      0,
+			StateRoot: []byte("A"),
+		},
 	}
-	signingRoot, err := ssz.SigningRoot(header1)
+	signingRoot, err := ssz.HashTreeRoot(header1.Header)
 	if err != nil {
 		t.Errorf("Could not get signing root of beacon block header: %v", err)
 	}
 	header1.Signature = privKeys[proposerIdx].Sign(signingRoot[:], domain).Marshal()[:]
 
-	header2 := &ethpb.BeaconBlockHeader{
+	header2 := &ethpb.SignedBeaconBlockHeader{
-		Slot:      0,
+		Header: &ethpb.BeaconBlockHeader{
-		StateRoot: []byte("B"),
+			Slot:      0,
+			StateRoot: []byte("B"),
+		},
 	}
-	signingRoot, err = ssz.SigningRoot(header2)
+	signingRoot, err = ssz.HashTreeRoot(header2.Header)
 	if err != nil {
 		t.Errorf("Could not get signing root of beacon block header: %v", err)
 	}
@@ -577,26 +601,6 @@ func TestProcessAttesterSlashings_DataNotSlashable(t *testing.T) {
 }
 
 func TestProcessAttesterSlashings_IndexedAttestationFailedToVerify(t *testing.T) {
-	slashings := []*ethpb.AttesterSlashing{
-		{
-			Attestation_1: &ethpb.IndexedAttestation{
-				Data: &ethpb.AttestationData{
-					Source: &ethpb.Checkpoint{Epoch: 1},
-					Target: &ethpb.Checkpoint{Epoch: 0},
-				},
-				CustodyBit_0Indices: []uint64{0, 1, 2},
-				CustodyBit_1Indices: []uint64{0, 1, 2},
-			},
-			Attestation_2: &ethpb.IndexedAttestation{
-				Data: &ethpb.AttestationData{
-					Source: &ethpb.Checkpoint{Epoch: 0},
-					Target: &ethpb.Checkpoint{Epoch: 0},
-				},
-				CustodyBit_0Indices: []uint64{0, 1, 2},
-				CustodyBit_1Indices: []uint64{0, 1, 2},
-			},
-		},
-	}
 	registry := []*ethpb.Validator{}
 	currentSlot := uint64(0)
 
@@ -604,39 +608,33 @@ func TestProcessAttesterSlashings_IndexedAttestationFailedToVerify(t *testing.T)
 		Validators: registry,
 		Slot:       currentSlot,
 	}
-	block := &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
-			AttesterSlashings: slashings,
-		},
-	}
-	want := fmt.Sprint("expected no bit 1 indices")
 
-	if _, err := blocks.ProcessAttesterSlashings(context.Background(), beaconState, block.Body); !strings.Contains(err.Error(), want) {
+	slashings := []*ethpb.AttesterSlashing{
-		t.Errorf("Expected %s, received %v", want, err)
-	}
-
-	slashings = []*ethpb.AttesterSlashing{
 		{
 			Attestation_1: &ethpb.IndexedAttestation{
 				Data: &ethpb.AttestationData{
 					Source: &ethpb.Checkpoint{Epoch: 1},
 					Target: &ethpb.Checkpoint{Epoch: 0},
 				},
-				CustodyBit_0Indices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+1),
+				AttestingIndices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+1),
 			},
 			Attestation_2: &ethpb.IndexedAttestation{
 				Data: &ethpb.AttestationData{
 					Source: &ethpb.Checkpoint{Epoch: 0},
 					Target: &ethpb.Checkpoint{Epoch: 0},
 				},
-				CustodyBit_0Indices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+1),
+				AttestingIndices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+1),
 			},
 		},
 	}
 
-	block.Body.AttesterSlashings = slashings
+	block := &ethpb.BeaconBlock{
-	want = fmt.Sprint("over max number of allowed indices")
+		Body: &ethpb.BeaconBlockBody{
+			AttesterSlashings: slashings,
+		},
+	}
 
+	want := fmt.Sprint("validator indices count exceeds MAX_VALIDATORS_PER_COMMITTEE")
 	if _, err := blocks.ProcessAttesterSlashings(context.Background(), beaconState, block.Body); !strings.Contains(err.Error(), want) {
 		t.Errorf("Expected %s, received %v", want, err)
 	}
@@ -653,13 +651,9 @@ func TestProcessAttesterSlashings_AppliesCorrectStatus(t *testing.T) {
 			Source: &ethpb.Checkpoint{Epoch: 1},
 			Target: &ethpb.Checkpoint{Epoch: 0},
 		},
-		CustodyBit_0Indices: []uint64{0, 1},
+		AttestingIndices: []uint64{0, 1},
 	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err := ssz.HashTreeRoot(att1.Data)
-		Data:       att1.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -674,13 +668,9 @@ func TestProcessAttesterSlashings_AppliesCorrectStatus(t *testing.T) {
 			Source: &ethpb.Checkpoint{Epoch: 0},
 			Target: &ethpb.Checkpoint{Epoch: 0},
 		},
-		CustodyBit_0Indices: []uint64{0, 1},
+		AttestingIndices: []uint64{0, 1},
 	}
-	dataAndCustodyBit = &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err = ssz.HashTreeRoot(att2.Data)
-		Data:       att2.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err = ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -781,7 +771,6 @@ func TestProcessAttestations_NeitherCurrentNorPrevEpoch(t *testing.T) {
 
 func TestProcessAttestations_CurrentEpochFFGDataMismatches(t *testing.T) {
 	aggBits := bitfield.NewBitlist(3)
-	custodyBits := bitfield.NewBitlist(3)
 	attestations := []*ethpb.Attestation{
 		{
 			Data: &ethpb.AttestationData{
@@ -789,7 +778,6 @@ func TestProcessAttestations_CurrentEpochFFGDataMismatches(t *testing.T) {
 				Source: &ethpb.Checkpoint{Epoch: 1},
 			},
 			AggregationBits: aggBits,
-			CustodyBits:     custodyBits,
 		},
 	}
 	block := &ethpb.BeaconBlock{
@@ -829,16 +817,14 @@ func TestProcessAttestations_PrevEpochFFGDataMismatches(t *testing.T) {
 
 	aggBits := bitfield.NewBitlist(3)
 	aggBits.SetBitAt(0, true)
-	custodyBits := bitfield.NewBitlist(3)
 	attestations := []*ethpb.Attestation{
 		{
 			Data: &ethpb.AttestationData{
 				Source: &ethpb.Checkpoint{Epoch: 1},
 				Target: &ethpb.Checkpoint{Epoch: 1},
-				Slot:   1,
+				Slot:   params.BeaconConfig().SlotsPerEpoch,
 			},
 			AggregationBits: aggBits,
-			CustodyBits:     custodyBits,
 		},
 	}
 	block := &ethpb.BeaconBlock{
@@ -878,13 +864,11 @@ func TestProcessAttestations_InvalidAggregationBitsLength(t *testing.T) {
 	beaconState, _ := testutil.DeterministicGenesisState(t, 100)
 
 	aggBits := bitfield.NewBitlist(4)
-	custodyBits := bitfield.NewBitlist(4)
 	att := &ethpb.Attestation{
 		Data: &ethpb.AttestationData{
 			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
 			Target: &ethpb.Checkpoint{Epoch: 0}},
 		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
 	}
 
 	block := &ethpb.BeaconBlock{
@@ -910,14 +894,12 @@ func TestProcessAttestations_OK(t *testing.T) {
 
 	aggBits := bitfield.NewBitlist(3)
 	aggBits.SetBitAt(0, true)
-	custodyBits := bitfield.NewBitlist(3)
 	att := &ethpb.Attestation{
 		Data: &ethpb.AttestationData{
 			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
 			Target: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
 		},
 		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
 	}
 
 	beaconState.CurrentJustifiedCheckpoint.Root = []byte("hello-world")
@@ -931,11 +913,7 @@ func TestProcessAttestations_OK(t *testing.T) {
 	if err != nil {
 		t.Error(err)
 	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err := ssz.HashTreeRoot(att.Data)
-		Data:       att.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -972,11 +950,9 @@ func TestProcessAggregatedAttestation_OverlappingBits(t *testing.T) {
 	aggBits1.SetBitAt(0, true)
 	aggBits1.SetBitAt(1, true)
 	aggBits1.SetBitAt(2, true)
-	custodyBits1 := bitfield.NewBitlist(4)
 	att1 := &ethpb.Attestation{
 		Data:            data,
 		AggregationBits: aggBits1,
-		CustodyBits:     custodyBits1,
 	}
 
 	beaconState.CurrentJustifiedCheckpoint.Root = []byte("hello-world")
@@ -990,11 +966,7 @@ func TestProcessAggregatedAttestation_OverlappingBits(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	dataAndCustodyBit1 := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err := ssz.HashTreeRoot(att1.Data)
-		Data:       att1.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit1)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -1009,11 +981,9 @@ func TestProcessAggregatedAttestation_OverlappingBits(t *testing.T) {
 	aggBits2.SetBitAt(1, true)
 	aggBits2.SetBitAt(2, true)
 	aggBits2.SetBitAt(3, true)
-	custodyBits2 := bitfield.NewBitlist(4)
 	att2 := &ethpb.Attestation{
 		Data:            data,
 		AggregationBits: aggBits2,
-		CustodyBits:     custodyBits2,
 	}
 
 	committee, err = helpers.BeaconCommitteeFromState(beaconState, att2.Data.Slot, att2.Data.CommitteeIndex)
@@ -1024,11 +994,7 @@ func TestProcessAggregatedAttestation_OverlappingBits(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	dataAndCustodyBit2 := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err = ssz.HashTreeRoot(data)
-		Data:       att2.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err = ssz.HashTreeRoot(dataAndCustodyBit2)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -1055,11 +1021,9 @@ func TestProcessAggregatedAttestation_NoOverlappingBits(t *testing.T) {
 	aggBits1 := bitfield.NewBitlist(9)
 	aggBits1.SetBitAt(0, true)
 	aggBits1.SetBitAt(1, true)
-	custodyBits1 := bitfield.NewBitlist(9)
 	att1 := &ethpb.Attestation{
 		Data:            data,
 		AggregationBits: aggBits1,
-		CustodyBits:     custodyBits1,
 	}
 
 	beaconState.CurrentJustifiedCheckpoint.Root = []byte("hello-world")
@@ -1073,11 +1037,7 @@ func TestProcessAggregatedAttestation_NoOverlappingBits(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	dataAndCustodyBit1 := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err := ssz.HashTreeRoot(data)
-		Data:       att1.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit1)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -1091,11 +1051,9 @@ func TestProcessAggregatedAttestation_NoOverlappingBits(t *testing.T) {
 	aggBits2 := bitfield.NewBitlist(9)
 	aggBits2.SetBitAt(2, true)
 	aggBits2.SetBitAt(3, true)
-	custodyBits2 := bitfield.NewBitlist(9)
 	att2 := &ethpb.Attestation{
 		Data:            data,
 		AggregationBits: aggBits2,
-		CustodyBits:     custodyBits2,
 	}
 
 	committee, err = helpers.BeaconCommitteeFromState(beaconState, att2.Data.Slot, att2.Data.CommitteeIndex)
@@ -1106,11 +1064,7 @@ func TestProcessAggregatedAttestation_NoOverlappingBits(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	dataAndCustodyBit2 := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err = ssz.HashTreeRoot(data)
-		Data:       att2.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err = ssz.HashTreeRoot(dataAndCustodyBit2)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -1138,6 +1092,21 @@ func TestProcessAggregatedAttestation_NoOverlappingBits(t *testing.T) {
 	}
 }
 
+func TestProcessAttestationsNoVerify_IncorrectSlotTargetEpoch(t *testing.T) {
+	beaconState, _ := testutil.DeterministicGenesisState(t, 1)
+
+	att := &ethpb.Attestation{
+		Data: &ethpb.AttestationData{
+			Slot:   params.BeaconConfig().SlotsPerEpoch,
+			Target: &ethpb.Checkpoint{},
+		},
+	}
+	wanted := fmt.Sprintf("data slot is not in the same epoch as target %d != %d", helpers.SlotToEpoch(att.Data.Slot), att.Data.Target.Epoch)
+	if _, err := blocks.ProcessAttestationNoVerify(context.TODO(), beaconState, att); err.Error() != wanted {
+		t.Error("Did not get wanted error")
+	}
+}
+
 func TestProcessAttestationsNoVerify_OK(t *testing.T) {
 	// Attestation with an empty signature
 
@@ -1145,14 +1114,12 @@ func TestProcessAttestationsNoVerify_OK(t *testing.T) {
 
 	aggBits := bitfield.NewBitlist(3)
 	aggBits.SetBitAt(1, true)
-	custodyBits := bitfield.NewBitlist(3)
 	att := &ethpb.Attestation{
 		Data: &ethpb.AttestationData{
 			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
 			Target: &ethpb.Checkpoint{Epoch: 0},
 		},
 		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
 	}
 
 	zeroSig := [96]byte{}
@@ -1181,28 +1148,20 @@ func TestConvertToIndexed_OK(t *testing.T) {
 		RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector),
 	}
 	tests := []struct {
-		aggregationBitfield      bitfield.Bitlist
+		aggregationBitfield    bitfield.Bitlist
-		custodyBitfield          bitfield.Bitlist
+		wantedAttestingIndices []uint64
-		wantedCustodyBit0Indices []uint64
-		wantedCustodyBit1Indices []uint64
 	}{
 		{
-			aggregationBitfield:      bitfield.Bitlist{0x07},
+			aggregationBitfield:    bitfield.Bitlist{0x07},
-			custodyBitfield:          bitfield.Bitlist{0x05},
+			wantedAttestingIndices: []uint64{4, 30},
-			wantedCustodyBit0Indices: []uint64{4},
-			wantedCustodyBit1Indices: []uint64{30},
 		},
 		{
-			aggregationBitfield:      bitfield.Bitlist{0x07},
+			aggregationBitfield:    bitfield.Bitlist{0x03},
-			custodyBitfield:          bitfield.Bitlist{0x06},
+			wantedAttestingIndices: []uint64{30},
-			wantedCustodyBit0Indices: []uint64{30},
-			wantedCustodyBit1Indices: []uint64{4},
 		},
 		{
-			aggregationBitfield:      bitfield.Bitlist{0x07},
+			aggregationBitfield:    bitfield.Bitlist{0x01},
-			custodyBitfield:          bitfield.Bitlist{0x07},
+			wantedAttestingIndices: []uint64{},
-			wantedCustodyBit0Indices: []uint64{},
-			wantedCustodyBit1Indices: []uint64{4, 30},
 		},
 	}
 
@@ -1215,12 +1174,10 @@ func TestConvertToIndexed_OK(t *testing.T) {
 	}
 	for _, tt := range tests {
 		attestation.AggregationBits = tt.aggregationBitfield
-		attestation.CustodyBits = tt.custodyBitfield
 		wanted := &ethpb.IndexedAttestation{
-			CustodyBit_0Indices: tt.wantedCustodyBit0Indices,
+			AttestingIndices: tt.wantedAttestingIndices,
-			CustodyBit_1Indices: tt.wantedCustodyBit1Indices,
+			Data:             attestation.Data,
-			Data:                attestation.Data,
+			Signature:        attestation.Signature,
-			Signature:           attestation.Signature,
 		}
 
 		committee, err := helpers.BeaconCommitteeFromState(state, attestation.Data.Slot, attestation.Data.CommitteeIndex)
@@ -1269,7 +1226,7 @@ func TestVerifyIndexedAttestation_OK(t *testing.T) {
 					Epoch: 2,
 				},
 			},
-			CustodyBit_0Indices: []uint64{1},
+			AttestingIndices: []uint64{1},
 		}},
 		{attestation: &ethpb.IndexedAttestation{
 			Data: &ethpb.AttestationData{
@@ -1277,7 +1234,7 @@ func TestVerifyIndexedAttestation_OK(t *testing.T) {
 					Epoch: 1,
 				},
 			},
-			CustodyBit_0Indices: []uint64{47, 99},
+			AttestingIndices: []uint64{47, 99, 101},
 		}},
 		{attestation: &ethpb.IndexedAttestation{
 			Data: &ethpb.AttestationData{
@@ -1285,7 +1242,7 @@ func TestVerifyIndexedAttestation_OK(t *testing.T) {
 					Epoch: 4,
 				},
 			},
-			CustodyBit_0Indices: []uint64{21, 72},
+			AttestingIndices: []uint64{21, 72},
 		}},
 		{attestation: &ethpb.IndexedAttestation{
 			Data: &ethpb.AttestationData{
@@ -1293,25 +1250,20 @@ func TestVerifyIndexedAttestation_OK(t *testing.T) {
 					Epoch: 7,
 				},
 			},
-			CustodyBit_0Indices: []uint64{100, 121},
+			AttestingIndices: []uint64{100, 121, 122},
 		}},
 	}
 
 	for _, tt := range tests {
-		attDataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-			Data:       tt.attestation.Data,
-			CustodyBit: false,
-		}
-
 		domain := helpers.Domain(state.Fork, tt.attestation.Data.Target.Epoch, params.BeaconConfig().DomainBeaconAttester)
 
-		root, err := ssz.HashTreeRoot(attDataAndCustodyBit)
+		root, err := ssz.HashTreeRoot(tt.attestation.Data)
 		if err != nil {
 			t.Errorf("Could not find the ssz root: %v", err)
 			continue
 		}
 		var sig []*bls.Signature
-		for _, idx := range tt.attestation.CustodyBit_0Indices {
+		for _, idx := range tt.attestation.AttestingIndices {
 			validatorSig := keys[idx].Sign(root[:], domain)
 			sig = append(sig, validatorSig)
 		}
@@ -1329,20 +1281,59 @@ func TestVerifyIndexedAttestation_OK(t *testing.T) {
 
 func TestValidateIndexedAttestation_AboveMaxLength(t *testing.T) {
 	indexedAtt1 := &ethpb.IndexedAttestation{
-		CustodyBit_0Indices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+5),
+		AttestingIndices: make([]uint64, params.BeaconConfig().MaxValidatorsPerCommittee+5),
-		CustodyBit_1Indices: []uint64{},
 	}
 
 	for i := uint64(0); i < params.BeaconConfig().MaxValidatorsPerCommittee+5; i++ {
-		indexedAtt1.CustodyBit_0Indices[i] = i
+		indexedAtt1.AttestingIndices[i] = i
 	}
 
-	want := "over max number of allowed indices"
+	want := "validator indices count exceeds MAX_VALIDATORS_PER_COMMITTEE"
 	if err := blocks.VerifyIndexedAttestation(context.Background(), &pb.BeaconState{}, indexedAtt1); !strings.Contains(err.Error(), want) {
 		t.Errorf("Expected verification to fail return false, received: %v", err)
 	}
 }
 
+func TestProcessDeposits_SameValidatorMultipleDepositsSameBlock(t *testing.T) {
+	// Same validator created 3 valid deposits within the same block
+	testutil.ResetCache()
+	dep, _, _ := testutil.DeterministicDepositsAndKeysSameValidator(3)
+	eth1Data, err := testutil.DeterministicEth1Data(len(dep))
+	if err != nil {
+		t.Fatal(err)
+	}
+	block := &ethpb.BeaconBlock{
+		Body: &ethpb.BeaconBlockBody{
+			// 3 deposits from the same validator
+			Deposits: []*ethpb.Deposit{dep[0], dep[1], dep[2]},
+		},
+	}
+	registry := []*ethpb.Validator{
+		{
+			PublicKey:             []byte{1},
+			WithdrawalCredentials: []byte{1, 2, 3},
+		},
+	}
+	balances := []uint64{0}
+	beaconState := &pb.BeaconState{
+		Validators: registry,
+		Balances:   balances,
+		Eth1Data:   eth1Data,
+		Fork: &pb.Fork{
+			PreviousVersion: params.BeaconConfig().GenesisForkVersion,
+			CurrentVersion:  params.BeaconConfig().GenesisForkVersion,
+		},
+	}
+	newState, err := blocks.ProcessDeposits(context.Background(), beaconState, block.Body)
+	if err != nil {
+		t.Fatalf("Expected block deposits to process correctly, received: %v", err)
+	}
+
+	if len(newState.Validators) != 2 {
+		t.Errorf("Incorrect validator count. Wanted %d, got %d", 2, len(newState.Validators))
+	}
+}
+
 func TestProcessDeposits_MerkleBranchFailsVerification(t *testing.T) {
 	deposit := &ethpb.Deposit{
 		Data: &ethpb.Deposit_Data{
@@ -1432,7 +1423,7 @@ func TestProcessDeposits_RepeatedDeposit_IncreasesValidatorBalance(t *testing.T)
 			Amount:    1000,
 		},
 	}
-	sr, err := ssz.SigningRoot(deposit.Data)
+	sr, err := ssz.HashTreeRoot(deposit.Data)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -1590,9 +1581,11 @@ func TestProcessDeposit_SkipsInvalidDeposit(t *testing.T) {
 }
 
 func TestProcessVoluntaryExits_ValidatorNotActive(t *testing.T) {
-	exits := []*ethpb.VoluntaryExit{
+	exits := []*ethpb.SignedVoluntaryExit{
 		{
-			ValidatorIndex: 0,
+			Exit: &ethpb.VoluntaryExit{
+				ValidatorIndex: 0,
+			},
 		},
 	}
 	registry := []*ethpb.Validator{
@@ -1617,9 +1610,11 @@ func TestProcessVoluntaryExits_ValidatorNotActive(t *testing.T) {
 }
 
 func TestProcessVoluntaryExits_InvalidExitEpoch(t *testing.T) {
-	exits := []*ethpb.VoluntaryExit{
+	exits := []*ethpb.SignedVoluntaryExit{
 		{
-			Epoch: 10,
+			Exit: &ethpb.VoluntaryExit{
+				Epoch: 10,
+			},
 		},
 	}
 	registry := []*ethpb.Validator{
@@ -1645,10 +1640,12 @@ func TestProcessVoluntaryExits_InvalidExitEpoch(t *testing.T) {
 }
 
 func TestProcessVoluntaryExits_NotActiveLongEnoughToExit(t *testing.T) {
-	exits := []*ethpb.VoluntaryExit{
+	exits := []*ethpb.SignedVoluntaryExit{
 		{
-			ValidatorIndex: 0,
+			Exit: &ethpb.VoluntaryExit{
-			Epoch:          0,
+				ValidatorIndex: 0,
+				Epoch:          0,
+			},
 		},
 	}
 	registry := []*ethpb.Validator{
@@ -1673,10 +1670,12 @@ func TestProcessVoluntaryExits_NotActiveLongEnoughToExit(t *testing.T) {
 }
 
 func TestProcessVoluntaryExits_AppliesCorrectStatus(t *testing.T) {
-	exits := []*ethpb.VoluntaryExit{
+	exits := []*ethpb.SignedVoluntaryExit{
 		{
-			ValidatorIndex: 0,
+			Exit: &ethpb.VoluntaryExit{
-			Epoch:          0,
+				ValidatorIndex: 0,
+				Epoch:          0,
+			},
 		},
 	}
 	registry := []*ethpb.Validator{
@@ -1697,7 +1696,7 @@ func TestProcessVoluntaryExits_AppliesCorrectStatus(t *testing.T) {
 
 	priv := bls.RandKey()
 	state.Validators[0].PublicKey = priv.PublicKey().Marshal()[:]
-	signingRoot, err := ssz.SigningRoot(exits[0])
+	signingRoot, err := ssz.HashTreeRoot(exits[0].Exit)
 	if err != nil {
 		t.Error(err)
 	}

beacon-chain/core/blocks/block_test.go

@@ -11,11 +11,11 @@ func TestGenesisBlock_InitializedCorrectly(t *testing.T) {
 	stateHash := []byte{0}
 	b1 := blocks.NewGenesisBlock(stateHash)
 
-	if b1.ParentRoot == nil {
+	if b1.Block.ParentRoot == nil {
 		t.Error("genesis block missing ParentHash field")
 	}
 
-	if !bytes.Equal(b1.StateRoot, stateHash) {
+	if !bytes.Equal(b1.Block.StateRoot, stateHash) {
 		t.Error("genesis block StateRootHash32 isn't initialized correctly")
 	}
 }

beacon-chain/core/blocks/spectest/block_header_test.go

@@ -52,7 +52,8 @@ func runBlockHeaderTest(t *testing.T, config string) {
 				t.Fatal(err)
 			}
 
-			beaconState, err := blocks.ProcessBlockHeader(preBeaconState, block)
+			// Spectest blocks are not signed, so we'll call NoVerify to skip sig verification.
+			beaconState, err := blocks.ProcessBlockHeaderNoVerify(preBeaconState, block)
 			if postSSZExists {
 				if err != nil {
 					t.Fatalf("Unexpected error: %v", err)

beacon-chain/core/blocks/spectest/block_processing_test.go

@@ -53,7 +53,7 @@ func runBlockProcessingTest(t *testing.T, config string) {
 				if err != nil {
 					t.Fatal(err)
 				}
-				block := &ethpb.BeaconBlock{}
+				block := &ethpb.SignedBeaconBlock{}
 				if err := ssz.Unmarshal(blockFile, block); err != nil {
 					t.Fatalf("Failed to unmarshal: %v", err)
 				}

beacon-chain/core/blocks/spectest/voluntary_exit_test.go

@@ -24,12 +24,12 @@ func runVoluntaryExitTest(t *testing.T, config string) {
 			if err != nil {
 				t.Fatal(err)
 			}
-			voluntaryExit := &ethpb.VoluntaryExit{}
+			voluntaryExit := &ethpb.SignedVoluntaryExit{}
 			if err := ssz.Unmarshal(exitFile, voluntaryExit); err != nil {
 				t.Fatalf("Failed to unmarshal: %v", err)
 			}
 
-			body := &ethpb.BeaconBlockBody{VoluntaryExits: []*ethpb.VoluntaryExit{voluntaryExit}}
+			body := &ethpb.BeaconBlockBody{VoluntaryExits: []*ethpb.SignedVoluntaryExit{voluntaryExit}}
 			testutil.RunBlockOperationTest(t, folderPath, body, blocks.ProcessVoluntaryExits)
 		})
 	}

beacon-chain/core/epoch/epoch_processing.go

@@ -19,6 +19,21 @@ import (
 	"github.com/prysmaticlabs/prysm/shared/params"
 )
 
+var epochState *pb.BeaconState
+
+// sortableIndices implements the Sort interface to sort newly activated validator indices
+// by activation epoch and by index number.
+type sortableIndices []uint64
+
+func (s sortableIndices) Len() int      { return len(s) }
+func (s sortableIndices) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
+func (s sortableIndices) Less(i, j int) bool {
+	if epochState.Validators[s[i]].ActivationEligibilityEpoch == epochState.Validators[s[j]].ActivationEligibilityEpoch {
+		return s[i] < s[j]
+	}
+	return epochState.Validators[s[i]].ActivationEligibilityEpoch < epochState.Validators[s[j]].ActivationEligibilityEpoch
+}
+
 // MatchedAttestations is an object that contains the correctly
 // voted attestations based on source, target and head criteria.
 type MatchedAttestations struct {
@@ -117,38 +132,33 @@ func AttestingBalance(state *pb.BeaconState, atts []*pb.PendingAttestation) (uin
 // Spec pseudocode definition:
 //   def process_registry_updates(state: BeaconState) -> None:
 //    # Process activation eligibility and ejections
-//    for index, validator in enumerate(state.validator_registry):
+//    for index, validator in enumerate(state.validators):
-//        if (
+//        if is_eligible_for_activation_queue(validator):
-//            validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH and
+//            validator.activation_eligibility_epoch = get_current_epoch(state) + 1
-//            validator.effective_balance >= MAX_EFFECTIVE_BALANCE
-//        ):
-//            validator.activation_eligibility_epoch = get_current_epoch(state)
 //
 //        if is_active_validator(validator, get_current_epoch(state)) and validator.effective_balance <= EJECTION_BALANCE:
-//            initiate_validator_exit(state, index)
+//            initiate_validator_exit(state, ValidatorIndex(index))
 //
-//    # Queue validators eligible for activation and not dequeued for activation prior to finalized epoch
+//    # Queue validators eligible for activation and not yet dequeued for activation
 //    activation_queue = sorted([
-//        index for index, validator in enumerate(state.validator_registry) if
+//        index for index, validator in enumerate(state.validators)
-//        validator.activation_eligibility_epoch != FAR_FUTURE_EPOCH and
+//        if is_eligible_for_activation(state, validator)
-//        validator.activation_epoch >= get_delayed_activation_exit_epoch(state.finalized_epoch)
+//        # Order by the sequence of activation_eligibility_epoch setting and then index
-//    ], key=lambda index: state.validator_registry[index].activation_eligibility_epoch)
+//    ], key=lambda index: (state.validators[index].activation_eligibility_epoch, index))
-//    # Dequeued validators for activation up to churn limit (without resetting activation epoch)
+//    # Dequeued validators for activation up to churn limit
-//    for index in activation_queue[:get_churn_limit(state)]:
+//    for index in activation_queue[:get_validator_churn_limit(state)]:
-//        validator = state.validator_registry[index]
+//        validator = state.validators[index]
-//        if validator.activation_epoch == FAR_FUTURE_EPOCH:
+//        validator.activation_epoch = compute_activation_exit_epoch(get_current_epoch(state))
-//            validator.activation_epoch = get_delayed_activation_exit_epoch(get_current_epoch(state))
 func ProcessRegistryUpdates(state *pb.BeaconState) (*pb.BeaconState, error) {
 	currentEpoch := helpers.CurrentEpoch(state)
 
 	var err error
 	for idx, validator := range state.Validators {
 		// Process the validators for activation eligibility.
-		eligibleToActivate := validator.ActivationEligibilityEpoch == params.BeaconConfig().FarFutureEpoch
+		if helpers.IsEligibleForActivationQueue(validator) {
-		properBalance := validator.EffectiveBalance >= params.BeaconConfig().MaxEffectiveBalance
+			validator.ActivationEligibilityEpoch = helpers.CurrentEpoch(state) + 1
-		if eligibleToActivate && properBalance {
-			validator.ActivationEligibilityEpoch = currentEpoch
 		}
+
 		// Process the validators for ejection.
 		isActive := helpers.IsActiveValidator(validator, currentEpoch)
 		belowEjectionBalance := validator.EffectiveBalance <= params.BeaconConfig().EjectionBalance
@@ -160,18 +170,16 @@ func ProcessRegistryUpdates(state *pb.BeaconState) (*pb.BeaconState, error) {
 		}
 	}
 
-	// Queue the validators whose eligible to activate and sort them by activation eligibility epoch number
+	// Queue validators eligible for activation and not yet dequeued for activation.
 	var activationQ []uint64
 	for idx, validator := range state.Validators {
-		eligibleActivated := validator.ActivationEligibilityEpoch != params.BeaconConfig().FarFutureEpoch
+		if helpers.IsEligibleForActivation(state, validator) {
-		canBeActive := validator.ActivationEpoch >= helpers.DelayedActivationExitEpoch(state.FinalizedCheckpoint.Epoch)
-		if eligibleActivated && canBeActive {
 			activationQ = append(activationQ, uint64(idx))
 		}
 	}
-	sort.Slice(activationQ, func(i, j int) bool {
+
-		return state.Validators[i].ActivationEligibilityEpoch < state.Validators[j].ActivationEligibilityEpoch
+	epochState = state
-	})
+	sort.Sort(sortableIndices(activationQ))
 
 	// Only activate just enough validators according to the activation churn limit.
 	limit := len(activationQ)
@@ -189,12 +197,12 @@ func ProcessRegistryUpdates(state *pb.BeaconState) (*pb.BeaconState, error) {
 	if int(churnLimit) < limit {
 		limit = int(churnLimit)
 	}
+
 	for _, index := range activationQ[:limit] {
 		validator := state.Validators[index]
-		if validator.ActivationEpoch == params.BeaconConfig().FarFutureEpoch {
+		validator.ActivationEpoch = helpers.DelayedActivationExitEpoch(currentEpoch)
-			validator.ActivationEpoch = helpers.DelayedActivationExitEpoch(currentEpoch)
-		}
 	}
+
 	return state, nil
 }
 

beacon-chain/core/epoch/epoch_processing_test.go

@@ -497,7 +497,7 @@ func TestProcessRegistryUpdates_NoRotation(t *testing.T) {
 func TestProcessRegistryUpdates_EligibleToActivate(t *testing.T) {
 	state := &pb.BeaconState{
 		Slot:                5 * params.BeaconConfig().SlotsPerEpoch,
-		FinalizedCheckpoint: &ethpb.Checkpoint{},
+		FinalizedCheckpoint: &ethpb.Checkpoint{Epoch: 6},
 	}
 	limit, err := helpers.ValidatorChurnLimit(0)
 	if err != nil {
@@ -516,7 +516,7 @@ func TestProcessRegistryUpdates_EligibleToActivate(t *testing.T) {
 		t.Error(err)
 	}
 	for i, validator := range newState.Validators {
-		if validator.ActivationEligibilityEpoch != currentEpoch {
+		if validator.ActivationEligibilityEpoch != currentEpoch+1 {
 			t.Errorf("Could not update registry %d, wanted activation eligibility epoch %d got %d",
 				i, currentEpoch, validator.ActivationEligibilityEpoch)
 		}

beacon-chain/core/exit/BUILD.bazel

@@ -0,0 +1,37 @@
+load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["validation.go"],
+    importpath = "github.com/prysmaticlabs/prysm/beacon-chain/core/exit",
+    visibility = [
+        "//beacon-chain:__subpackages__",
+    ],
+    deps = [
+        "//beacon-chain/core/helpers:go_default_library",
+        "//proto/beacon/p2p/v1:go_default_library",
+        "//shared/bls:go_default_library",
+        "//shared/mathutil:go_default_library",
+        "//shared/params:go_default_library",
+        "//shared/roughtime:go_default_library",
+        "@com_github_pkg_errors//:go_default_library",
+        "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
+        "@com_github_prysmaticlabs_go_ssz//:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["validation_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//beacon-chain/blockchain/testing:go_default_library",
+        "//beacon-chain/core/blocks:go_default_library",
+        "//beacon-chain/core/state:go_default_library",
+        "//beacon-chain/db/testing:go_default_library",
+        "//shared/params:go_default_library",
+        "//shared/testutil:go_default_library",
+        "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
+        "@com_github_prysmaticlabs_go_ssz//:go_default_library",
+    ],
+)

beacon-chain/core/exit/validation.go

@@ -0,0 +1,66 @@
+package exit
+
+import (
+	"fmt"
+	"time"
+
+	"github.com/pkg/errors"
+	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	"github.com/prysmaticlabs/go-ssz"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
+	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
+	"github.com/prysmaticlabs/prysm/shared/bls"
+	"github.com/prysmaticlabs/prysm/shared/mathutil"
+	"github.com/prysmaticlabs/prysm/shared/params"
+	"github.com/prysmaticlabs/prysm/shared/roughtime"
+)
+
+// ValidateVoluntaryExit validates the voluntary exit.
+// If it is invalid for some reason an error, if valid it will return no error.
+func ValidateVoluntaryExit(state *pb.BeaconState, genesisTime time.Time, signed *ethpb.SignedVoluntaryExit) error {
+	if signed == nil || signed.Exit == nil {
+		return errors.New("nil signed voluntary exit")
+	}
+	ve := signed.Exit
+	if ve.ValidatorIndex >= uint64(len(state.Validators)) {
+		return fmt.Errorf("unknown validator index %d", ve.ValidatorIndex)
+	}
+	validator := state.Validators[ve.ValidatorIndex]
+
+	if !helpers.IsActiveValidator(validator, ve.Epoch) {
+		return fmt.Errorf("validator %d not active at epoch %d", ve.ValidatorIndex, ve.Epoch)
+	}
+	if validator.ExitEpoch != params.BeaconConfig().FarFutureEpoch {
+		return fmt.Errorf("validator %d already exiting or exited", ve.ValidatorIndex)
+	}
+
+	secondsPerEpoch := params.BeaconConfig().SecondsPerSlot * params.BeaconConfig().SlotsPerEpoch
+	currentEpoch := uint64(roughtime.Now().Unix()-genesisTime.Unix()) / secondsPerEpoch
+	earliestRequestedExitEpoch := mathutil.Max(ve.Epoch, currentEpoch)
+	earliestExitEpoch := validator.ActivationEpoch + params.BeaconConfig().PersistentCommitteePeriod
+	if earliestRequestedExitEpoch < earliestExitEpoch {
+		return fmt.Errorf("validator %d cannot exit before epoch %d", ve.ValidatorIndex, earliestExitEpoch)
+	}
+
+	// Confirm signature is valid
+	root, err := ssz.HashTreeRoot(ve)
+	if err != nil {
+		return errors.Wrap(err, "cannot confirm signature")
+	}
+	sig, err := bls.SignatureFromBytes(signed.Signature)
+	if err != nil {
+		return errors.Wrap(err, "malformed signature")
+	}
+	validatorPubKey, err := bls.PublicKeyFromBytes(validator.PublicKey)
+	if err != nil {
+		return errors.Wrap(err, "invalid validator public key")
+	}
+	domain := bls.ComputeDomain(params.BeaconConfig().DomainVoluntaryExit)
+	verified := sig.Verify(root[:], validatorPubKey, domain)
+	if !verified {
+		return errors.New("incorrect signature")
+	}
+
+	// Parameters are valid.
+	return nil
+}

beacon-chain/core/exit/validation_test.go

@@ -0,0 +1,125 @@
+package exit_test
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+
+	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	"github.com/prysmaticlabs/go-ssz"
+	mockChain "github.com/prysmaticlabs/prysm/beacon-chain/blockchain/testing"
+	blk "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/exit"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
+	dbutil "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
+	"github.com/prysmaticlabs/prysm/shared/params"
+	"github.com/prysmaticlabs/prysm/shared/testutil"
+)
+
+// Set genesis to a small set for faster test processing.
+func init() {
+	p := params.BeaconConfig()
+	p.MinGenesisActiveValidatorCount = 8
+	params.OverrideBeaconConfig(p)
+}
+
+func TestValidation(t *testing.T) {
+	tests := []struct {
+		name           string
+		epoch          uint64
+		validatorIndex uint64
+		signature      []byte
+		err            error
+	}{
+		{
+			name:           "MissingValidator",
+			epoch:          2048,
+			validatorIndex: 16,
+			err:            errors.New("unknown validator index 16"),
+		},
+		{
+			name:           "EarlyExit",
+			epoch:          2047,
+			validatorIndex: 0,
+			err:            errors.New("validator 0 cannot exit before epoch 2048"),
+		},
+		{
+			name:           "NoSignature",
+			epoch:          2048,
+			validatorIndex: 0,
+			err:            errors.New("malformed signature: signature must be 96 bytes"),
+		},
+		{
+			name:           "InvalidSignature",
+			epoch:          2048,
+			validatorIndex: 0,
+			signature:      []byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
+			err:            errors.New("malformed signature: could not unmarshal bytes into signature: err blsSignatureDeserialize 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"),
+		},
+		{
+			name:           "IncorrectSignature",
+			epoch:          2048,
+			validatorIndex: 0,
+			signature:      []byte{0xab, 0xb0, 0x12, 0x4c, 0x75, 0x74, 0xf2, 0x81, 0xa2, 0x93, 0xf4, 0x18, 0x5c, 0xad, 0x3c, 0xb2, 0x26, 0x81, 0xd5, 0x20, 0x91, 0x7c, 0xe4, 0x66, 0x65, 0x24, 0x3e, 0xac, 0xb0, 0x51, 0x00, 0x0d, 0x8b, 0xac, 0xf7, 0x5e, 0x14, 0x51, 0x87, 0x0c, 0xa6, 0xb3, 0xb9, 0xe6, 0xc9, 0xd4, 0x1a, 0x7b, 0x02, 0xea, 0xd2, 0x68, 0x5a, 0x84, 0x18, 0x8a, 0x4f, 0xaf, 0xd3, 0x82, 0x5d, 0xaf, 0x6a, 0x98, 0x96, 0x25, 0xd7, 0x19, 0xcc, 0xd2, 0xd8, 0x3a, 0x40, 0x10, 0x1f, 0x4a, 0x45, 0x3f, 0xca, 0x62, 0x87, 0x8c, 0x89, 0x0e, 0xca, 0x62, 0x23, 0x63, 0xf9, 0xdd, 0xb8, 0xf3, 0x67, 0xa9, 0x1e, 0x84},
+			err:            errors.New("incorrect signature"),
+		},
+		{
+			name:           "Good",
+			epoch:          2048,
+			validatorIndex: 0,
+			signature:      []byte{0xb3, 0xe1, 0x9d, 0xc6, 0x7c, 0x78, 0x6c, 0xcf, 0x33, 0x1d, 0xb9, 0x6f, 0x59, 0x64, 0x44, 0xe1, 0x29, 0xd0, 0x87, 0x03, 0x26, 0x6e, 0x49, 0x1c, 0x05, 0xae, 0x16, 0x7b, 0x04, 0x0f, 0x3f, 0xf8, 0x82, 0x77, 0x60, 0xfc, 0xcf, 0x2f, 0x59, 0xc7, 0x40, 0x0b, 0x2c, 0xa9, 0x23, 0x8a, 0x6c, 0x8d, 0x01, 0x21, 0x5e, 0xa8, 0xac, 0x36, 0x70, 0x31, 0xb0, 0xe1, 0xa8, 0xb8, 0x8f, 0x93, 0x8c, 0x1c, 0xa2, 0x86, 0xe7, 0x22, 0x00, 0x6a, 0x7d, 0x36, 0xc0, 0x2b, 0x86, 0x2c, 0xf5, 0xf9, 0x10, 0xb9, 0xf2, 0xbd, 0x5e, 0xa6, 0x5f, 0x12, 0x86, 0x43, 0x20, 0x4d, 0xa2, 0x9d, 0x8b, 0xe6, 0x6f, 0x09},
+		},
+	}
+
+	db := dbutil.SetupDB(t)
+	defer dbutil.TeardownDB(t, db)
+	ctx := context.Background()
+	deposits, _, _ := testutil.DeterministicDepositsAndKeys(params.BeaconConfig().MinGenesisActiveValidatorCount)
+	beaconState, err := state.GenesisBeaconState(deposits, 0, &ethpb.Eth1Data{BlockHash: make([]byte, 32)})
+	if err != nil {
+		t.Fatal(err)
+	}
+	block := blk.NewGenesisBlock([]byte{})
+	if err := db.SaveBlock(ctx, block); err != nil {
+		t.Fatalf("Could not save genesis block: %v", err)
+	}
+	genesisRoot, err := ssz.HashTreeRoot(block.Block)
+	if err != nil {
+		t.Fatalf("Could not get signing root %v", err)
+	}
+
+	// Set genesis time to be 100 epochs ago
+	genesisTime := time.Now().Add(time.Duration(-100*int64(params.BeaconConfig().SecondsPerSlot*params.BeaconConfig().SlotsPerEpoch)) * time.Second)
+	mockChainService := &mockChain.ChainService{State: beaconState, Root: genesisRoot[:], Genesis: genesisTime}
+	headState, err := mockChainService.HeadState(context.Background())
+	if err != nil {
+		t.Fatal("Failed to obtain head state")
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			req := &ethpb.SignedVoluntaryExit{
+				Exit: &ethpb.VoluntaryExit{
+					Epoch:          test.epoch,
+					ValidatorIndex: test.validatorIndex,
+				},
+				Signature: test.signature,
+			}
+
+			err := exit.ValidateVoluntaryExit(headState, genesisTime, req)
+			if test.err == nil {
+				if err != nil {
+					t.Errorf("Unexpected error: received %v", err)
+				}
+			} else {
+				if err == nil {
+					t.Error("Failed to receive expected error")
+				}
+				if err.Error() != test.err.Error() {
+					t.Errorf("Unexpected error: expected %s, received %s", test.err.Error(), err.Error())
+				}
+			}
+		})
+	}
+}

beacon-chain/core/feed/operation/BUILD.bazel

@@ -9,7 +9,6 @@ go_library(
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/core/feed/operation",
     visibility = ["//beacon-chain:__subpackages__"],
     deps = [
-        "//proto/beacon/p2p/v1:go_default_library",
         "//shared/event:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
     ],

beacon-chain/core/feed/operation/events.go

@@ -2,7 +2,6 @@ package operation
 
 import (
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 )
 
 const (
@@ -27,11 +26,11 @@ type UnAggregatedAttReceivedData struct {
 // AggregatedAttReceivedData is the data sent with AggregatedAttReceived events.
 type AggregatedAttReceivedData struct {
 	// Attestation is the aggregated attestation object.
-	Attestation *pb.AggregateAndProof
+	Attestation *ethpb.AggregateAttestationAndProof
 }
 
-// ExitRecievedData is the data sent with ExitReceived events.
+// ExitReceivedData is the data sent with ExitReceived events.
-type ExitRecievedData struct {
+type ExitReceivedData struct {
 	// Exit is the voluntary exit object.
-	Exit *ethpb.VoluntaryExit
+	Exit *ethpb.SignedVoluntaryExit
 }

beacon-chain/core/helpers/attestation.go

@@ -109,7 +109,7 @@ func AggregateAttestation(a1 *ethpb.Attestation, a2 *ethpb.Attestation) (*ethpb.
 // SlotSignature returns the signed signature of the hash tree root of input slot.
 //
 // Spec pseudocode definition:
-//   def slot_signature(state: BeaconState, slot: Slot, privkey: int) -> BLSSignature:
+//   def get_slot_signature(state: BeaconState, slot: Slot, privkey: int) -> BLSSignature:
 //    domain = get_domain(state, DOMAIN_BEACON_ATTESTER, compute_epoch_at_slot(slot))
 //    return bls_sign(privkey, hash_tree_root(slot), domain)
 func SlotSignature(state *pb.BeaconState, slot uint64, privKey *bls.SecretKey) (*bls.Signature, error) {
@@ -157,3 +157,9 @@ func AggregateSignature(attestations []*ethpb.Attestation) (*bls.Signature, erro
 	}
 	return aggregateSignatures(sigs), nil
 }
+
+// IsAggregated returns true if the attestation is an aggregated attestation,
+// false otherwise.
+func IsAggregated(attestation *ethpb.Attestation) bool {
+	return attestation.AggregationBits.Count() > 1
+}

beacon-chain/core/helpers/attestation_aggregation_bench_test.go

@@ -89,7 +89,6 @@ func BenchmarkAggregateAttestations(b *testing.B) {
 			atts[i] = &ethpb.Attestation{
 				AggregationBits: b,
 				Data:            nil,
-				CustodyBits:     nil,
 				Signature:       bls.NewAggregateSignature().Marshal(),
 			}
 		}

beacon-chain/core/helpers/attestation_test.go

@@ -176,7 +176,6 @@ func TestAggregateAttestations(t *testing.T) {
 			atts[i] = &ethpb.Attestation{
 				AggregationBits: b,
 				Data:            nil,
-				CustodyBits:     nil,
 				Signature:       sig.Marshal(),
 			}
 		}

beacon-chain/core/helpers/committee.go

@@ -356,7 +356,7 @@ func VerifyBitfieldLength(bf bitfield.Bitfield, committeeSize uint64) error {
 	return nil
 }
 
-// VerifyAttestationBitfieldLengths verifies that an attestations aggregation and custody bitfields are
+// VerifyAttestationBitfieldLengths verifies that an attestations aggregation bitfields is
 // a valid length matching the size of the committee.
 func VerifyAttestationBitfieldLengths(state *pb.BeaconState, att *ethpb.Attestation) error {
 	committee, err := BeaconCommitteeFromState(state, att.Data.Slot, att.Data.CommitteeIndex)
@@ -371,9 +371,6 @@ func VerifyAttestationBitfieldLengths(state *pb.BeaconState, att *ethpb.Attestat
 	if err := VerifyBitfieldLength(att.AggregationBits, uint64(len(committee))); err != nil {
 		return errors.Wrap(err, "failed to verify aggregation bitfield")
 	}
-	if err := VerifyBitfieldLength(att.CustodyBits, uint64(len(committee))); err != nil {
-		return errors.Wrap(err, "failed to verify custody bitfield")
-	}
 	return nil
 }
 

beacon-chain/core/helpers/committee_test.go

@@ -481,13 +481,11 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 	tests := []struct {
 		attestation         *ethpb.Attestation
 		stateSlot           uint64
-		invalidCustodyBits  bool
 		verificationFailure bool
 	}{
 		{
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0x05},
-				CustodyBits:     bitfield.Bitlist{0x05},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 5,
 					Target:         &ethpb.Checkpoint{},
@@ -499,7 +497,6 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0x06},
-				CustodyBits:     bitfield.Bitlist{0x06},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 10,
 					Target:         &ethpb.Checkpoint{},
@@ -510,7 +507,6 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 		{
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0x06},
-				CustodyBits:     bitfield.Bitlist{0x06},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 20,
 					Target:         &ethpb.Checkpoint{},
@@ -521,20 +517,16 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 		{
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0x06},
-				CustodyBits:     bitfield.Bitlist{0x10},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 20,
 					Target:         &ethpb.Checkpoint{},
 				},
 			},
-			stateSlot:           20,
+			stateSlot: 20,
-			verificationFailure: true,
-			invalidCustodyBits:  true,
 		},
 		{
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0xFF, 0xC0, 0x01},
-				CustodyBits:     bitfield.Bitlist{0xFF, 0xC0, 0x01},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 5,
 					Target:         &ethpb.Checkpoint{},
@@ -546,7 +538,6 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 		{
 			attestation: &ethpb.Attestation{
 				AggregationBits: bitfield.Bitlist{0xFF, 0x01},
-				CustodyBits:     bitfield.Bitlist{0xFF, 0x01},
 				Data: &ethpb.AttestationData{
 					CommitteeIndex: 20,
 					Target:         &ethpb.Checkpoint{},
@@ -561,11 +552,6 @@ func TestVerifyAttestationBitfieldLengths_OK(t *testing.T) {
 		state.Slot = tt.stateSlot
 		err := VerifyAttestationBitfieldLengths(state, tt.attestation)
 		if tt.verificationFailure {
-			if tt.invalidCustodyBits {
-				if !strings.Contains(err.Error(), "custody bitfield") {
-					t.Errorf("%d expected custody bits to fail: %v", i, err)
-				}
-			}
 			if err == nil {
 				t.Error("verification succeeded when it was supposed to fail")
 			}

beacon-chain/core/helpers/slot_epoch.go

@@ -2,6 +2,7 @@ package helpers
 
 import (
 	"fmt"
+	"time"
 
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/params"
@@ -83,6 +84,11 @@ func IsEpochEnd(slot uint64) bool {
 	return IsEpochStart(slot + 1)
 }
 
+// SlotsSinceEpochStarts returns number of slots since the start of the epoch.
+func SlotsSinceEpochStarts(slot uint64) uint64 {
+	return slot - StartSlot(SlotToEpoch(slot))
+}
+
 // Allow for slots "from the future" within a certain tolerance.
 const timeShiftTolerance = 10 // ms
 
@@ -95,3 +101,8 @@ func VerifySlotTime(genesisTime uint64, slot uint64) error {
 	}
 	return nil
 }
+
+// SlotsSince computes the number of time slots that have occurred since the given timestamp.
+func SlotsSince(time time.Time) uint64 {
+	return uint64(roughtime.Since(time).Seconds()) / params.BeaconConfig().SecondsPerSlot
+}

beacon-chain/core/helpers/slot_epoch_test.go

@@ -156,3 +156,21 @@ func TestIsEpochEnd(t *testing.T) {
 		}
 	}
 }
+
+func TestSlotsSinceEpochStarts(t *testing.T) {
+	tests := []struct {
+		slots       uint64
+		wantedSlots uint64
+	}{
+		{slots: 0, wantedSlots: 0},
+		{slots: 1, wantedSlots: 1},
+		{slots: params.BeaconConfig().SlotsPerEpoch - 1, wantedSlots: params.BeaconConfig().SlotsPerEpoch - 1},
+		{slots: params.BeaconConfig().SlotsPerEpoch + 1, wantedSlots: 1},
+		{slots: 10*params.BeaconConfig().SlotsPerEpoch + 2, wantedSlots: 2},
+	}
+	for _, tt := range tests {
+		if got := SlotsSinceEpochStarts(tt.slots); got != tt.wantedSlots {
+			t.Errorf("SlotsSinceEpochStarts() = %v, want %v", got, tt.wantedSlots)
+		}
+	}
+}

beacon-chain/core/helpers/validators.go

@@ -158,11 +158,14 @@ func BeaconProposerIndex(state *pb.BeaconState) (uint64, error) {
 		return 0, errors.Wrap(err, "could not get active indices")
 	}
 
-	return ComputeProposerIndex(state, indices, seedWithSlotHash)
+	return ComputeProposerIndex(state.Validators, indices, seedWithSlotHash)
 }
 
 // ComputeProposerIndex returns the index sampled by effective balance, which is used to calculate proposer.
 //
+// Note: This method signature deviates slightly from the spec recommended definition. The full
+// state object is not required to compute the proposer index.
+//
 // Spec pseudocode definition:
 //  def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Hash) -> ValidatorIndex:
 //    """
@@ -178,21 +181,29 @@ func BeaconProposerIndex(state *pb.BeaconState) (uint64, error) {
 //        if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
 //            return ValidatorIndex(candidate_index)
 //        i += 1
-func ComputeProposerIndex(state *pb.BeaconState, indices []uint64, seed [32]byte) (uint64, error) {
+func ComputeProposerIndex(validators []*ethpb.Validator, activeIndices []uint64, seed [32]byte) (uint64, error) {
-	length := uint64(len(indices))
+	length := uint64(len(activeIndices))
 	if length == 0 {
-		return 0, errors.New("empty indices list")
+		return 0, errors.New("empty active indices list")
 	}
 	maxRandomByte := uint64(1<<8 - 1)
 
 	for i := uint64(0); ; i++ {
-		candidateIndex, err := ComputeShuffledIndex(i%length, length, seed, true)
+		candidateIndex, err := ComputeShuffledIndex(i%length, length, seed, true /* shuffle */)
 		if err != nil {
 			return 0, err
 		}
+		candidateIndex = activeIndices[candidateIndex]
+		if int(candidateIndex) >= len(validators) {
+			return 0, errors.New("active index out of range")
+		}
 		b := append(seed[:], bytesutil.Bytes8(i/32)...)
 		randomByte := hashutil.Hash(b)[i%32]
-		effectiveBal := state.Validators[candidateIndex].EffectiveBalance
+		v := validators[candidateIndex]
+		var effectiveBal uint64
+		if v != nil {
+			effectiveBal = v.EffectiveBalance
+		}
 		if effectiveBal*maxRandomByte >= params.BeaconConfig().MaxEffectiveBalance*uint64(randomByte) {
 			return candidateIndex, nil
 		}
@@ -220,3 +231,38 @@ func Domain(fork *pb.Fork, epoch uint64, domainType []byte) uint64 {
 	}
 	return bls.Domain(domainType, forkVersion)
 }
+
+// IsEligibleForActivationQueue checks if the validator is eligible to
+// be places into the activation queue.
+//
+// Spec pseudocode definition:
+//  def is_eligible_for_activation_queue(validator: Validator) -> bool:
+//    """
+//    Check if ``validator`` is eligible to be placed into the activation queue.
+//    """
+//    return (
+//        validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH
+//        and validator.effective_balance == MAX_EFFECTIVE_BALANCE
+//    )
+func IsEligibleForActivationQueue(validator *ethpb.Validator) bool {
+	return validator.ActivationEligibilityEpoch == params.BeaconConfig().FarFutureEpoch &&
+		validator.EffectiveBalance == params.BeaconConfig().MaxEffectiveBalance
+}
+
+// IsEligibleForActivation checks if the validator is eligible for activation.
+//
+// Spec pseudocode definition:
+//  def is_eligible_for_activation(state: BeaconState, validator: Validator) -> bool:
+//    """
+//    Check if ``validator`` is eligible for activation.
+//    """
+//    return (
+//        # Placement in queue is finalized
+//        validator.activation_eligibility_epoch <= state.finalized_checkpoint.epoch
+//        # Has not yet been activated
+//        and validator.activation_epoch == FAR_FUTURE_EPOCH
+//    )
+func IsEligibleForActivation(state *pb.BeaconState, validator *ethpb.Validator) bool {
+	return validator.ActivationEligibilityEpoch <= state.FinalizedCheckpoint.Epoch &&
+		validator.ActivationEpoch == params.BeaconConfig().FarFutureEpoch
+}

beacon-chain/core/helpers/validators_test.go

@@ -403,3 +403,180 @@ func TestActiveValidatorIndices(t *testing.T) {
 		})
 	}
 }
+
+func TestComputeProposerIndex(t *testing.T) {
+	seed := bytesutil.ToBytes32([]byte("seed"))
+	type args struct {
+		validators []*ethpb.Validator
+		indices    []uint64
+		seed       [32]byte
+	}
+	tests := []struct {
+		name    string
+		args    args
+		want    uint64
+		wantErr bool
+	}{
+		{
+			name: "all_active_indices",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{0, 1, 2, 3, 4},
+				seed:    seed,
+			},
+			want: 2,
+		},
+		{ // Regression test for https://github.com/prysmaticlabs/prysm/issues/4259.
+			name: "1_active_index",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{3},
+				seed:    seed,
+			},
+			want: 3,
+		},
+		{
+			name: "empty_active_indices",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{},
+				seed:    seed,
+			},
+			wantErr: true,
+		},
+		{
+			name: "active_indices_out_of_range",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{100},
+				seed:    seed,
+			},
+			wantErr: true,
+		},
+		{
+			name: "second_half_active",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{5, 6, 7, 8, 9},
+				seed:    seed,
+			},
+			want: 7,
+		},
+		{
+			name: "nil_validator",
+			args: args{
+				validators: []*ethpb.Validator{
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					nil, // Should never happen, but would cause a panic when it does happen.
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+					&ethpb.Validator{EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+				},
+				indices: []uint64{0, 1, 2, 3, 4},
+				seed:    seed,
+			},
+			want: 4,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got, err := ComputeProposerIndex(tt.args.validators, tt.args.indices, tt.args.seed)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("ComputeProposerIndex() error = %v, wantErr %v", err, tt.wantErr)
+				return
+			}
+			if got != tt.want {
+				t.Errorf("ComputeProposerIndex() got = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestIsEligibleForActivationQueue(t *testing.T) {
+	tests := []struct {
+		name      string
+		validator *ethpb.Validator
+		want      bool
+	}{
+		{"Eligible",
+			&ethpb.Validator{ActivationEligibilityEpoch: params.BeaconConfig().FarFutureEpoch, EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+			true},
+		{"Incorrect activation eligibility epoch",
+			&ethpb.Validator{ActivationEligibilityEpoch: 1, EffectiveBalance: params.BeaconConfig().MaxEffectiveBalance},
+			false},
+		{"Not enough balance",
+			&ethpb.Validator{ActivationEligibilityEpoch: params.BeaconConfig().FarFutureEpoch, EffectiveBalance: 1},
+			false},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := IsEligibleForActivationQueue(tt.validator); got != tt.want {
+				t.Errorf("IsEligibleForActivationQueue() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestIsIsEligibleForActivation(t *testing.T) {
+	tests := []struct {
+		name      string
+		validator *ethpb.Validator
+		state     *pb.BeaconState
+		want      bool
+	}{
+		{"Eligible",
+			&ethpb.Validator{ActivationEligibilityEpoch: 1, ActivationEpoch: params.BeaconConfig().FarFutureEpoch},
+			&pb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Epoch: 2}},
+			true},
+		{"Not yet finalized",
+			&ethpb.Validator{ActivationEligibilityEpoch: 1, ActivationEpoch: params.BeaconConfig().FarFutureEpoch},
+			&pb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{}},
+			false},
+		{"Incorrect activation epoch",
+			&ethpb.Validator{ActivationEligibilityEpoch: 1},
+			&pb.BeaconState{FinalizedCheckpoint: &ethpb.Checkpoint{Epoch: 2}},
+			false},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := IsEligibleForActivation(tt.state, tt.validator); got != tt.want {
+				t.Errorf("IsEligibleForActivation() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}

beacon-chain/core/state/benchmarks/benchmark_files/generate_bench_files.go

@@ -94,14 +94,14 @@ func generateMarshalledFullStateAndBlock() error {
 	if err != nil {
 		return err
 	}
-	block.Body.Attestations = append(atts, block.Body.Attestations...)
+	block.Block.Body.Attestations = append(atts, block.Block.Body.Attestations...)
 
 	s, err := state.CalculateStateRoot(context.Background(), beaconState, block)
 	if err != nil {
 		return err
 	}
-	block.StateRoot = s[:]
+	block.Block.StateRoot = s[:]
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		return err
 	}

beacon-chain/core/state/benchmarks/benchmarks_test.go

@@ -20,6 +20,7 @@ import (
 var runAmount = 25
 
 func TestBenchmarkExecuteStateTransition(t *testing.T) {
+	t.Skip("TODO(4098): Regenerate test data with v0.9.2 spec")
 	SetConfig()
 	beaconState, err := beaconState1Epoch()
 	if err != nil {
@@ -204,7 +205,7 @@ func beaconState2FullEpochs() (*pb.BeaconState, error) {
 	return beaconState, nil
 }
 
-func fullBlock() (*ethpb.BeaconBlock, error) {
+func fullBlock() (*ethpb.SignedBeaconBlock, error) {
 	path, err := bazel.Runfile(FullBlockFileName)
 	if err != nil {
 		return nil, err
@@ -213,7 +214,7 @@ func fullBlock() (*ethpb.BeaconBlock, error) {
 	if err != nil {
 		return nil, err
 	}
-	beaconBlock := &ethpb.BeaconBlock{}
+	beaconBlock := &ethpb.SignedBeaconBlock{}
 	if err := ssz.Unmarshal(blockBytes, beaconBlock); err != nil {
 		return nil, err
 	}

beacon-chain/core/state/interop/write_block_to_disk.go

@@ -12,12 +12,12 @@ import (
 )
 
 // WriteBlockToDisk as a block ssz. Writes to temp directory. Debug!
-func WriteBlockToDisk(block *ethpb.BeaconBlock, failed bool) {
+func WriteBlockToDisk(block *ethpb.SignedBeaconBlock, failed bool) {
 	if !featureconfig.Get().WriteSSZStateTransitions {
 		return
 	}
 
-	filename := fmt.Sprintf("beacon_block_%d.ssz", block.Slot)
+	filename := fmt.Sprintf("beacon_block_%d.ssz", block.Block.Slot)
 	if failed {
 		filename = "failed_" + filename
 	}

beacon-chain/core/state/state.go

@@ -4,12 +4,13 @@
 package state
 
 import (
+	"context"
+
 	"github.com/pkg/errors"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-ssz"
 	b "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
-	"github.com/prysmaticlabs/prysm/shared/mathutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/trieutil"
 )
@@ -51,10 +52,54 @@ import (
 //	    state.active_index_roots[index] = active_index_root
 //	    state.compact_committees_roots[index] = committee_root
 //	  return state
+// This method differs from the spec so as to process deposits beforehand instead of the end of the function.
 func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data *ethpb.Eth1Data) (*pb.BeaconState, error) {
 	if eth1Data == nil {
 		return nil, errors.New("no eth1data provided for genesis state")
 	}
+	state := EmptyGenesisState()
+	state.Eth1Data = eth1Data
+	var err error
+	// Process initial deposits.
+	validatorMap := make(map[[48]byte]int)
+	leaves := [][]byte{}
+	for _, deposit := range deposits {
+		hash, err := ssz.HashTreeRoot(deposit.Data)
+		if err != nil {
+			return nil, err
+		}
+		leaves = append(leaves, hash[:])
+	}
+	var trie *trieutil.SparseMerkleTrie
+	if len(leaves) > 0 {
+		trie, err = trieutil.GenerateTrieFromItems(leaves, int(params.BeaconConfig().DepositContractTreeDepth))
+		if err != nil {
+			return nil, err
+		}
+	} else {
+		trie, err = trieutil.NewTrie(int(params.BeaconConfig().DepositContractTreeDepth))
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	depositRoot := trie.Root()
+	state.Eth1Data.DepositRoot = depositRoot[:]
+	for i, deposit := range deposits {
+		state, err = b.ProcessPreGenesisDeposit(context.Background(), state, deposit, validatorMap)
+		if err != nil {
+			return nil, errors.Wrapf(err, "could not process validator deposit %d", i)
+		}
+	}
+	return OptimizedGenesisBeaconState(genesisTime, state, state.Eth1Data)
+}
+
+// OptimizedGenesisBeaconState is used to create a state that has already processed deposits. This is to efficiently
+// create a mainnet state at chainstart.
+func OptimizedGenesisBeaconState(genesisTime uint64, bState *pb.BeaconState, eth1Data *ethpb.Eth1Data) (*pb.BeaconState, error) {
+	if eth1Data == nil {
+		return nil, errors.New("no eth1data provided for genesis state")
+	}
 
 	randaoMixes := make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector)
 	for i := 0; i < len(randaoMixes); i++ {
@@ -82,8 +127,6 @@ func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data
 
 	slashings := make([]uint64, params.BeaconConfig().EpochsPerSlashingsVector)
 
-	eth1Data.DepositCount = uint64(len(deposits))
-
 	state := &pb.BeaconState{
 		// Misc fields.
 		Slot:        0,
@@ -96,8 +139,8 @@ func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data
 		},
 
 		// Validator registry fields.
-		Validators: []*ethpb.Validator{},
+		Validators: bState.Validators,
-		Balances:   []uint64{},
+		Balances:   bState.Balances,
 
 		// Randomness and committees.
 		RandaoMixes: randaoMixes,
@@ -127,7 +170,7 @@ func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data
 		// Eth1 data.
 		Eth1Data:         eth1Data,
 		Eth1DataVotes:    []*ethpb.Eth1Data{},
-		Eth1DepositIndex: 0,
+		Eth1DepositIndex: bState.Eth1DepositIndex,
 	}
 
 	bodyRoot, err := ssz.HashTreeRoot(&ethpb.BeaconBlockBody{})
@@ -139,54 +182,38 @@ func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data
 		ParentRoot: zeroHash,
 		StateRoot:  zeroHash,
 		BodyRoot:   bodyRoot[:],
-		Signature:  params.BeaconConfig().EmptySignature[:],
-	}
-
-	// Process initial deposits.
-	validatorMap := make(map[[48]byte]int)
-	leaves := [][]byte{}
-	for _, deposit := range deposits {
-		hash, err := ssz.HashTreeRoot(deposit.Data)
-		if err != nil {
-			return nil, err
-		}
-		leaves = append(leaves, hash[:])
-	}
-	var trie *trieutil.SparseMerkleTrie
-	if len(leaves) > 0 {
-		trie, err = trieutil.GenerateTrieFromItems(leaves, int(params.BeaconConfig().DepositContractTreeDepth))
-		if err != nil {
-			return nil, err
-		}
-	} else {
-		trie, err = trieutil.NewTrie(int(params.BeaconConfig().DepositContractTreeDepth))
-		if err != nil {
-			return nil, err
-		}
-	}
-
-	depositRoot := trie.Root()
-	state.Eth1Data.DepositRoot = depositRoot[:]
-	for i, deposit := range deposits {
-		state, err = b.ProcessDeposit(state, deposit, validatorMap)
-		if err != nil {
-			return nil, errors.Wrapf(err, "could not process validator deposit %d", i)
-		}
-	}
-	// Process genesis activations
-	for i, validator := range state.Validators {
-		balance := state.Balances[i]
-		validator.EffectiveBalance = mathutil.Min(balance-balance%params.BeaconConfig().EffectiveBalanceIncrement, params.BeaconConfig().MaxEffectiveBalance)
-		if state.Validators[i].EffectiveBalance ==
-			params.BeaconConfig().MaxEffectiveBalance {
-			state.Validators[i].ActivationEligibilityEpoch = 0
-			state.Validators[i].ActivationEpoch = 0
-		}
 	}
 
 	return state, nil
 }
 
+// EmptyGenesisState returns an empty beacon state object.
+func EmptyGenesisState() *pb.BeaconState {
+	state := &pb.BeaconState{
+		// Misc fields.
+		Slot: 0,
+		Fork: &pb.Fork{
+			PreviousVersion: params.BeaconConfig().GenesisForkVersion,
+			CurrentVersion:  params.BeaconConfig().GenesisForkVersion,
+			Epoch:           0,
+		},
+		// Validator registry fields.
+		Validators: []*ethpb.Validator{},
+		Balances:   []uint64{},
+
+		JustificationBits:         []byte{0},
+		HistoricalRoots:           [][]byte{},
+		CurrentEpochAttestations:  []*pb.PendingAttestation{},
+		PreviousEpochAttestations: []*pb.PendingAttestation{},
+
+		// Eth1 data.
+		Eth1Data:         &ethpb.Eth1Data{},
+		Eth1DataVotes:    []*ethpb.Eth1Data{},
+		Eth1DepositIndex: 0,
+	}
+	return state
+}
+
 // IsValidGenesisState gets called whenever there's a deposit event,
 // it checks whether there's enough effective balance to trigger and
 // if the minimum genesis time arrived already.

beacon-chain/core/state/transition.go

@@ -42,48 +42,41 @@ import (
 func ExecuteStateTransition(
 	ctx context.Context,
 	state *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	signed *ethpb.SignedBeaconBlock,
 ) (*pb.BeaconState, error) {
 	if ctx.Err() != nil {
 		return nil, ctx.Err()
 	}
+	if signed == nil || signed.Block == nil {
+		return nil, errors.New("nil block")
+	}
 
 	b.ClearEth1DataVoteCache()
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.ChainService.ExecuteStateTransition")
 	defer span.End()
 	var err error
 	// Execute per slots transition.
-	state, err = ProcessSlots(ctx, state, block.Slot)
+	state, err = ProcessSlots(ctx, state, signed.Block.Slot)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not process slot")
 	}
 
 	// Execute per block transition.
-	if block != nil {
+	state, err = ProcessBlock(ctx, state, signed)
-		state, err = ProcessBlock(ctx, state, block)
+	if err != nil {
-		if err != nil {
+		return nil, errors.Wrapf(err, "could not process block in slot %d", signed.Block.Slot)
-			return nil, errors.Wrapf(err, "could not process block in slot %d", block.Slot)
-		}
 	}
 
-	interop.WriteBlockToDisk(block, false)
+	interop.WriteBlockToDisk(signed, false)
 	interop.WriteStateToDisk(state)
 
-	var postStateRoot [32]byte
+	postStateRoot, err := stateutil.HashTreeRootState(state)
-	if featureconfig.Get().EnableCustomStateSSZ {
+	if err != nil {
-		postStateRoot, err = stateutil.HashTreeRootState(state)
+		return nil, errors.Wrap(err, "could not tree hash processed state")
-		if err != nil {
-			return nil, errors.Wrap(err, "could not tree hash processed state")
-		}
-	} else {
-		postStateRoot, err = ssz.HashTreeRoot(state)
-		if err != nil {
-			return nil, errors.Wrap(err, "could not tree hash processed state")
-		}
 	}
-	if !bytes.Equal(postStateRoot[:], block.StateRoot) {
+	if !bytes.Equal(postStateRoot[:], signed.Block.StateRoot) {
 		return state, fmt.Errorf("validate state root failed, wanted: %#x, received: %#x",
-			postStateRoot[:], block.StateRoot)
+			postStateRoot[:], signed.Block.StateRoot)
 	}
 
 	return state, nil
@@ -107,11 +100,14 @@ func ExecuteStateTransition(
 func ExecuteStateTransitionNoVerify(
 	ctx context.Context,
 	state *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	signed *ethpb.SignedBeaconBlock,
 ) (*pb.BeaconState, error) {
 	if ctx.Err() != nil {
 		return nil, ctx.Err()
 	}
+	if signed == nil || signed.Block == nil {
+		return nil, errors.New("nil block")
+	}
 
 	b.ClearEth1DataVoteCache()
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.ChainService.ExecuteStateTransitionNoVerify")
@@ -119,17 +115,15 @@ func ExecuteStateTransitionNoVerify(
 	var err error
 
 	// Execute per slots transition.
-	state, err = ProcessSlots(ctx, state, block.Slot)
+	state, err = ProcessSlots(ctx, state, signed.Block.Slot)
 	if err != nil {
 		return nil, errors.Wrap(err, "could not process slot")
 	}
 
 	// Execute per block transition.
-	if block != nil {
+	state, err = processBlockNoVerify(ctx, state, signed)
-		state, err = processBlockNoVerify(ctx, state, block)
+	if err != nil {
-		if err != nil {
+		return nil, errors.Wrap(err, "could not process block")
-			return nil, errors.Wrap(err, "could not process block")
-		}
 	}
 
 	return state, nil
@@ -154,7 +148,7 @@ func ExecuteStateTransitionNoVerify(
 func CalculateStateRoot(
 	ctx context.Context,
 	state *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	signed *ethpb.SignedBeaconBlock,
 ) ([32]byte, error) {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.ChainService.CalculateStateRoot")
 	defer span.End()
@@ -162,29 +156,27 @@ func CalculateStateRoot(
 		traceutil.AnnotateError(span, ctx.Err())
 		return [32]byte{}, ctx.Err()
 	}
+	if signed == nil || signed.Block == nil {
+		return [32]byte{}, errors.New("nil block")
+	}
 
 	stateCopy := proto.Clone(state).(*pb.BeaconState)
 	b.ClearEth1DataVoteCache()
 
 	var err error
 	// Execute per slots transition.
-	stateCopy, err = ProcessSlots(ctx, stateCopy, block.Slot)
+	stateCopy, err = ProcessSlots(ctx, stateCopy, signed.Block.Slot)
 	if err != nil {
 		return [32]byte{}, errors.Wrap(err, "could not process slot")
 	}
 
 	// Execute per block transition.
-	if block != nil {
+	stateCopy, err = processBlockNoVerify(ctx, stateCopy, signed)
-		stateCopy, err = processBlockNoVerify(ctx, stateCopy, block)
+	if err != nil {
-		if err != nil {
+		return [32]byte{}, errors.Wrap(err, "could not process block")
-			return [32]byte{}, errors.Wrap(err, "could not process block")
-		}
 	}
 
-	if featureconfig.Get().EnableCustomStateSSZ {
+	return stateutil.HashTreeRootState(stateCopy)
-		return stateutil.HashTreeRootState(stateCopy)
-	}
-	return ssz.HashTreeRoot(stateCopy)
 }
 
 // ProcessSlot happens every slot and focuses on the slot counter and block roots record updates.
@@ -208,20 +200,10 @@ func ProcessSlot(ctx context.Context, state *pb.BeaconState) (*pb.BeaconState, e
 	defer span.End()
 	span.AddAttributes(trace.Int64Attribute("slot", int64(state.Slot)))
 
-	var prevStateRoot [32]byte
+	prevStateRoot, err := stateutil.HashTreeRootState(state)
-	var err error
+	if err != nil {
-	if featureconfig.Get().EnableCustomStateSSZ {
+		traceutil.AnnotateError(span, err)
-		prevStateRoot, err = stateutil.HashTreeRootState(state)
+		return nil, errors.Wrap(err, "could not tree hash prev state root")
-		if err != nil {
-			traceutil.AnnotateError(span, err)
-			return nil, errors.Wrap(err, "could not tree hash prev state root")
-		}
-	} else {
-		prevStateRoot, err = ssz.HashTreeRoot(state)
-		if err != nil {
-			traceutil.AnnotateError(span, err)
-			return nil, errors.Wrap(err, "could not tree hash prev state root")
-		}
 	}
 	state.StateRoots[state.Slot%params.BeaconConfig().SlotsPerHistoricalRoot] = prevStateRoot[:]
 
@@ -230,7 +212,7 @@ func ProcessSlot(ctx context.Context, state *pb.BeaconState) (*pb.BeaconState, e
 	if bytes.Equal(state.LatestBlockHeader.StateRoot, zeroHash[:]) {
 		state.LatestBlockHeader.StateRoot = prevStateRoot[:]
 	}
-	prevBlockRoot, err := ssz.SigningRoot(state.LatestBlockHeader)
+	prevBlockRoot, err := ssz.HashTreeRoot(state.LatestBlockHeader)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not determine prev block root")
@@ -344,30 +326,30 @@ func ProcessSlots(ctx context.Context, state *pb.BeaconState, slot uint64) (*pb.
 func ProcessBlock(
 	ctx context.Context,
 	state *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	signed *ethpb.SignedBeaconBlock,
 ) (*pb.BeaconState, error) {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.ChainService.state.ProcessBlock")
 	defer span.End()
 
-	state, err := b.ProcessBlockHeader(state, block)
+	state, err := b.ProcessBlockHeader(state, signed)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process block header")
 	}
 
-	state, err = b.ProcessRandao(state, block.Body)
+	state, err = b.ProcessRandao(state, signed.Block.Body)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not verify and process randao")
 	}
 
-	state, err = b.ProcessEth1DataInBlock(state, block)
+	state, err = b.ProcessEth1DataInBlock(state, signed.Block)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process eth1 data")
 	}
 
-	state, err = ProcessOperations(ctx, state, block.Body)
+	state, err = ProcessOperations(ctx, state, signed.Block.Body)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process block operation")
@@ -394,30 +376,30 @@ func ProcessBlock(
 func processBlockNoVerify(
 	ctx context.Context,
 	state *pb.BeaconState,
-	block *ethpb.BeaconBlock,
+	signed *ethpb.SignedBeaconBlock,
 ) (*pb.BeaconState, error) {
 	ctx, span := trace.StartSpan(ctx, "beacon-chain.ChainService.state.ProcessBlock")
 	defer span.End()
 
-	state, err := b.ProcessBlockHeaderNoVerify(state, block)
+	state, err := b.ProcessBlockHeaderNoVerify(state, signed.Block)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process block header")
 	}
 
-	state, err = b.ProcessRandaoNoVerify(state, block.Body)
+	state, err = b.ProcessRandaoNoVerify(state, signed.Block.Body)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not verify and process randao")
 	}
 
-	state, err = b.ProcessEth1DataInBlock(state, block)
+	state, err = b.ProcessEth1DataInBlock(state, signed.Block)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process eth1 data")
 	}
 
-	state, err = processOperationsNoVerify(ctx, state, block.Body)
+	state, err = processOperationsNoVerify(ctx, state, signed.Block.Body)
 	if err != nil {
 		traceutil.AnnotateError(span, err)
 		return nil, errors.Wrap(err, "could not process block operation")

beacon-chain/core/state/transition_test.go

@@ -27,8 +27,10 @@ func TestExecuteStateTransition_IncorrectSlot(t *testing.T) {
 	beaconState := &pb.BeaconState{
 		Slot: 5,
 	}
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot: 4,
+		Block: &ethpb.BeaconBlock{
+			Slot: 4,
+		},
 	}
 	want := "expected state.slot"
 	if _, err := state.ExecuteStateTransition(context.Background(), beaconState, block); !strings.Contains(err.Error(), want) {
@@ -49,7 +51,7 @@ func TestExecuteStateTransition_FullProcess(t *testing.T) {
 	beaconState.Eth1DataVotes = []*ethpb.Eth1Data{eth1Data}
 
 	oldMix := beaconState.RandaoMixes[1]
-	parentRoot, err := ssz.SigningRoot(beaconState.LatestBlockHeader)
+	parentRoot, err := ssz.HashTreeRoot(beaconState.LatestBlockHeader)
 	if err != nil {
 		t.Error(err)
 	}
@@ -61,12 +63,14 @@ func TestExecuteStateTransition_FullProcess(t *testing.T) {
 		t.Fatal(err)
 	}
 	beaconState.Slot--
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot:       beaconState.Slot + 1,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: parentRoot[:],
+			Slot:       beaconState.Slot + 1,
-		Body: &ethpb.BeaconBlockBody{
+			ParentRoot: parentRoot[:],
-			RandaoReveal: randaoReveal,
+			Body: &ethpb.BeaconBlockBody{
-			Eth1Data:     eth1Data,
+				RandaoReveal: randaoReveal,
+				Eth1Data:     eth1Data,
+			},
 		},
 	}
 
@@ -75,9 +79,9 @@ func TestExecuteStateTransition_FullProcess(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	block.StateRoot = stateRoot[:]
+	block.Block.StateRoot = stateRoot[:]
 
-	sig, err := testutil.BlockSignature(beaconState, block, privKeys)
+	sig, err := testutil.BlockSignature(beaconState, block.Block, privKeys)
 	if err != nil {
 		t.Error(err)
 	}
@@ -105,12 +109,12 @@ func TestProcessBlock_IncorrectProposerSlashing(t *testing.T) {
 		t.Fatal(err)
 	}
 	slashing := &ethpb.ProposerSlashing{
-		Header_1: &ethpb.BeaconBlockHeader{Slot: params.BeaconConfig().SlotsPerEpoch},
+		Header_1: &ethpb.SignedBeaconBlockHeader{Header: &ethpb.BeaconBlockHeader{Slot: params.BeaconConfig().SlotsPerEpoch}},
-		Header_2: &ethpb.BeaconBlockHeader{Slot: params.BeaconConfig().SlotsPerEpoch * 2},
+		Header_2: &ethpb.SignedBeaconBlockHeader{Header: &ethpb.BeaconBlockHeader{Slot: params.BeaconConfig().SlotsPerEpoch * 2}},
 	}
-	block.Body.ProposerSlashings = []*ethpb.ProposerSlashing{slashing}
+	block.Block.Body.ProposerSlashings = []*ethpb.ProposerSlashing{slashing}
 
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -143,15 +147,14 @@ func TestProcessBlock_IncorrectProcessBlockAttestations(t *testing.T) {
 			Source: &ethpb.Checkpoint{Epoch: 0},
 		},
 		AggregationBits: bitfield.NewBitlist(3),
-		CustodyBits:     bitfield.NewBitlist(3),
 	}
 
 	block, err := testutil.GenerateFullBlock(beaconState, privKeys, nil, 1)
 	if err != nil {
 		t.Fatal(err)
 	}
-	block.Body.Attestations = []*ethpb.Attestation{att}
+	block.Block.Body.Attestations = []*ethpb.Attestation{att}
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -182,12 +185,16 @@ func TestProcessBlock_IncorrectProcessExits(t *testing.T) {
 	proposerSlashings := []*ethpb.ProposerSlashing{
 		{
 			ProposerIndex: 3,
-			Header_1: &ethpb.BeaconBlockHeader{
+			Header_1: &ethpb.SignedBeaconBlockHeader{
-				Slot:      1,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 1,
+				},
 				Signature: []byte("A"),
 			},
-			Header_2: &ethpb.BeaconBlockHeader{
+			Header_2: &ethpb.SignedBeaconBlockHeader{
-				Slot:      1,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 1,
+				},
 				Signature: []byte("B"),
 			},
 		},
@@ -199,14 +206,14 @@ func TestProcessBlock_IncorrectProcessExits(t *testing.T) {
 					Source: &ethpb.Checkpoint{Epoch: 0},
 					Target: &ethpb.Checkpoint{Epoch: 0},
 				},
-				CustodyBit_0Indices: []uint64{0, 1},
+				AttestingIndices: []uint64{0, 1},
 			},
 			Attestation_2: &ethpb.IndexedAttestation{
 				Data: &ethpb.AttestationData{
 					Source: &ethpb.Checkpoint{Epoch: 1},
 					Target: &ethpb.Checkpoint{Epoch: 0},
 				},
-				CustodyBit_0Indices: []uint64{0, 1},
+				AttestingIndices: []uint64{0, 1},
 			},
 		},
 	}
@@ -221,39 +228,40 @@ func TestProcessBlock_IncorrectProcessExits(t *testing.T) {
 			Target: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
 		},
 		AggregationBits: bitfield.Bitlist{0xC0, 0xC0, 0xC0, 0xC0, 0x01},
-		CustodyBits:     bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
 	}
 	attestations := []*ethpb.Attestation{blockAtt}
-	var exits []*ethpb.VoluntaryExit
+	var exits []*ethpb.SignedVoluntaryExit
 	for i := uint64(0); i < params.BeaconConfig().MaxVoluntaryExits+1; i++ {
-		exits = append(exits, &ethpb.VoluntaryExit{})
+		exits = append(exits, &ethpb.SignedVoluntaryExit{})
 	}
 	genesisBlock := blocks.NewGenesisBlock([]byte{})
-	bodyRoot, err := ssz.HashTreeRoot(genesisBlock)
+	bodyRoot, err := ssz.HashTreeRoot(genesisBlock.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
 	beaconState.LatestBlockHeader = &ethpb.BeaconBlockHeader{
-		Slot:       genesisBlock.Slot,
+		Slot:       genesisBlock.Block.Slot,
-		ParentRoot: genesisBlock.ParentRoot,
+		ParentRoot: genesisBlock.Block.ParentRoot,
 		BodyRoot:   bodyRoot[:],
 	}
-	parentRoot, err := ssz.SigningRoot(beaconState.LatestBlockHeader)
+	parentRoot, err := ssz.HashTreeRoot(beaconState.LatestBlockHeader)
 	if err != nil {
 		t.Fatal(err)
 	}
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		ParentRoot: parentRoot[:],
+		Block: &ethpb.BeaconBlock{
-		Slot:       1,
+			ParentRoot: parentRoot[:],
-		Body: &ethpb.BeaconBlockBody{
+			Slot:       1,
-			RandaoReveal:      []byte{},
+			Body: &ethpb.BeaconBlockBody{
-			ProposerSlashings: proposerSlashings,
+				RandaoReveal:      []byte{},
-			AttesterSlashings: attesterSlashings,
+				ProposerSlashings: proposerSlashings,
-			Attestations:      attestations,
+				AttesterSlashings: attesterSlashings,
-			VoluntaryExits:    exits,
+				Attestations:      attestations,
-			Eth1Data: &ethpb.Eth1Data{
+				VoluntaryExits:    exits,
-				DepositRoot: []byte{2},
+				Eth1Data: &ethpb.Eth1Data{
-				BlockHash:   []byte{3},
+					DepositRoot: []byte{2},
+					BlockHash:   []byte{3},
+				},
 			},
 		},
 	}
@@ -268,13 +276,13 @@ func TestProcessBlock_IncorrectProcessExits(t *testing.T) {
 func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 	beaconState, privKeys := testutil.DeterministicGenesisState(t, 32)
 	genesisBlock := blocks.NewGenesisBlock([]byte{})
-	bodyRoot, err := ssz.HashTreeRoot(genesisBlock)
+	bodyRoot, err := ssz.HashTreeRoot(genesisBlock.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
 	beaconState.LatestBlockHeader = &ethpb.BeaconBlockHeader{
-		Slot:       genesisBlock.Slot,
+		Slot:       genesisBlock.Block.Slot,
-		ParentRoot: genesisBlock.ParentRoot,
+		ParentRoot: genesisBlock.Block.ParentRoot,
 		BodyRoot:   bodyRoot[:],
 	}
 	beaconState.Slashings = make([]uint64, params.BeaconConfig().EpochsPerSlashingsVector)
@@ -292,21 +300,25 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 		params.BeaconConfig().DomainBeaconProposer,
 	)
 
-	header1 := &ethpb.BeaconBlockHeader{
+	header1 := &ethpb.SignedBeaconBlockHeader{
-		Slot:      1,
+		Header: &ethpb.BeaconBlockHeader{
-		StateRoot: []byte("A"),
+			Slot:      1,
+			StateRoot: []byte("A"),
+		},
 	}
-	signingRoot, err := ssz.SigningRoot(header1)
+	signingRoot, err := ssz.HashTreeRoot(header1.Header)
 	if err != nil {
 		t.Errorf("Could not get signing root of beacon block header: %v", err)
 	}
 	header1.Signature = privKeys[proposerSlashIdx].Sign(signingRoot[:], domain).Marshal()[:]
 
-	header2 := &ethpb.BeaconBlockHeader{
+	header2 := &ethpb.SignedBeaconBlockHeader{
-		Slot:      1,
+		Header: &ethpb.BeaconBlockHeader{
-		StateRoot: []byte("B"),
+			Slot:      1,
+			StateRoot: []byte("B"),
+		},
 	}
-	signingRoot, err = ssz.SigningRoot(header2)
+	signingRoot, err = ssz.HashTreeRoot(header2.Header)
 	if err != nil {
 		t.Errorf("Could not get signing root of beacon block header: %v", err)
 	}
@@ -325,13 +337,9 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 		Data: &ethpb.AttestationData{
 			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte{'A'}},
 			Target: &ethpb.Checkpoint{Epoch: 0}},
-		CustodyBit_0Indices: []uint64{0, 1},
+		AttestingIndices: []uint64{0, 1},
 	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err := ssz.HashTreeRoot(att1.Data)
-		Data:       att1.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -345,13 +353,9 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 		Data: &ethpb.AttestationData{
 			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte{'B'}},
 			Target: &ethpb.Checkpoint{Epoch: 0}},
-		CustodyBit_0Indices: []uint64{0, 1},
+		AttestingIndices: []uint64{0, 1},
 	}
-	dataAndCustodyBit = &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err = ssz.HashTreeRoot(att2.Data)
-		Data:       att2.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err = ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -375,7 +379,6 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 
 	aggBits := bitfield.NewBitlist(1)
 	aggBits.SetBitAt(0, true)
-	custodyBits := bitfield.NewBitlist(1)
 	blockAtt := &ethpb.Attestation{
 		Data: &ethpb.AttestationData{
 			Slot:   beaconState.Slot - 1,
@@ -385,7 +388,6 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 				Root:  []byte("hello-world"),
 			}},
 		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
 	}
 
 	committee, err := helpers.BeaconCommitteeFromState(beaconState, blockAtt.Data.Slot, blockAtt.Data.CommitteeIndex)
@@ -396,11 +398,7 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 	if err != nil {
 		t.Error(err)
 	}
-	dataAndCustodyBit = &pb.AttestationDataAndCustodyBit{
+	hashTreeRoot, err = ssz.HashTreeRoot(blockAtt.Data)
-		Data:       blockAtt.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err = ssz.HashTreeRoot(dataAndCustodyBit)
 	if err != nil {
 		t.Error(err)
 	}
@@ -411,18 +409,20 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 	}
 	blockAtt.Signature = bls.AggregateSignatures(sigs).Marshal()[:]
 
-	exit := &ethpb.VoluntaryExit{
+	exit := &ethpb.SignedVoluntaryExit{
-		ValidatorIndex: 10,
+		Exit: &ethpb.VoluntaryExit{
-		Epoch:          0,
+			ValidatorIndex: 10,
+			Epoch:          0,
+		},
 	}
-	signingRoot, err = ssz.SigningRoot(exit)
+	signingRoot, err = ssz.HashTreeRoot(exit.Exit)
 	if err != nil {
 		t.Errorf("Could not get signing root of beacon block header: %v", err)
 	}
 	domain = helpers.Domain(beaconState.Fork, currentEpoch, params.BeaconConfig().DomainVoluntaryExit)
-	exit.Signature = privKeys[exit.ValidatorIndex].Sign(signingRoot[:], domain).Marshal()[:]
+	exit.Signature = privKeys[exit.Exit.ValidatorIndex].Sign(signingRoot[:], domain).Marshal()[:]
 
-	parentRoot, err := ssz.SigningRoot(beaconState.LatestBlockHeader)
+	parentRoot, err := ssz.HashTreeRoot(beaconState.LatestBlockHeader)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -431,23 +431,25 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		ParentRoot: parentRoot[:],
+		Block: &ethpb.BeaconBlock{
-		Slot:       beaconState.Slot,
+			ParentRoot: parentRoot[:],
-		Body: &ethpb.BeaconBlockBody{
+			Slot:       beaconState.Slot,
-			RandaoReveal:      randaoReveal,
+			Body: &ethpb.BeaconBlockBody{
-			ProposerSlashings: proposerSlashings,
+				RandaoReveal:      randaoReveal,
-			AttesterSlashings: attesterSlashings,
+				ProposerSlashings: proposerSlashings,
-			Attestations:      []*ethpb.Attestation{blockAtt},
+				AttesterSlashings: attesterSlashings,
-			VoluntaryExits:    []*ethpb.VoluntaryExit{exit},
+				Attestations:      []*ethpb.Attestation{blockAtt},
-			Eth1Data: &ethpb.Eth1Data{
+				VoluntaryExits:    []*ethpb.SignedVoluntaryExit{exit},
-				DepositRoot: []byte{2},
+				Eth1Data: &ethpb.Eth1Data{
-				BlockHash:   []byte{3},
+					DepositRoot: []byte{2},
+					BlockHash:   []byte{3},
+				},
 			},
 		},
 	}
 
-	sig, err := testutil.BlockSignature(beaconState, block, privKeys)
+	sig, err := testutil.BlockSignature(beaconState, block.Block, privKeys)
 	if err != nil {
 		t.Error(err)
 	}
@@ -466,10 +468,10 @@ func TestProcessBlock_PassesProcessingConditions(t *testing.T) {
 		t.Error("Expected validator at index 1 to be slashed, received false")
 	}
 
-	received := beaconState.Validators[exit.ValidatorIndex].ExitEpoch
+	received := beaconState.Validators[exit.Exit.ValidatorIndex].ExitEpoch
 	wanted := params.BeaconConfig().FarFutureEpoch
 	if received == wanted {
-		t.Errorf("Expected validator at index %d to be exiting, did not expect: %d", exit.ValidatorIndex, wanted)
+		t.Errorf("Expected validator at index %d to be exiting, did not expect: %d", exit.Exit.ValidatorIndex, wanted)
 	}
 }
 
@@ -542,12 +544,16 @@ func BenchmarkProcessBlk_65536Validators_FullBlock(b *testing.B) {
 	proposerSlashings := []*ethpb.ProposerSlashing{
 		{
 			ProposerIndex: 1,
-			Header_1: &ethpb.BeaconBlockHeader{
+			Header_1: &ethpb.SignedBeaconBlockHeader{
-				Slot:      0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 				Signature: []byte("A"),
 			},
-			Header_2: &ethpb.BeaconBlockHeader{
+			Header_2: &ethpb.SignedBeaconBlockHeader{
-				Slot:      0,
+				Header: &ethpb.BeaconBlockHeader{
+					Slot: 0,
+				},
 				Signature: []byte("B"),
 			},
 		},
@@ -557,12 +563,12 @@ func BenchmarkProcessBlk_65536Validators_FullBlock(b *testing.B) {
 	attesterSlashings := []*ethpb.AttesterSlashing{
 		{
 			Attestation_1: &ethpb.IndexedAttestation{
-				Data:                &ethpb.AttestationData{},
+				Data:             &ethpb.AttestationData{},
-				CustodyBit_0Indices: []uint64{2, 3},
+				AttestingIndices: []uint64{2, 3},
 			},
 			Attestation_2: &ethpb.IndexedAttestation{
-				Data:                &ethpb.AttestationData{},
+				Data:             &ethpb.AttestationData{},
-				CustodyBit_0Indices: []uint64{2, 3},
+				AttestingIndices: []uint64{2, 3},
 			},
 		},
 	}
@@ -614,21 +620,22 @@ func BenchmarkProcessBlk_65536Validators_FullBlock(b *testing.B) {
 				Source: &ethpb.Checkpoint{Root: []byte("hello-world")}},
 			AggregationBits: bitfield.Bitlist{0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0,
 				0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0xC0, 0x01},
-			CustodyBits: bitfield.NewBitlist(0),
 		}
 	}
 
-	blk := &ethpb.BeaconBlock{
+	blk := &ethpb.SignedBeaconBlock{
-		Slot: s.Slot,
+		Block: &ethpb.BeaconBlock{
-		Body: &ethpb.BeaconBlockBody{
+			Slot: s.Slot,
-			Eth1Data: &ethpb.Eth1Data{
+			Body: &ethpb.BeaconBlockBody{
-				DepositRoot: root[:],
+				Eth1Data: &ethpb.Eth1Data{
-				BlockHash:   root[:],
+					DepositRoot: root[:],
+					BlockHash:   root[:],
+				},
+				RandaoReveal:      epochSignature.Marshal(),
+				Attestations:      attestations,
+				ProposerSlashings: proposerSlashings,
+				AttesterSlashings: attesterSlashings,
 			},
-			RandaoReveal:      epochSignature.Marshal(),
-			Attestations:      attestations,
-			ProposerSlashings: proposerSlashings,
-			AttesterSlashings: attesterSlashings,
 		},
 	}
 
@@ -662,7 +669,6 @@ func TestProcessBlk_AttsBasedOnValidatorCount(t *testing.T) {
 
 	bitCount := validatorCount / params.BeaconConfig().SlotsPerEpoch
 	aggBits := bitfield.NewBitlist(bitCount)
-	custodyBits := bitfield.NewBitlist(bitCount)
 	for i := uint64(1); i < bitCount; i++ {
 		aggBits.SetBitAt(i, true)
 	}
@@ -674,7 +680,6 @@ func TestProcessBlk_AttsBasedOnValidatorCount(t *testing.T) {
 				Source: &ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]},
 				Target: &ethpb.Checkpoint{Epoch: 0}},
 			AggregationBits: aggBits,
-			CustodyBits:     custodyBits,
 		}
 
 		committee, err := helpers.BeaconCommitteeFromState(s, att.Data.Slot, att.Data.CommitteeIndex)
@@ -685,15 +690,10 @@ func TestProcessBlk_AttsBasedOnValidatorCount(t *testing.T) {
 		if err != nil {
 			t.Error(err)
 		}
-		dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-			Data:       att.Data,
-			CustodyBit: false,
-		}
-
 		domain := helpers.Domain(s.Fork, 0, params.BeaconConfig().DomainBeaconAttester)
 		sigs := make([]*bls.Signature, len(attestingIndices))
 		for i, indice := range attestingIndices {
-			hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
+			hashTreeRoot, err := ssz.HashTreeRoot(att.Data)
 			if err != nil {
 				t.Error(err)
 			}
@@ -705,17 +705,19 @@ func TestProcessBlk_AttsBasedOnValidatorCount(t *testing.T) {
 	}
 
 	epochSignature, _ := testutil.RandaoReveal(s, helpers.CurrentEpoch(s), privKeys)
-	parentRoot, _ := ssz.SigningRoot(s.LatestBlockHeader)
+	parentRoot, _ := ssz.HashTreeRoot(s.LatestBlockHeader)
-	blk := &ethpb.BeaconBlock{
+	blk := &ethpb.SignedBeaconBlock{
-		Slot:       s.Slot,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: parentRoot[:],
+			Slot:       s.Slot,
-		Body: &ethpb.BeaconBlockBody{
+			ParentRoot: parentRoot[:],
-			Eth1Data:     &ethpb.Eth1Data{},
+			Body: &ethpb.BeaconBlockBody{
-			RandaoReveal: epochSignature,
+				Eth1Data:     &ethpb.Eth1Data{},
-			Attestations: atts,
+				RandaoReveal: epochSignature,
+				Attestations: atts,
+			},
 		},
 	}
-	sig, _ := testutil.BlockSignature(s, blk, privKeys)
+	sig, _ := testutil.BlockSignature(s, blk.Block, privKeys)
 	blk.Signature = sig.Marshal()
 
 	config := params.BeaconConfig()
@@ -824,7 +826,7 @@ func TestProcessOperation_OverMaxVoluntaryExits(t *testing.T) {
 	maxExits := params.BeaconConfig().MaxVoluntaryExits
 	block := &ethpb.BeaconBlock{
 		Body: &ethpb.BeaconBlockBody{
-			VoluntaryExits: make([]*ethpb.VoluntaryExit, maxExits+1),
+			VoluntaryExits: make([]*ethpb.SignedVoluntaryExit, maxExits+1),
 		},
 	}
 

beacon-chain/db/iface/BUILD.bazel

@@ -8,6 +8,7 @@ go_library(
     visibility = ["//beacon-chain/db:__subpackages__"],
     deps = [
         "//beacon-chain/db/filters:go_default_library",
+        "//proto/beacon/db:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",

beacon-chain/db/iface/interface.go

@@ -9,6 +9,7 @@ import (
 	eth "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
+	"github.com/prysmaticlabs/prysm/proto/beacon/db"
 	ethereum_beacon_p2p_v1 "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 )
 
@@ -29,17 +30,17 @@ type Database interface {
 	SaveAttestation(ctx context.Context, att *eth.Attestation) error
 	SaveAttestations(ctx context.Context, atts []*eth.Attestation) error
 	// Block related methods.
-	Block(ctx context.Context, blockRoot [32]byte) (*eth.BeaconBlock, error)
+	Block(ctx context.Context, blockRoot [32]byte) (*eth.SignedBeaconBlock, error)
-	HeadBlock(ctx context.Context) (*eth.BeaconBlock, error)
+	HeadBlock(ctx context.Context) (*eth.SignedBeaconBlock, error)
-	Blocks(ctx context.Context, f *filters.QueryFilter) ([]*eth.BeaconBlock, error)
+	Blocks(ctx context.Context, f *filters.QueryFilter) ([]*eth.SignedBeaconBlock, error)
 	BlockRoots(ctx context.Context, f *filters.QueryFilter) ([][32]byte, error)
 	HasBlock(ctx context.Context, blockRoot [32]byte) bool
 	DeleteBlock(ctx context.Context, blockRoot [32]byte) error
 	DeleteBlocks(ctx context.Context, blockRoots [][32]byte) error
-	SaveBlock(ctx context.Context, block *eth.BeaconBlock) error
+	SaveBlock(ctx context.Context, block *eth.SignedBeaconBlock) error
-	SaveBlocks(ctx context.Context, blocks []*eth.BeaconBlock) error
+	SaveBlocks(ctx context.Context, blocks []*eth.SignedBeaconBlock) error
 	SaveHeadBlockRoot(ctx context.Context, blockRoot [32]byte) error
-	GenesisBlock(ctx context.Context) (*ethpb.BeaconBlock, error)
+	GenesisBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error)
 	SaveGenesisBlockRoot(ctx context.Context, blockRoot [32]byte) error
 	IsFinalizedBlock(ctx context.Context, blockRoot [32]byte) bool
 	// Validator related methods.
@@ -85,4 +86,7 @@ type Database interface {
 	// Deposit contract related handlers.
 	DepositContractAddress(ctx context.Context) ([]byte, error)
 	SaveDepositContractAddress(ctx context.Context, addr common.Address) error
+	//Powchain operations
+	PowchainData(ctx context.Context) (*db.ETH1ChainData, error)
+	SavePowchainData(ctx context.Context, data *db.ETH1ChainData) error
 }

beacon-chain/db/kafka/BUILD.bazel

@@ -11,6 +11,7 @@ go_library(
     deps = [
         "//beacon-chain/db/filters:go_default_library",
         "//beacon-chain/db/iface:go_default_library",
+        "//proto/beacon/db:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/traceutil:go_default_library",

beacon-chain/db/kafka/export_wrapper.go

@@ -100,7 +100,7 @@ func (e Exporter) SaveAttestations(ctx context.Context, atts []*eth.Attestation)
 }
 
 // SaveBlock publishes to the kafka topic for beacon blocks.
-func (e Exporter) SaveBlock(ctx context.Context, block *eth.BeaconBlock) error {
+func (e Exporter) SaveBlock(ctx context.Context, block *eth.SignedBeaconBlock) error {
 	go func() {
 		if err := e.publish(ctx, "beacon_block", block); err != nil {
 			log.WithError(err).Error("Failed to publish block")
@@ -111,7 +111,7 @@ func (e Exporter) SaveBlock(ctx context.Context, block *eth.BeaconBlock) error {
 }
 
 // SaveBlocks publishes to the kafka topic for beacon blocks.
-func (e Exporter) SaveBlocks(ctx context.Context, blocks []*eth.BeaconBlock) error {
+func (e Exporter) SaveBlocks(ctx context.Context, blocks []*eth.SignedBeaconBlock) error {
 	go func() {
 		for _, block := range blocks {
 			if err := e.publish(ctx, "beacon_block", block); err != nil {

beacon-chain/db/kafka/passthrough.go

@@ -7,6 +7,7 @@ import (
 	eth "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
+	"github.com/prysmaticlabs/prysm/proto/beacon/db"
 	ethereum_beacon_p2p_v1 "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 )
 
@@ -51,17 +52,17 @@ func (e Exporter) DeleteAttestations(ctx context.Context, attDataRoots [][32]byt
 }
 
 // Block -- passthrough.
-func (e Exporter) Block(ctx context.Context, blockRoot [32]byte) (*eth.BeaconBlock, error) {
+func (e Exporter) Block(ctx context.Context, blockRoot [32]byte) (*eth.SignedBeaconBlock, error) {
 	return e.db.Block(ctx, blockRoot)
 }
 
 // HeadBlock -- passthrough.
-func (e Exporter) HeadBlock(ctx context.Context) (*eth.BeaconBlock, error) {
+func (e Exporter) HeadBlock(ctx context.Context) (*eth.SignedBeaconBlock, error) {
 	return e.db.HeadBlock(ctx)
 }
 
 // Blocks -- passthrough.
-func (e Exporter) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*eth.BeaconBlock, error) {
+func (e Exporter) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*eth.SignedBeaconBlock, error) {
 	return e.db.Blocks(ctx, f)
 }
 
@@ -201,7 +202,7 @@ func (e Exporter) SaveHeadBlockRoot(ctx context.Context, blockRoot [32]byte) err
 }
 
 // GenesisBlock -- passthrough.
-func (e Exporter) GenesisBlock(ctx context.Context) (*ethpb.BeaconBlock, error) {
+func (e Exporter) GenesisBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error) {
 	return e.db.GenesisBlock(ctx)
 }
 
@@ -284,3 +285,13 @@ func (e Exporter) DeleteStates(ctx context.Context, blockRoots [][32]byte) error
 func (e Exporter) IsFinalizedBlock(ctx context.Context, blockRoot [32]byte) bool {
 	return e.db.IsFinalizedBlock(ctx, blockRoot)
 }
+
+// PowchainData -- passthrough
+func (e Exporter) PowchainData(ctx context.Context) (*db.ETH1ChainData, error) {
+	return e.db.PowchainData(ctx)
+}
+
+// SavePowchainData -- passthrough
+func (e Exporter) SavePowchainData(ctx context.Context, data *db.ETH1ChainData) error {
+	return e.db.SavePowchainData(ctx, data)
+}

beacon-chain/db/kv/BUILD.bazel

@@ -12,8 +12,8 @@ go_library(
         "encoding.go",
         "finalized_block_roots.go",
         "kv.go",
-        "migrate_snappy.go",
         "operations.go",
+        "powchain.go",
         "prune_states.go",
         "schema.go",
         "slashings.go",
@@ -30,7 +30,6 @@ go_library(
         "//proto/beacon/db:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
-        "//shared/featureconfig:go_default_library",
         "//shared/params:go_default_library",
         "//shared/sliceutil:go_default_library",
         "//shared/traceutil:go_default_library",
@@ -60,9 +59,7 @@ go_test(
         "deposit_contract_test.go",
         "finalized_block_roots_test.go",
         "kv_test.go",
-        "migrate_snappy_test.go",
         "operations_test.go",
-        "prune_states_test.go",
         "slashings_test.go",
         "state_test.go",
         "validators_test.go",
@@ -72,7 +69,6 @@ go_test(
         "//beacon-chain/db/filters:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
-        "//shared/featureconfig:go_default_library",
         "//shared/params:go_default_library",
         "//shared/testutil:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",

beacon-chain/db/kv/archive_test.go

@@ -39,19 +39,23 @@ func TestStore_ArchivedActiveValidatorChanges(t *testing.T) {
 		ProposerSlashings: []*ethpb.ProposerSlashing{
 			{
 				ProposerIndex: 1212,
-				Header_1: &ethpb.BeaconBlockHeader{
+				Header_1: &ethpb.SignedBeaconBlockHeader{
-					Slot:       10,
+					Header: &ethpb.BeaconBlockHeader{
-					ParentRoot: someRoot[:],
+						Slot:       10,
-					StateRoot:  someRoot[:],
+						ParentRoot: someRoot[:],
-					BodyRoot:   someRoot[:],
+						StateRoot:  someRoot[:],
-					Signature:  make([]byte, 96),
+						BodyRoot:   someRoot[:],
+					},
+					Signature: make([]byte, 96),
 				},
-				Header_2: &ethpb.BeaconBlockHeader{
+				Header_2: &ethpb.SignedBeaconBlockHeader{
-					Slot:       10,
+					Header: &ethpb.BeaconBlockHeader{
-					ParentRoot: someRoot[:],
+						Slot:       10,
-					StateRoot:  someRoot[:],
+						ParentRoot: someRoot[:],
-					BodyRoot:   someRoot[:],
+						StateRoot:  someRoot[:],
-					Signature:  make([]byte, 96),
+						BodyRoot:   someRoot[:],
+					},
+					Signature: make([]byte, 96),
 				},
 			},
 		},

beacon-chain/db/kv/attestations_test.go

@@ -21,7 +21,6 @@ func TestStore_AttestationCRUD(t *testing.T) {
 	att := &ethpb.Attestation{
 		Data:            &ethpb.AttestationData{Slot: 10},
 		AggregationBits: bitfield.Bitlist{0b00000001, 0b1},
-		CustodyBits:     bitfield.NewBitlist(8),
 	}
 	ctx := context.Background()
 	attDataRoot, err := ssz.HashTreeRoot(att.Data)
@@ -72,7 +71,6 @@ func TestStore_AttestationsBatchDelete(t *testing.T) {
 				Slot:            uint64(i),
 			},
 			AggregationBits: bitfield.Bitlist{0b00000001, 0b1},
-			CustodyBits:     bitfield.NewBitlist(8),
 		}
 		if i%2 == 0 {
 			r, err := ssz.HashTreeRoot(totalAtts[i].Data)

beacon-chain/db/kv/backup.go

@@ -32,7 +32,7 @@ func (k *Store) Backup(ctx context.Context) error {
 	if err := os.MkdirAll(backupsDir, os.ModePerm); err != nil {
 		return err
 	}
-	backupPath := path.Join(backupsDir, fmt.Sprintf("prysm_beacondb_at_slot_%07d.backup", head.Slot))
+	backupPath := path.Join(backupsDir, fmt.Sprintf("prysm_beacondb_at_slot_%07d.backup", head.Block.Slot))
 	logrus.WithField("prefix", "db").WithField("backup", backupPath).Info("Writing backup database.")
 	return k.db.View(func(tx *bolt.Tx) error {
 		return tx.CopyFile(backupPath, 0666)

beacon-chain/db/kv/backup_test.go

@@ -16,13 +16,12 @@ func TestStore_Backup(t *testing.T) {
 	defer teardownDB(t, db)
 	ctx := context.Background()
 
-	head := &eth.BeaconBlock{}
+	head := &eth.SignedBeaconBlock{Block: &eth.BeaconBlock{Slot: 5000}}
-	head.Slot = 5000
 
 	if err := db.SaveBlock(ctx, head); err != nil {
 		t.Fatal(err)
 	}
-	root, err := ssz.SigningRoot(head)
+	root, err := ssz.HashTreeRoot(head.Block)
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/db/kv/blocks.go

@@ -18,31 +18,31 @@ import (
 )
 
 // Block retrieval by root.
-func (k *Store) Block(ctx context.Context, blockRoot [32]byte) (*ethpb.BeaconBlock, error) {
+func (k *Store) Block(ctx context.Context, blockRoot [32]byte) (*ethpb.SignedBeaconBlock, error) {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.Block")
 	defer span.End()
 	// Return block from cache if it exists.
 	if v, ok := k.blockCache.Get(string(blockRoot[:])); v != nil && ok {
-		return v.(*ethpb.BeaconBlock), nil
+		return v.(*ethpb.SignedBeaconBlock), nil
 	}
-	var block *ethpb.BeaconBlock
+	var block *ethpb.SignedBeaconBlock
 	err := k.db.View(func(tx *bolt.Tx) error {
 		bkt := tx.Bucket(blocksBucket)
 		enc := bkt.Get(blockRoot[:])
 		if enc == nil {
 			return nil
 		}
-		block = &ethpb.BeaconBlock{}
+		block = &ethpb.SignedBeaconBlock{}
 		return decode(enc, block)
 	})
 	return block, err
 }
 
 // HeadBlock returns the latest canonical block in eth2.
-func (k *Store) HeadBlock(ctx context.Context) (*ethpb.BeaconBlock, error) {
+func (k *Store) HeadBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error) {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.HeadBlock")
 	defer span.End()
-	var headBlock *ethpb.BeaconBlock
+	var headBlock *ethpb.SignedBeaconBlock
 	err := k.db.View(func(tx *bolt.Tx) error {
 		bkt := tx.Bucket(blocksBucket)
 		headRoot := bkt.Get(headBlockRootKey)
@@ -53,17 +53,17 @@ func (k *Store) HeadBlock(ctx context.Context) (*ethpb.BeaconBlock, error) {
 		if enc == nil {
 			return nil
 		}
-		headBlock = &ethpb.BeaconBlock{}
+		headBlock = &ethpb.SignedBeaconBlock{}
 		return decode(enc, headBlock)
 	})
 	return headBlock, err
 }
 
 // Blocks retrieves a list of beacon blocks by filter criteria.
-func (k *Store) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*ethpb.BeaconBlock, error) {
+func (k *Store) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*ethpb.SignedBeaconBlock, error) {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.Blocks")
 	defer span.End()
-	blocks := make([]*ethpb.BeaconBlock, 0)
+	blocks := make([]*ethpb.SignedBeaconBlock, 0)
 	err := k.db.View(func(tx *bolt.Tx) error {
 		bkt := tx.Bucket(blocksBucket)
 
@@ -112,7 +112,7 @@ func (k *Store) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*ethpb.Be
 		}
 		for i := 0; i < len(keys); i++ {
 			encoded := bkt.Get(keys[i])
-			block := &ethpb.BeaconBlock{}
+			block := &ethpb.SignedBeaconBlock{}
 			if err := decode(encoded, block); err != nil {
 				return err
 			}
@@ -208,11 +208,11 @@ func (k *Store) DeleteBlock(ctx context.Context, blockRoot [32]byte) error {
 		if enc == nil {
 			return nil
 		}
-		block := &ethpb.BeaconBlock{}
+		block := &ethpb.SignedBeaconBlock{}
 		if err := decode(enc, block); err != nil {
 			return err
 		}
-		indicesByBucket := createBlockIndicesFromBlock(block)
+		indicesByBucket := createBlockIndicesFromBlock(block.Block)
 		if err := deleteValueForIndices(indicesByBucket, blockRoot[:], tx); err != nil {
 			return errors.Wrap(err, "could not delete root for DB indices")
 		}
@@ -233,11 +233,11 @@ func (k *Store) DeleteBlocks(ctx context.Context, blockRoots [][32]byte) error {
 			if enc == nil {
 				return nil
 			}
-			block := &ethpb.BeaconBlock{}
+			block := &ethpb.SignedBeaconBlock{}
 			if err := decode(enc, block); err != nil {
 				return err
 			}
-			indicesByBucket := createBlockIndicesFromBlock(block)
+			indicesByBucket := createBlockIndicesFromBlock(block.Block)
 			if err := deleteValueForIndices(indicesByBucket, blockRoot[:], tx); err != nil {
 				return errors.Wrap(err, "could not delete root for DB indices")
 			}
@@ -251,10 +251,10 @@ func (k *Store) DeleteBlocks(ctx context.Context, blockRoots [][32]byte) error {
 }
 
 // SaveBlock to the db.
-func (k *Store) SaveBlock(ctx context.Context, block *ethpb.BeaconBlock) error {
+func (k *Store) SaveBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.SaveBlock")
 	defer span.End()
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(signed.Block)
 	if err != nil {
 		return err
 	}
@@ -266,27 +266,27 @@ func (k *Store) SaveBlock(ctx context.Context, block *ethpb.BeaconBlock) error {
 		if existingBlock := bkt.Get(blockRoot[:]); existingBlock != nil {
 			return nil
 		}
-		enc, err := encode(block)
+		enc, err := encode(signed)
 		if err != nil {
 			return err
 		}
-		indicesByBucket := createBlockIndicesFromBlock(block)
+		indicesByBucket := createBlockIndicesFromBlock(signed.Block)
 		if err := updateValueForIndices(indicesByBucket, blockRoot[:], tx); err != nil {
 			return errors.Wrap(err, "could not update DB indices")
 		}
-		k.blockCache.Set(string(blockRoot[:]), block, int64(len(enc)))
+		k.blockCache.Set(string(blockRoot[:]), signed, int64(len(enc)))
 		return bkt.Put(blockRoot[:], enc)
 	})
 }
 
 // SaveBlocks via bulk updates to the db.
-func (k *Store) SaveBlocks(ctx context.Context, blocks []*ethpb.BeaconBlock) error {
+func (k *Store) SaveBlocks(ctx context.Context, blocks []*ethpb.SignedBeaconBlock) error {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.SaveBlocks")
 	defer span.End()
 
 	return k.db.Update(func(tx *bolt.Tx) error {
 		for _, block := range blocks {
-			blockRoot, err := ssz.SigningRoot(block)
+			blockRoot, err := ssz.HashTreeRoot(block.Block)
 			if err != nil {
 				return err
 			}
@@ -298,7 +298,7 @@ func (k *Store) SaveBlocks(ctx context.Context, blocks []*ethpb.BeaconBlock) err
 			if err != nil {
 				return err
 			}
-			indicesByBucket := createBlockIndicesFromBlock(block)
+			indicesByBucket := createBlockIndicesFromBlock(block.Block)
 			if err := updateValueForIndices(indicesByBucket, blockRoot[:], tx); err != nil {
 				return errors.Wrap(err, "could not update DB indices")
 			}
@@ -325,10 +325,10 @@ func (k *Store) SaveHeadBlockRoot(ctx context.Context, blockRoot [32]byte) error
 }
 
 // GenesisBlock retrieves the genesis block of the beacon chain.
-func (k *Store) GenesisBlock(ctx context.Context) (*ethpb.BeaconBlock, error) {
+func (k *Store) GenesisBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error) {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.GenesisBlock")
 	defer span.End()
-	var block *ethpb.BeaconBlock
+	var block *ethpb.SignedBeaconBlock
 	err := k.db.View(func(tx *bolt.Tx) error {
 		bkt := tx.Bucket(blocksBucket)
 		root := bkt.Get(genesisBlockRootKey)
@@ -336,7 +336,7 @@ func (k *Store) GenesisBlock(ctx context.Context) (*ethpb.BeaconBlock, error) {
 		if enc == nil {
 			return nil
 		}
-		block = &ethpb.BeaconBlock{}
+		block = &ethpb.SignedBeaconBlock{}
 		return decode(enc, block)
 	})
 	return block, err

beacon-chain/db/kv/blocks_test.go

@@ -20,16 +20,20 @@ func TestStore_SaveBlock_NoDuplicates(t *testing.T) {
 	slot := uint64(20)
 	ctx := context.Background()
 	// First we save a previous block to ensure the cache max size is reached.
-	prevBlock := &ethpb.BeaconBlock{
+	prevBlock := &ethpb.SignedBeaconBlock{
-		Slot:       slot - 1,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: []byte{1, 2, 3},
+			Slot:       slot - 1,
+			ParentRoot: []byte{1, 2, 3},
+		},
 	}
 	if err := db.SaveBlock(ctx, prevBlock); err != nil {
 		t.Fatal(err)
 	}
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot:       slot,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: []byte{1, 2, 3},
+			Slot:       slot,
+			ParentRoot: []byte{1, 2, 3},
+		},
 	}
 	// Even with a full cache, saving new blocks should not cause
 	// duplicated blocks in the DB.
@@ -54,11 +58,13 @@ func TestStore_BlocksCRUD(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
 	ctx := context.Background()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot:       20,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: []byte{1, 2, 3},
+			Slot:       20,
+			ParentRoot: []byte{1, 2, 3},
+		},
 	}
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -95,16 +101,19 @@ func TestStore_BlocksBatchDelete(t *testing.T) {
 	defer teardownDB(t, db)
 	ctx := context.Background()
 	numBlocks := 1000
-	totalBlocks := make([]*ethpb.BeaconBlock, numBlocks)
+	totalBlocks := make([]*ethpb.SignedBeaconBlock, numBlocks)
 	blockRoots := make([][32]byte, 0)
-	oddBlocks := make([]*ethpb.BeaconBlock, 0)
+	oddBlocks := make([]*ethpb.SignedBeaconBlock, 0)
 	for i := 0; i < len(totalBlocks); i++ {
-		totalBlocks[i] = &ethpb.BeaconBlock{
+		totalBlocks[i] = &ethpb.SignedBeaconBlock{
-			Slot:       uint64(i),
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent"),
+				Slot:       uint64(i),
+				ParentRoot: []byte("parent"),
+			},
 		}
+
 		if i%2 == 0 {
-			r, err := ssz.SigningRoot(totalBlocks[i])
+			r, err := ssz.HashTreeRoot(totalBlocks[i].Block)
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -133,7 +142,7 @@ func TestStore_BlocksBatchDelete(t *testing.T) {
 		t.Fatal(err)
 	}
 	sort.Slice(retrieved, func(i, j int) bool {
-		return retrieved[i].Slot < retrieved[j].Slot
+		return retrieved[i].Block.Slot < retrieved[j].Block.Slot
 	})
 	if !reflect.DeepEqual(retrieved, oddBlocks) {
 		t.Errorf("Wanted %v, received %v", oddBlocks, retrieved)
@@ -144,11 +153,13 @@ func TestStore_GenesisBlock(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
 	ctx := context.Background()
-	genesisBlock := &ethpb.BeaconBlock{
+	genesisBlock := &ethpb.SignedBeaconBlock{
-		Slot:       0,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: []byte{1, 2, 3},
+			Slot:       0,
+			ParentRoot: []byte{1, 2, 3},
+		},
 	}
-	blockRoot, err := ssz.SigningRoot(genesisBlock)
+	blockRoot, err := ssz.HashTreeRoot(genesisBlock.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -171,11 +182,13 @@ func TestStore_BlocksCRUD_NoCache(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
 	ctx := context.Background()
-	block := &ethpb.BeaconBlock{
+	block := &ethpb.SignedBeaconBlock{
-		Slot:       20,
+		Block: &ethpb.BeaconBlock{
-		ParentRoot: []byte{1, 2, 3},
+			Slot:       20,
+			ParentRoot: []byte{1, 2, 3},
+		},
 	}
-	blockRoot, err := ssz.SigningRoot(block)
+	blockRoot, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -211,26 +224,36 @@ func TestStore_BlocksCRUD_NoCache(t *testing.T) {
 func TestStore_Blocks_FiltersCorrectly(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
-	blocks := []*ethpb.BeaconBlock{
+	blocks := []*ethpb.SignedBeaconBlock{
 		{
-			Slot:       4,
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent"),
+				Slot:       4,
+				ParentRoot: []byte("parent"),
+			},
 		},
 		{
-			Slot:       5,
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent2"),
+				Slot:       5,
+				ParentRoot: []byte("parent2"),
+			},
 		},
 		{
-			Slot:       6,
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent2"),
+				Slot:       6,
+				ParentRoot: []byte("parent2"),
+			},
 		},
 		{
-			Slot:       7,
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent3"),
+				Slot:       7,
+				ParentRoot: []byte("parent3"),
+			},
 		},
 		{
-			Slot:       8,
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent4"),
+				Slot:       8,
+				ParentRoot: []byte("parent4"),
+			},
 		},
 	}
 	ctx := context.Background()
@@ -307,11 +330,13 @@ func TestStore_Blocks_FiltersCorrectly(t *testing.T) {
 func TestStore_Blocks_Retrieve_SlotRange(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
-	b := make([]*ethpb.BeaconBlock, 500)
+	b := make([]*ethpb.SignedBeaconBlock, 500)
 	for i := 0; i < 500; i++ {
-		b[i] = &ethpb.BeaconBlock{
+		b[i] = &ethpb.SignedBeaconBlock{
-			ParentRoot: []byte("parent"),
+			Block: &ethpb.BeaconBlock{
-			Slot:       uint64(i),
+				ParentRoot: []byte("parent"),
+				Slot:       uint64(i),
+			},
 		}
 	}
 	ctx := context.Background()
@@ -332,11 +357,13 @@ func TestStore_Blocks_Retrieve_Epoch(t *testing.T) {
 	db := setupDB(t)
 	defer teardownDB(t, db)
 	slots := params.BeaconConfig().SlotsPerEpoch * 7
-	b := make([]*ethpb.BeaconBlock, slots)
+	b := make([]*ethpb.SignedBeaconBlock, slots)
 	for i := uint64(0); i < slots; i++ {
-		b[i] = &ethpb.BeaconBlock{
+		b[i] = &ethpb.SignedBeaconBlock{
-			ParentRoot: []byte("parent"),
+			Block: &ethpb.BeaconBlock{
-			Slot:       i,
+				ParentRoot: []byte("parent"),
+				Slot:       i,
+			},
 		}
 	}
 	ctx := context.Background()

beacon-chain/db/kv/checkpoint_test.go

@@ -48,12 +48,14 @@ func TestStore_FinalizedCheckpoint_CanSaveRetrieve(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	blk := &ethpb.BeaconBlock{
+	blk := &ethpb.SignedBeaconBlock{
-		ParentRoot: genesis[:],
+		Block: &ethpb.BeaconBlock{
-		Slot:       40,
+			ParentRoot: genesis[:],
+			Slot:       40,
+		},
 	}
 
-	root, err := ssz.SigningRoot(blk)
+	root, err := ssz.HashTreeRoot(blk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/db/kv/encoding.go

@@ -3,16 +3,12 @@ package kv
 import (
 	"github.com/gogo/protobuf/proto"
 	"github.com/golang/snappy"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 )
 
 func decode(data []byte, dst proto.Message) error {
-	if featureconfig.Get().EnableSnappyDBCompression {
+	data, err := snappy.Decode(nil, data)
-		var err error
+	if err != nil {
-		data, err = snappy.Decode(nil, data)
+		return err
-		if err != nil {
-			return err
-		}
 	}
 	if err := proto.Unmarshal(data, dst); err != nil {
 		return err
@@ -26,9 +22,5 @@ func encode(msg proto.Message) ([]byte, error) {
 		return nil, err
 	}
 
-	if !featureconfig.Get().EnableSnappyDBCompression {
-		return enc, nil
-	}
-
 	return snappy.Encode(nil, enc), nil
 }

beacon-chain/db/kv/finalized_block_roots.go

@@ -77,16 +77,17 @@ func (kv *Store) updateFinalizedBlockRoots(ctx context.Context, tx *bolt.Tx, che
 			break
 		}
 
-		block, err := kv.Block(ctx, bytesutil.ToBytes32(root))
+		signedBlock, err := kv.Block(ctx, bytesutil.ToBytes32(root))
 		if err != nil {
 			traceutil.AnnotateError(span, err)
 			return err
 		}
-		if block == nil {
+		if signedBlock == nil || signedBlock.Block == nil {
 			err := fmt.Errorf("missing block in database: block root=%#x", root)
 			traceutil.AnnotateError(span, err)
 			return err
 		}
+		block := signedBlock.Block
 
 		container := &dbpb.FinalizedBlockRootContainer{
 			ParentRoot: block.ParentRoot,

beacon-chain/db/kv/finalized_block_roots_test.go

@@ -28,7 +28,7 @@ func TestStore_IsFinalizedBlock(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	root, err := ssz.SigningRoot(blks[slotsPerEpoch])
+	root, err := ssz.HashTreeRoot(blks[slotsPerEpoch].Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -49,7 +49,7 @@ func TestStore_IsFinalizedBlock(t *testing.T) {
 
 	// All blocks up to slotsPerEpoch*2 should be in the finalized index.
 	for i := 0; i < slotsPerEpoch*2; i++ {
-		root, err := ssz.SigningRoot(blks[i])
+		root, err := ssz.HashTreeRoot(blks[i].Block)
 		if err != nil {
 			t.Fatal(err)
 		}
@@ -58,7 +58,7 @@ func TestStore_IsFinalizedBlock(t *testing.T) {
 		}
 	}
 	for i := slotsPerEpoch * 3; i < len(blks); i++ {
-		root, err := ssz.SigningRoot(blks[i])
+		root, err := ssz.HashTreeRoot(blks[i].Block)
 		if err != nil {
 			t.Fatal(err)
 		}
@@ -152,25 +152,27 @@ func TestStore_IsFinalized_ForkEdgeCase(t *testing.T) {
 	}
 }
 
-func sszRootOrDie(t *testing.T, block *ethpb.BeaconBlock) []byte {
+func sszRootOrDie(t *testing.T, block *ethpb.SignedBeaconBlock) []byte {
-	root, err := ssz.SigningRoot(block)
+	root, err := ssz.HashTreeRoot(block.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
 	return root[:]
 }
 
-func makeBlocks(t *testing.T, i, n int, previousRoot [32]byte) []*ethpb.BeaconBlock {
+func makeBlocks(t *testing.T, i, n int, previousRoot [32]byte) []*ethpb.SignedBeaconBlock {
-	blocks := make([]*ethpb.BeaconBlock, n)
+	blocks := make([]*ethpb.SignedBeaconBlock, n)
 	for j := i; j < n+i; j++ {
 		parentRoot := make([]byte, 32)
 		copy(parentRoot, previousRoot[:])
-		blocks[j-i] = &ethpb.BeaconBlock{
+		blocks[j-i] = &ethpb.SignedBeaconBlock{
-			Slot:       uint64(j + 1),
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: parentRoot,
+				Slot:       uint64(j + 1),
+				ParentRoot: parentRoot,
+			},
 		}
 		var err error
-		previousRoot, err = ssz.SigningRoot(blocks[j-i])
+		previousRoot, err = ssz.HashTreeRoot(blocks[j-i].Block)
 		if err != nil {
 			t.Fatal(err)
 		}

beacon-chain/db/kv/kv.go

@@ -93,6 +93,7 @@ func NewKVStore(dirPath string) (*Store, error) {
 			archivedCommitteeInfoBucket,
 			archivedBalancesBucket,
 			archivedValidatorParticipationBucket,
+			powchainBucket,
 			// Indices buckets.
 			attestationHeadBlockRootBucket,
 			attestationSourceRootIndicesBucket,
@@ -109,10 +110,6 @@ func NewKVStore(dirPath string) (*Store, error) {
 		return nil, err
 	}
 
-	if err := kv.ensureSnappy(); err != nil {
-		return nil, err
-	}
-
 	if err := kv.pruneStates(context.TODO()); err != nil {
 		return nil, err
 	}

beacon-chain/db/kv/migrate_snappy.go

@@ -1,73 +0,0 @@
-package kv
-
-import (
-	"errors"
-
-	"github.com/boltdb/bolt"
-	"github.com/golang/snappy"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-	"github.com/sirupsen/logrus"
-)
-
-var snappyKey = []byte("snappy")
-
-func (kv *Store) ensureSnappy() error {
-	var isMigrated bool
-
-	kv.db.View(func(tx *bolt.Tx) error {
-		bkt := tx.Bucket(migrationBucket)
-		v := bkt.Get(snappyKey)
-		isMigrated = len(v) == 1 && v[0] == 0x01
-		return nil
-	})
-
-	if !featureconfig.Get().EnableSnappyDBCompression {
-		if isMigrated {
-			return errors.New("beaconDB has been migrated to snappy compression, run with flag --snappy")
-		}
-		return nil
-	}
-
-	if isMigrated {
-		return nil
-	}
-
-	log := logrus.WithField("prefix", "kv")
-	log.Info("Compressing database to snappy compression. This might take a while...")
-
-	bucketsToMigrate := [][]byte{
-		attestationsBucket,
-		blocksBucket,
-		stateBucket,
-		proposerSlashingsBucket,
-		attesterSlashingsBucket,
-		voluntaryExitsBucket,
-		checkpointBucket,
-		archivedValidatorSetChangesBucket,
-		archivedCommitteeInfoBucket,
-		archivedBalancesBucket,
-		archivedValidatorParticipationBucket,
-		finalizedBlockRootsIndexBucket,
-	}
-
-	return kv.db.Update(func(tx *bolt.Tx) error {
-		for _, b := range bucketsToMigrate {
-			log.WithField("bucket", string(b)).Debug("Compressing bucket.")
-			if err := migrateBucketToSnappy(tx.Bucket(b)); err != nil {
-				return err
-			}
-		}
-		bkt := tx.Bucket(migrationBucket)
-		return bkt.Put(snappyKey, []byte{0x01})
-	})
-}
-
-func migrateBucketToSnappy(bkt *bolt.Bucket) error {
-	c := bkt.Cursor()
-	for key, val := c.First(); key != nil; key, val = c.Next() {
-		if err := bkt.Put(key, snappy.Encode(nil, val)); err != nil {
-			return err
-		}
-	}
-	return nil
-}

beacon-chain/db/kv/migrate_snappy_test.go

@@ -1,48 +0,0 @@
-package kv
-
-import (
-	"context"
-	"testing"
-
-	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	"github.com/prysmaticlabs/go-ssz"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-)
-
-// Sanity check that an object can be accessed after migration.
-func TestStore_MigrateSnappy(t *testing.T) {
-	db := setupDB(t)
-	ctx := context.Background()
-
-	block := &ethpb.BeaconBlock{
-		Slot: 200,
-	}
-	root, err := ssz.SigningRoot(block)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := db.SaveBlock(ctx, block); err != nil {
-		t.Fatal(err)
-	}
-	path := db.databasePath
-	db.Close()
-
-	c := featureconfig.Get()
-	c.EnableSnappyDBCompression = true
-	featureconfig.Init(c)
-
-	db2, err := NewKVStore(path)
-	if err != nil {
-		t.Fatalf("Failed to instantiate DB: %v", err)
-	}
-	defer teardownDB(t, db2)
-
-	blk, err := db.Block(ctx, root)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	if !ssz.DeepEqual(blk, block) {
-		t.Fatal("Blocks not same")
-	}
-}

beacon-chain/db/kv/operations.go

@@ -44,7 +44,7 @@ func (k *Store) HasVoluntaryExit(ctx context.Context, exitRoot [32]byte) bool {
 func (k *Store) SaveVoluntaryExit(ctx context.Context, exit *ethpb.VoluntaryExit) error {
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.SaveVoluntaryExit")
 	defer span.End()
-	exitRoot, err := ssz.SigningRoot(exit)
+	exitRoot, err := ssz.HashTreeRoot(exit)
 	if err != nil {
 		return err
 	}

beacon-chain/db/kv/operations_test.go

@@ -16,7 +16,7 @@ func TestStore_VoluntaryExits_CRUD(t *testing.T) {
 	exit := &ethpb.VoluntaryExit{
 		Epoch: 5,
 	}
-	exitRoot, err := ssz.SigningRoot(exit)
+	exitRoot, err := ssz.HashTreeRoot(exit)
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/db/kv/powchain.go

@@ -0,0 +1,43 @@
+package kv
+
+import (
+	"context"
+
+	"github.com/boltdb/bolt"
+	"github.com/gogo/protobuf/proto"
+	"github.com/prysmaticlabs/prysm/proto/beacon/db"
+	"go.opencensus.io/trace"
+)
+
+// SavePowchainData saves the pow chain data.
+func (k *Store) SavePowchainData(ctx context.Context, data *db.ETH1ChainData) error {
+	ctx, span := trace.StartSpan(ctx, "BeaconDB.SavePowchainData")
+	defer span.End()
+
+	return k.db.Update(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket(powchainBucket)
+		enc, err := proto.Marshal(data)
+		if err != nil {
+			return err
+		}
+		return bkt.Put(powchainDataKey, enc)
+	})
+}
+
+// PowchainData retrieves the powchain data.
+func (k *Store) PowchainData(ctx context.Context) (*db.ETH1ChainData, error) {
+	ctx, span := trace.StartSpan(ctx, "BeaconDB.PowchainData")
+	defer span.End()
+
+	var data *db.ETH1ChainData
+	err := k.db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket(powchainBucket)
+		enc := bkt.Get(powchainDataKey)
+		if len(enc) == 0 {
+			return nil
+		}
+		data = &db.ETH1ChainData{}
+		return proto.Unmarshal(enc, data)
+	})
+	return data, err
+}

beacon-chain/db/kv/prune_states.go

@@ -7,7 +7,6 @@ import (
 	"github.com/boltdb/bolt"
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 	"github.com/sirupsen/logrus"
 )
 
@@ -16,13 +15,6 @@ var pruneStatesKey = []byte("prune-states")
 func (kv *Store) pruneStates(ctx context.Context) error {
 	var pruned bool
 
-	if !featureconfig.Get().PruneEpochBoundaryStates {
-		return kv.db.Update(func(tx *bolt.Tx) error {
-			bkt := tx.Bucket(migrationBucket)
-			return bkt.Put(pruneStatesKey, []byte{0x00})
-		})
-	}
-
 	kv.db.View(func(tx *bolt.Tx) error {
 		bkt := tx.Bucket(migrationBucket)
 		v := bkt.Get(pruneStatesKey)

beacon-chain/db/kv/prune_states_test.go

@@ -1,67 +0,0 @@
-package kv
-
-import (
-	"context"
-	"testing"
-
-	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	"github.com/prysmaticlabs/go-ssz"
-	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
-)
-
-// Sanity check that states are pruned
-func TestStore_PruneStates(t *testing.T) {
-	db := setupDB(t)
-	ctx := context.Background()
-
-	numBlocks := 33
-	blocks := make([]*ethpb.BeaconBlock, numBlocks)
-	blockRoots := make([][32]byte, 0)
-	for i := 0; i < len(blocks); i++ {
-		blocks[i] = &ethpb.BeaconBlock{
-			Slot: uint64(i),
-		}
-		r, err := ssz.SigningRoot(blocks[i])
-		if err != nil {
-			t.Fatal(err)
-		}
-		if err := db.SaveState(ctx, &pb.BeaconState{Slot: uint64(i)}, r); err != nil {
-			t.Fatal(err)
-		}
-		if err := db.SaveBlock(ctx, blocks[i]); err != nil {
-			t.Fatal(err)
-		}
-		blockRoots = append(blockRoots, r)
-	}
-	db.SaveFinalizedCheckpoint(ctx, &ethpb.Checkpoint{Epoch: 1, Root: blockRoots[numBlocks-1][:]})
-
-	path := db.databasePath
-	db.Close()
-
-	c := featureconfig.Get()
-	c.PruneEpochBoundaryStates = true
-	featureconfig.Init(c)
-
-	db2, err := NewKVStore(path)
-	if err != nil {
-		t.Fatalf("Failed to instantiate DB: %v", err)
-	}
-	defer teardownDB(t, db2)
-
-	s, err := db2.State(ctx, blockRoots[31])
-	if err != nil {
-		t.Fatal(err)
-	}
-	if s == nil {
-		t.Error("finalized state should not be deleted")
-	}
-
-	s, err = db2.State(ctx, blockRoots[30])
-	if err != nil {
-		t.Fatal(err)
-	}
-	if s != nil {
-		t.Error("regular state should be deleted")
-	}
-}

beacon-chain/db/kv/schema.go

@@ -20,6 +20,7 @@ var (
 	archivedCommitteeInfoBucket          = []byte("archived-committee-info")
 	archivedBalancesBucket               = []byte("archived-balances")
 	archivedValidatorParticipationBucket = []byte("archived-validator-participation")
+	powchainBucket                       = []byte("powchain")
 
 	// Key indices buckets.
 	blockParentRootIndicesBucket        = []byte("block-parent-root-indices")
@@ -37,6 +38,7 @@ var (
 	depositContractAddressKey = []byte("deposit-contract")
 	justifiedCheckpointKey    = []byte("justified-checkpoint")
 	finalizedCheckpointKey    = []byte("finalized-checkpoint")
+	powchainDataKey           = []byte("powchain-data")
 
 	// Migration bucket.
 	migrationBucket = []byte("migrations")

beacon-chain/db/kv/state_test.go

@@ -109,15 +109,17 @@ func TestStore_StatesBatchDelete(t *testing.T) {
 	defer teardownDB(t, db)
 	ctx := context.Background()
 	numBlocks := 100
-	totalBlocks := make([]*ethpb.BeaconBlock, numBlocks)
+	totalBlocks := make([]*ethpb.SignedBeaconBlock, numBlocks)
 	blockRoots := make([][32]byte, 0)
 	evenBlockRoots := make([][32]byte, 0)
 	for i := 0; i < len(totalBlocks); i++ {
-		totalBlocks[i] = &ethpb.BeaconBlock{
+		totalBlocks[i] = &ethpb.SignedBeaconBlock{
-			Slot:       uint64(i),
+			Block: &ethpb.BeaconBlock{
-			ParentRoot: []byte("parent"),
+				Slot:       uint64(i),
+				ParentRoot: []byte("parent"),
+			},
 		}
-		r, err := ssz.SigningRoot(totalBlocks[i])
+		r, err := ssz.HashTreeRoot(totalBlocks[i].Block)
 		if err != nil {
 			t.Fatal(err)
 		}
@@ -180,15 +182,17 @@ func TestStore_DeleteFinalizedState(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	blk := &ethpb.BeaconBlock{
+	blk := &ethpb.SignedBeaconBlock{
-		ParentRoot: genesis[:],
+		Block: &ethpb.BeaconBlock{
-		Slot:       100,
+			ParentRoot: genesis[:],
+			Slot:       100,
+		},
 	}
 	if err := db.SaveBlock(ctx, blk); err != nil {
 		t.Fatal(err)
 	}
 
-	finalizedBlockRoot, err := ssz.SigningRoot(blk)
+	finalizedBlockRoot, err := ssz.HashTreeRoot(blk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -217,15 +221,17 @@ func TestStore_DeleteHeadState(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	blk := &ethpb.BeaconBlock{
+	blk := &ethpb.SignedBeaconBlock{
-		ParentRoot: genesis[:],
+		Block: &ethpb.BeaconBlock{
-		Slot:       100,
+			ParentRoot: genesis[:],
+			Slot:       100,
+		},
 	}
 	if err := db.SaveBlock(ctx, blk); err != nil {
 		t.Fatal(err)
 	}
 
-	headBlockRoot, err := ssz.SigningRoot(blk)
+	headBlockRoot, err := ssz.HashTreeRoot(blk.Block)
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/interop-cold-start/BUILD.bazel

@@ -17,6 +17,7 @@ go_library(
         "//shared:go_default_library",
         "//shared/bytesutil:go_default_library",
         "//shared/interop:go_default_library",
+        "//shared/stateutil:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
         "@com_github_prysmaticlabs_go_ssz//:go_default_library",

beacon-chain/interop-cold-start/service.go

@@ -16,6 +16,7 @@ import (
 	"github.com/prysmaticlabs/prysm/shared"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/interop"
+	"github.com/prysmaticlabs/prysm/shared/stateutil"
 )
 
 var _ = shared.Service(&Service{})
@@ -122,6 +123,11 @@ func (s *Service) ChainStartEth1Data() *ethpb.Eth1Data {
 	return &ethpb.Eth1Data{}
 }
 
+// PreGenesisState returns an empty beacon state.
+func (s *Service) PreGenesisState() *pb.BeaconState {
+	return &pb.BeaconState{}
+}
+
 // DepositByPubkey mocks out the deposit cache functionality for interop.
 func (s *Service) DepositByPubkey(ctx context.Context, pubKey []byte) (*ethpb.Deposit, *big.Int) {
 	return &ethpb.Deposit{}, big.NewInt(1)
@@ -134,12 +140,12 @@ func (s *Service) DepositsNumberAndRootAtHeight(ctx context.Context, blockHeight
 
 func (s *Service) saveGenesisState(ctx context.Context, genesisState *pb.BeaconState) error {
 	s.chainStartDeposits = make([]*ethpb.Deposit, len(genesisState.Validators))
-	stateRoot, err := ssz.HashTreeRoot(genesisState)
+	stateRoot, err := stateutil.HashTreeRootState(genesisState)
 	if err != nil {
 		return errors.Wrap(err, "could not tree hash genesis state")
 	}
 	genesisBlk := blocks.NewGenesisBlock(stateRoot[:])
-	genesisBlkRoot, err := ssz.SigningRoot(genesisBlk)
+	genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
 	if err != nil {
 		return errors.Wrap(err, "could not get genesis block root")
 	}

beacon-chain/node/BUILD.bazel

@@ -16,7 +16,7 @@ go_library(
         "//beacon-chain/flags:go_default_library",
         "//beacon-chain/gateway:go_default_library",
         "//beacon-chain/interop-cold-start:go_default_library",
-        "//beacon-chain/operations:go_default_library",
+        "//beacon-chain/operations/attestations:go_default_library",
         "//beacon-chain/p2p:go_default_library",
         "//beacon-chain/powchain:go_default_library",
         "//beacon-chain/rpc:go_default_library",

beacon-chain/node/node.go

@@ -23,7 +23,7 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/flags"
 	"github.com/prysmaticlabs/prysm/beacon-chain/gateway"
 	interopcoldstart "github.com/prysmaticlabs/prysm/beacon-chain/interop-cold-start"
-	"github.com/prysmaticlabs/prysm/beacon-chain/operations"
+	"github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations"
 	"github.com/prysmaticlabs/prysm/beacon-chain/p2p"
 	"github.com/prysmaticlabs/prysm/beacon-chain/powchain"
 	"github.com/prysmaticlabs/prysm/beacon-chain/rpc"
@@ -52,13 +52,15 @@ const testSkipPowFlag = "test-skip-pow"
 // full PoS node. It handles the lifecycle of the entire system and registers
 // services to a service registry.
 type BeaconNode struct {
-	ctx          *cli.Context
+	ctx             *cli.Context
-	services     *shared.ServiceRegistry
+	services        *shared.ServiceRegistry
-	lock         sync.RWMutex
+	lock            sync.RWMutex
-	stop         chan struct{} // Channel to wait for termination notifications.
+	stop            chan struct{} // Channel to wait for termination notifications.
-	db           db.Database
+	db              db.Database
-	depositCache *depositcache.DepositCache
+	attestationPool attestations.Pool
-	stateFeed    *event.Feed
+	depositCache    *depositcache.DepositCache
+	stateFeed       *event.Feed
+	opFeed          *event.Feed
 }
 
 // NewBeaconNode creates a new node instance, sets up configuration options, and registers
@@ -77,13 +79,6 @@ func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
 	flags.ConfigureGlobalFlags(ctx)
 	registry := shared.NewServiceRegistry()
 
-	beacon := &BeaconNode{
-		ctx:       ctx,
-		services:  registry,
-		stop:      make(chan struct{}),
-		stateFeed: new(event.Feed),
-	}
-
 	// Use custom config values if the --no-custom-config flag is not set.
 	if !ctx.GlobalBool(flags.NoCustomConfigFlag.Name) {
 		if featureconfig.Get().MinimalConfig {
@@ -99,6 +94,15 @@ func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
 		}
 	}
 
+	beacon := &BeaconNode{
+		ctx:             ctx,
+		services:        registry,
+		stop:            make(chan struct{}),
+		stateFeed:       new(event.Feed),
+		opFeed:          new(event.Feed),
+		attestationPool: attestations.NewPool(),
+	}
+
 	if err := beacon.startDB(ctx); err != nil {
 		return nil, err
 	}
@@ -111,7 +115,7 @@ func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
 		return nil, err
 	}
 
-	if err := beacon.registerOperationService(ctx); err != nil {
+	if err := beacon.registerAttestationPool(ctx); err != nil {
 		return nil, err
 	}
 
@@ -157,6 +161,11 @@ func (b *BeaconNode) StateFeed() *event.Feed {
 	return b.stateFeed
 }
 
+// OperationFeed implements opfeed.Notifier.
+func (b *BeaconNode) OperationFeed() *event.Feed {
+	return b.opFeed
+}
+
 // Start the BeaconNode and kicks off every registered service.
 func (b *BeaconNode) Start() {
 	b.lock.Lock()
@@ -287,17 +296,13 @@ func (b *BeaconNode) registerBlockchainService(ctx *cli.Context) error {
 	if err := b.services.FetchService(&web3Service); err != nil {
 		return err
 	}
-	var opsService *operations.Service
-	if err := b.services.FetchService(&opsService); err != nil {
-		return err
-	}
 
 	maxRoutines := ctx.GlobalInt64(cmd.MaxGoroutines.Name)
 	blockchainService, err := blockchain.NewService(context.Background(), &blockchain.Config{
 		BeaconDB:          b.db,
 		DepositCache:      b.depositCache,
 		ChainStartFetcher: web3Service,
-		OpsPoolService:    opsService,
+		AttPool:           b.attestationPool,
 		P2p:               b.fetchP2P(ctx),
 		MaxRoutines:       maxRoutines,
 		StateNotifier:     b,
@@ -308,12 +313,14 @@ func (b *BeaconNode) registerBlockchainService(ctx *cli.Context) error {
 	return b.services.RegisterService(blockchainService)
 }
 
-func (b *BeaconNode) registerOperationService(ctx *cli.Context) error {
+func (b *BeaconNode) registerAttestationPool(ctx *cli.Context) error {
-	operationService := operations.NewService(context.Background(), &operations.Config{
+	attPoolService, err := attestations.NewService(context.Background(), &attestations.Config{
-		BeaconDB: b.db,
+		Pool: b.attestationPool,
 	})
-
+	if err != nil {
-	return b.services.RegisterService(operationService)
+		return err
+	}
+	return b.services.RegisterService(attPoolService)
 }
 
 func (b *BeaconNode) registerPOWChainService(cliCtx *cli.Context) error {
@@ -362,11 +369,6 @@ func (b *BeaconNode) registerPOWChainService(cliCtx *cli.Context) error {
 }
 
 func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
-	var operationService *operations.Service
-	if err := b.services.FetchService(&operationService); err != nil {
-		return err
-	}
-
 	var web3Service *powchain.Service
 	if err := b.services.FetchService(&web3Service); err != nil {
 		return err
@@ -385,10 +387,10 @@ func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
 	rs := prysmsync.NewRegularSync(&prysmsync.Config{
 		DB:            b.db,
 		P2P:           b.fetchP2P(ctx),
-		Operations:    operationService,
 		Chain:         chainService,
 		InitialSync:   initSync,
 		StateNotifier: b,
+		AttPool:       b.attestationPool,
 	})
 
 	return b.services.RegisterService(rs)
@@ -418,11 +420,6 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
 		return err
 	}
 
-	var operationService *operations.Service
-	if err := b.services.FetchService(&operationService); err != nil {
-		return err
-	}
-
 	var web3Service *powchain.Service
 	if err := b.services.FetchService(&web3Service); err != nil {
 		return err
@@ -466,8 +463,7 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
 		BlockReceiver:         chainService,
 		AttestationReceiver:   chainService,
 		GenesisTimeFetcher:    chainService,
-		AttestationsPool:      operationService,
+		AttestationsPool:      b.attestationPool,
-		OperationsHandler:     operationService,
 		POWChainService:       web3Service,
 		ChainStartFetcher:     chainStartFetcher,
 		MockEth1Votes:         mockEth1DataVotes,
@@ -475,6 +471,7 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
 		DepositFetcher:        depositFetcher,
 		PendingDepositFetcher: b.depositCache,
 		StateNotifier:         b,
+		OperationNotifier:     b,
 	})
 
 	return b.services.RegisterService(rpcService)

beacon-chain/operations/BUILD.bazel

@@ -1,63 +0,0 @@
-load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
-
-go_library(
-    name = "go_default_library",
-    srcs = [
-        "attestation.go",
-        "block.go",
-        "exit.go",
-        "log.go",
-        "recent_att_multi_map.go",
-        "service.go",
-    ],
-    importpath = "github.com/prysmaticlabs/prysm/beacon-chain/operations",
-    visibility = ["//beacon-chain:__subpackages__"],
-    deps = [
-        "//beacon-chain/core/blocks:go_default_library",
-        "//beacon-chain/core/helpers:go_default_library",
-        "//beacon-chain/core/state:go_default_library",
-        "//beacon-chain/db:go_default_library",
-        "//proto/beacon/db:go_default_library",
-        "//shared/event:go_default_library",
-        "//shared/hashutil:go_default_library",
-        "//shared/messagehandler:go_default_library",
-        "//shared/params:go_default_library",
-        "//shared/traceutil:go_default_library",
-        "@com_github_dgraph_io_ristretto//:go_default_library",
-        "@com_github_gogo_protobuf//proto:go_default_library",
-        "@com_github_pkg_errors//:go_default_library",
-        "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
-        "@com_github_prysmaticlabs_go_bitfield//:go_default_library",
-        "@com_github_prysmaticlabs_go_ssz//:go_default_library",
-        "@com_github_sirupsen_logrus//:go_default_library",
-        "@io_opencensus_go//trace:go_default_library",
-    ],
-)
-
-go_test(
-    name = "go_default_test",
-    size = "small",
-    srcs = [
-        "attestation_test.go",
-        "block_test.go",
-        "exit_test.go",
-        "recent_att_multi_map_test.go",
-        "service_test.go",
-    ],
-    embed = [":go_default_library"],
-    deps = [
-        "//beacon-chain/core/blocks:go_default_library",
-        "//beacon-chain/core/helpers:go_default_library",
-        "//beacon-chain/db/testing:go_default_library",
-        "//proto/beacon/db:go_default_library",
-        "//proto/beacon/p2p/v1:go_default_library",
-        "//shared/bls:go_default_library",
-        "//shared/hashutil:go_default_library",
-        "//shared/params:go_default_library",
-        "//shared/testutil:go_default_library",
-        "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
-        "@com_github_prysmaticlabs_go_bitfield//:go_default_library",
-        "@com_github_prysmaticlabs_go_ssz//:go_default_library",
-        "@com_github_sirupsen_logrus//hooks/test:go_default_library",
-    ],
-)

beacon-chain/operations/attestation.go

@@ -1,206 +0,0 @@
-package operations
-
-import (
-	"context"
-	"sync"
-
-	"github.com/gogo/protobuf/proto"
-	"github.com/pkg/errors"
-	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	"github.com/prysmaticlabs/go-ssz"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
-	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
-	"github.com/prysmaticlabs/prysm/shared/params"
-	"github.com/prysmaticlabs/prysm/shared/traceutil"
-	"go.opencensus.io/trace"
-)
-
-// Pool defines an interface for fetching the list of attestations
-// which have been observed by the beacon node but not yet included in
-// a beacon block by a proposer.
-type Pool interface {
-	AttestationPool(ctx context.Context, requestedSlot uint64) ([]*ethpb.Attestation, error)
-	AttestationPoolNoVerify(ctx context.Context) ([]*ethpb.Attestation, error)
-	AttestationPoolForForkchoice(ctx context.Context) ([]*ethpb.Attestation, error)
-	AttestationsBySlotCommittee(ctx context.Context, slot uint64, index uint64) ([]*ethpb.Attestation, error)
-}
-
-// Handler defines an interface for a struct equipped for receiving block operations.
-type Handler interface {
-	HandleAttestation(context.Context, proto.Message) error
-}
-
-// retrieves a lock for the specific data root.
-func (s *Service) retrieveLock(key [32]byte) *sync.Mutex {
-	keyString := string(key[:])
-	mutex := &sync.Mutex{}
-	item, ok := s.attestationLockCache.Get(keyString)
-	if !ok {
-		s.attestationLockCache.Set(keyString, mutex, 1)
-		return mutex
-	}
-	return item.(*sync.Mutex)
-}
-
-// AttestationPoolForForkchoice returns the attestations that have not been processed by the
-// fork choice service. It will not return the attestations which the validator vote has
-// already been counted.
-func (s *Service) AttestationPoolForForkchoice(ctx context.Context) ([]*ethpb.Attestation, error) {
-	s.attestationPoolLock.Lock()
-	defer s.attestationPoolLock.Unlock()
-
-	atts := make([]*ethpb.Attestation, 0, len(s.attestationPool))
-
-	for root, ac := range s.attestationPool {
-		for i, att := range ac.ToAttestations() {
-			if ac.SignaturePairs[i].VoteCounted {
-				continue
-			}
-			if s.recentAttestationBitlist.Contains(root, att.AggregationBits) {
-				continue
-			}
-			atts = append(atts, att)
-			ac.SignaturePairs[i].VoteCounted = true
-		}
-	}
-
-	return atts, nil
-}
-
-// AttestationPool returns the attestations that have not seen on the beacon chain,
-// the attestations are returned in target epoch ascending order and up to MaxAttestations
-// capacity. The attestations returned will be verified against the head state up to requested slot.
-// When fails attestation, the attestation will be removed from the pool.
-func (s *Service) AttestationPool(ctx context.Context, requestedSlot uint64) ([]*ethpb.Attestation, error) {
-	ctx, span := trace.StartSpan(ctx, "operations.AttestationPool")
-	defer span.End()
-
-	s.attestationPoolLock.Lock()
-	defer s.attestationPoolLock.Unlock()
-
-	atts := make([]*ethpb.Attestation, 0, len(s.attestationPool))
-
-	bState, err := s.beaconDB.HeadState(ctx)
-	if err != nil {
-		return nil, errors.New("could not retrieve attestations from DB")
-	}
-
-	if bState.Slot < requestedSlot {
-		bState, err = state.ProcessSlots(ctx, bState, requestedSlot)
-		if err != nil {
-			return nil, errors.Wrapf(err, "could not process slots up to %d", requestedSlot)
-		}
-	}
-
-	var validAttsCount uint64
-	for root, ac := range s.attestationPool {
-		for _, att := range ac.ToAttestations() {
-			if s.recentAttestationBitlist.Contains(root, att.AggregationBits) {
-				continue
-			}
-			if _, err = blocks.ProcessAttestation(ctx, bState, att); err != nil {
-				delete(s.attestationPool, root)
-				continue
-			}
-
-			validAttsCount++
-			// Stop the max attestation number per beacon block is reached.
-			if validAttsCount == params.BeaconConfig().MaxAttestations {
-				break
-			}
-
-			atts = append(atts, att)
-		}
-	}
-
-	return atts, nil
-}
-
-// AttestationPoolNoVerify returns every attestation from the attestation pool.
-func (s *Service) AttestationPoolNoVerify(ctx context.Context) ([]*ethpb.Attestation, error) {
-	s.attestationPoolLock.Lock()
-	defer s.attestationPoolLock.Unlock()
-
-	atts := make([]*ethpb.Attestation, 0, len(s.attestationPool))
-
-	for _, ac := range s.attestationPool {
-		atts = append(atts, ac.ToAttestations()...)
-	}
-
-	return atts, nil
-}
-
-// AttestationsBySlotCommittee returns the attestations from the attestations pool filtered
-// by slot and committee index.
-func (s *Service) AttestationsBySlotCommittee(ctx context.Context, slot uint64, index uint64) ([]*ethpb.Attestation, error) {
-	ctx, span := trace.StartSpan(ctx, "operations.AttestationsBySlotCommittee")
-	defer span.End()
-
-	s.attestationPoolLock.RLock()
-	defer s.attestationPoolLock.RUnlock()
-
-	atts := make([]*ethpb.Attestation, 0, len(s.attestationPool))
-
-	for _, ac := range s.attestationPool {
-		if ac.Data.Slot == slot && ac.Data.CommitteeIndex == index {
-			atts = append(atts, ac.ToAttestations()...)
-		}
-	}
-
-	return atts, nil
-}
-
-// HandleAttestation processes a received attestation message.
-func (s *Service) HandleAttestation(ctx context.Context, message proto.Message) error {
-	ctx, span := trace.StartSpan(ctx, "operations.HandleAttestation")
-	defer span.End()
-
-	s.attestationPoolLock.Lock()
-	defer s.attestationPoolLock.Unlock()
-
-	attestation := message.(*ethpb.Attestation)
-	root, err := ssz.HashTreeRoot(attestation.Data)
-	if err != nil {
-		traceutil.AnnotateError(span, err)
-		return err
-	}
-
-	if s.recentAttestationBitlist.Contains(root, attestation.AggregationBits) {
-		log.Debug("Attestation aggregation bits already included recently")
-		return nil
-	}
-
-	ac, ok := s.attestationPool[root]
-	if !ok {
-		s.attestationPool[root] = dbpb.NewContainerFromAttestations([]*ethpb.Attestation{attestation})
-		return nil
-	}
-
-	// Container already has attestation(s) that fully contain the the aggregation bits of this new
-	// attestation so there is nothing to insert or aggregate.
-	if ac.Contains(attestation) {
-		log.Debug("Attestation already fully contained in container")
-		return nil
-	}
-
-	beforeAggregation := append(ac.ToAttestations(), attestation)
-
-	// Filter any out attestation that is already fully included.
-	for i, att := range beforeAggregation {
-		if s.recentAttestationBitlist.Contains(root, att.AggregationBits) {
-			beforeAggregation = append(beforeAggregation[:i], beforeAggregation[i+1:]...)
-		}
-	}
-
-	aggregated, err := helpers.AggregateAttestations(beforeAggregation)
-	if err != nil {
-		traceutil.AnnotateError(span, err)
-		return err
-	}
-
-	s.attestationPool[root] = dbpb.NewContainerFromAttestations(aggregated)
-
-	return nil
-}

beacon-chain/operations/attestation_test.go

@@ -1,563 +0,0 @@
-package operations
-
-import (
-	"bytes"
-	"context"
-	"reflect"
-	"sync"
-	"testing"
-
-	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	"github.com/prysmaticlabs/go-bitfield"
-	"github.com/prysmaticlabs/go-ssz"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
-	dbutil "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
-	dbpb "github.com/prysmaticlabs/prysm/proto/beacon/db"
-	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
-	"github.com/prysmaticlabs/prysm/shared/bls"
-	"github.com/prysmaticlabs/prysm/shared/hashutil"
-	"github.com/prysmaticlabs/prysm/shared/params"
-	"github.com/prysmaticlabs/prysm/shared/testutil"
-)
-
-func TestHandleAttestation_Saves_NewAttestation(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	service := NewService(context.Background(), &Config{
-		BeaconDB: beaconDB,
-	})
-
-	beaconState, privKeys := testutil.DeterministicGenesisState(t, 100)
-
-	att := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			BeaconBlockRoot: []byte("block-root"),
-			Source:          &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-			Target:          &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-		},
-		AggregationBits: bitfield.Bitlist{0xCF, 0xC0, 0xC0, 0xC0, 0x01},
-		CustodyBits:     bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-	}
-	committee, err := helpers.BeaconCommitteeFromState(beaconState, att.Data.Slot, att.Data.CommitteeIndex)
-	if err != nil {
-		t.Fatal(err)
-	}
-	attestingIndices, err := helpers.AttestingIndices(att.AggregationBits, committee)
-	if err != nil {
-		t.Error(err)
-	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-		Data:       att.Data,
-		CustodyBit: false,
-	}
-	domain := helpers.Domain(beaconState.Fork, 0, params.BeaconConfig().DomainBeaconAttester)
-	sigs := make([]*bls.Signature, len(attestingIndices))
-	for i, indice := range attestingIndices {
-		hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
-		if err != nil {
-			t.Error(err)
-		}
-		sig := privKeys[indice].Sign(hashTreeRoot[:], domain)
-		sigs[i] = sig
-	}
-	att.Signature = bls.AggregateSignatures(sigs).Marshal()[:]
-
-	beaconState.CurrentJustifiedCheckpoint.Root = []byte("hello-world")
-	beaconState.CurrentEpochAttestations = []*pb.PendingAttestation{}
-
-	newBlock := &ethpb.BeaconBlock{
-		Slot: 0,
-	}
-	newBlockRoot, err := ssz.HashTreeRoot(newBlock)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveBlock(context.Background(), newBlock); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveState(context.Background(), beaconState, newBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(context.Background(), newBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	beaconState.Slot += params.BeaconConfig().MinAttestationInclusionDelay
-
-	if err := service.HandleAttestation(context.Background(), att); err != nil {
-		t.Error(err)
-	}
-}
-
-func TestHandleAttestation_Aggregates_LargeNumValidators(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	ctx := context.Background()
-	opsSrv := NewService(ctx, &Config{
-		BeaconDB: beaconDB,
-	})
-	opsSrv.attestationPool = make(map[[32]byte]*dbpb.AttestationContainer)
-
-	// First, we create a common attestation data.
-	data := &ethpb.AttestationData{
-		Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-		Target: &ethpb.Checkpoint{Epoch: 0},
-	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-		Data:       data,
-		CustodyBit: false,
-	}
-	root, err := ssz.HashTreeRoot(dataAndCustodyBit)
-	if err != nil {
-		t.Error(err)
-	}
-	att := &ethpb.Attestation{
-		Data:        data,
-		CustodyBits: bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-	}
-
-	// We setup the genesis state with 256 validators.
-	beaconState, privKeys := testutil.DeterministicGenesisState(t, 256)
-	stateRoot, err := ssz.HashTreeRoot(beaconState)
-	if err != nil {
-		t.Fatal(err)
-	}
-	block := blocks.NewGenesisBlock(stateRoot[:])
-	blockRoot, err := ssz.HashTreeRoot(block)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveBlock(ctx, block); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveState(ctx, beaconState, blockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(ctx, blockRoot); err != nil {
-		t.Fatal(err)
-	}
-
-	// Next up, we compute the committee for the attestation we're testing.
-	committee, err := helpers.BeaconCommitteeFromState(beaconState, att.Data.Slot, att.Data.CommitteeIndex)
-	if err != nil {
-		t.Fatal(err)
-	}
-	attDataRoot, err := ssz.HashTreeRoot(att.Data)
-	if err != nil {
-		t.Error(err)
-	}
-	totalAggBits := bitfield.NewBitlist(uint64(len(committee)))
-	domain := helpers.Domain(beaconState.Fork, 0, params.BeaconConfig().DomainBeaconAttester)
-
-	// For every single member of the committee, we sign the attestation data and handle
-	// the attestation through the operations service, which will perform basic aggregation
-	// and verification.
-	//
-	// We perform these operations concurrently with a wait group to more closely
-	// emulate a production environment.
-	var wg sync.WaitGroup
-	for i := 0; i < len(committee); i++ {
-		wg.Add(1)
-		go func(tt *testing.T, j int, w *sync.WaitGroup) {
-			defer w.Done()
-			newAtt := &ethpb.Attestation{
-				AggregationBits: bitfield.NewBitlist(uint64(len(committee))),
-				Data:            data,
-				CustodyBits:     bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-				Signature:       privKeys[committee[j]].Sign(root[:], domain).Marshal(),
-			}
-			newAtt.AggregationBits.SetBitAt(uint64(j), true)
-			if err := opsSrv.HandleAttestation(ctx, newAtt); err != nil {
-				tt.Fatalf("Could not handle attestation %d: %v", j, err)
-			}
-			totalAggBits = totalAggBits.Or(newAtt.AggregationBits)
-		}(t, i, &wg)
-	}
-	wg.Wait()
-
-	// We fetch the final attestation from the attestation pool, which should be an aggregation of
-	// all committee members effectively.
-	aggAtt := opsSrv.attestationPool[attDataRoot].ToAttestations()[0]
-	b1 := aggAtt.AggregationBits.Bytes()
-	b2 := totalAggBits.Bytes()
-
-	// We check if the aggregation bits are what we want.
-	if !bytes.Equal(b1, b2) {
-		t.Errorf("Wanted aggregation bytes %v, received %v", b2, b1)
-	}
-
-	// If the committee is larger than 1, the signature from the attestation fetched from the DB
-	// should be an aggregate of signatures and not equal to an individual signature from a validator.
-	if len(committee) > 1 && bytes.Equal(aggAtt.Signature, att.Signature) {
-		t.Errorf("Expected aggregate signature %#x to be different from individual sig %#x", aggAtt.Signature, att.Signature)
-	}
-}
-
-func TestHandleAttestation_Skips_PreviouslyAggregatedAttestations(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-
-	service := NewService(context.Background(), &Config{
-		BeaconDB: beaconDB,
-	})
-	service.attestationPool = make(map[[32]byte]*dbpb.AttestationContainer)
-
-	beaconState, privKeys := testutil.DeterministicGenesisState(t, 200)
-
-	beaconState.CurrentJustifiedCheckpoint.Root = []byte("hello-world")
-
-	att1 := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-		},
-		CustodyBits: bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-	}
-
-	committee, err := helpers.BeaconCommitteeFromState(beaconState, att1.Data.Slot, att1.Data.CommitteeIndex)
-	if err != nil {
-		t.Fatal(err)
-	}
-	aggregationBits := bitfield.NewBitlist(uint64(len(committee)))
-	aggregationBits.SetBitAt(0, true)
-	att1.AggregationBits = aggregationBits
-
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-		Data:       att1.Data,
-		CustodyBit: false,
-	}
-	hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
-	if err != nil {
-		t.Error(err)
-	}
-	domain := helpers.Domain(beaconState.Fork, 0, params.BeaconConfig().DomainBeaconAttester)
-	att1.Signature = privKeys[committee[0]].Sign(hashTreeRoot[:], domain).Marshal()
-
-	att2 := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-			Target: &ethpb.Checkpoint{Epoch: 0},
-		},
-		CustodyBits: bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-	}
-	aggregationBits = bitfield.NewBitlist(uint64(len(committee)))
-	aggregationBits.SetBitAt(1, true)
-	att2.AggregationBits = aggregationBits
-
-	att2.Signature = privKeys[committee[1]].Sign(hashTreeRoot[:], domain).Marshal()
-
-	att3 := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{Epoch: 0, Root: []byte("hello-world")},
-			Target: &ethpb.Checkpoint{Epoch: 0},
-		},
-		CustodyBits: bitfield.Bitlist{0x00, 0x00, 0x00, 0x00, 0x01},
-	}
-	aggregationBits = bitfield.NewBitlist(uint64(len(committee)))
-	aggregationBits.SetBitAt(0, true)
-	aggregationBits.SetBitAt(1, true)
-	att3.AggregationBits = aggregationBits
-
-	att3Sig1 := privKeys[committee[0]].Sign(hashTreeRoot[:], domain)
-	att3Sig2 := privKeys[committee[1]].Sign(hashTreeRoot[:], domain)
-	aggregatedSig := bls.AggregateSignatures([]*bls.Signature{att3Sig1, att3Sig2}).Marshal()
-	att3.Signature = aggregatedSig[:]
-
-	newBlock := &ethpb.BeaconBlock{
-		Slot: 0,
-	}
-	newBlockRoot, err := ssz.HashTreeRoot(newBlock)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveBlock(context.Background(), newBlock); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveState(context.Background(), beaconState, newBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(context.Background(), newBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	beaconState.Slot += params.BeaconConfig().MinAttestationInclusionDelay
-
-	if err := service.HandleAttestation(context.Background(), att1); err != nil {
-		t.Error(err)
-	}
-	if err := service.HandleAttestation(context.Background(), att2); err != nil {
-		t.Error(err)
-	}
-	if err := service.HandleAttestation(context.Background(), att1); err != nil {
-		t.Error(err)
-	}
-
-	attDataHash, err := ssz.HashTreeRoot(att2.Data)
-	if err != nil {
-		t.Error(err)
-	}
-	dbAtt := service.attestationPool[attDataHash].ToAttestations()[0]
-
-	dbAttBits := dbAtt.AggregationBits.Bytes()
-	aggregatedBits := att1.AggregationBits.Or(att2.AggregationBits).Bytes()
-	if !bytes.Equal(dbAttBits, aggregatedBits) {
-		t.Error("Expected aggregation bits to be equal.")
-	}
-
-	if !bytes.Equal(dbAtt.Signature, aggregatedSig) {
-		t.Error("Expected aggregated signatures to be equal")
-	}
-
-	if err := service.HandleAttestation(context.Background(), att2); err != nil {
-		t.Error(err)
-	}
-	dbAtt = service.attestationPool[attDataHash].ToAttestations()[0]
-
-	dbAttBits = dbAtt.AggregationBits.Bytes()
-	if !bytes.Equal(dbAttBits, aggregatedBits) {
-		t.Error("Expected aggregation bits to be equal.")
-	}
-
-	if !bytes.Equal(dbAtt.Signature, aggregatedSig) {
-		t.Error("Expected aggregated signatures to be equal")
-	}
-
-	if err := service.HandleAttestation(context.Background(), att3); err != nil {
-		t.Error(err)
-	}
-	dbAtt = service.attestationPool[attDataHash].ToAttestations()[0]
-
-	dbAttBits = dbAtt.AggregationBits.Bytes()
-	if !bytes.Equal(dbAttBits, aggregatedBits) {
-		t.Error("Expected aggregation bits to be equal.")
-	}
-
-	if !bytes.Equal(dbAtt.Signature, aggregatedSig) {
-		t.Error("Expected aggregated signatures to be equal")
-	}
-}
-
-func TestRetrieveAttestations_OK(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	service := NewService(context.Background(), &Config{BeaconDB: beaconDB})
-	service.attestationPool = make(map[[32]byte]*dbpb.AttestationContainer)
-
-	beaconState, privKeys := testutil.DeterministicGenesisState(t, 32)
-	aggBits := bitfield.NewBitlist(1)
-	aggBits.SetBitAt(0, true)
-	custodyBits := bitfield.NewBitlist(1)
-	att := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{Epoch: 0, Root: params.BeaconConfig().ZeroHash[:]},
-			Target: &ethpb.Checkpoint{Epoch: 0},
-		},
-		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
-	}
-	committee, err := helpers.BeaconCommitteeFromState(beaconState, att.Data.Slot, att.Data.CommitteeIndex)
-	if err != nil {
-		t.Fatal(err)
-	}
-	attestingIndices, err := helpers.AttestingIndices(att.AggregationBits, committee)
-	if err != nil {
-		t.Error(err)
-	}
-	dataAndCustodyBit := &pb.AttestationDataAndCustodyBit{
-		Data:       att.Data,
-		CustodyBit: false,
-	}
-	domain := helpers.Domain(beaconState.Fork, 0, params.BeaconConfig().DomainBeaconAttester)
-
-	sigs := make([]*bls.Signature, len(attestingIndices))
-
-	zeroSig := [96]byte{}
-	att.Signature = zeroSig[:]
-
-	for i, indice := range attestingIndices {
-		hashTreeRoot, err := ssz.HashTreeRoot(dataAndCustodyBit)
-		if err != nil {
-			t.Error(err)
-		}
-		sig := privKeys[indice].Sign(hashTreeRoot[:], domain)
-		sigs[i] = sig
-	}
-
-	beaconState.Slot += params.BeaconConfig().MinAttestationInclusionDelay
-	att.Signature = bls.AggregateSignatures(sigs).Marshal()[:]
-	beaconState.CurrentEpochAttestations = []*pb.PendingAttestation{}
-
-	r, _ := ssz.HashTreeRoot(att.Data)
-	service.attestationPool[r] = dbpb.NewContainerFromAttestations([]*ethpb.Attestation{att})
-
-	headBlockRoot := [32]byte{1, 2, 3}
-	if err := beaconDB.SaveState(context.Background(), beaconState, headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(context.Background(), headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-
-	// Test we can retrieve attestations from slot1 - slot61.
-	attestations, err := service.AttestationPool(context.Background(), 1)
-	if err != nil {
-		t.Fatalf("Could not retrieve attestations: %v", err)
-	}
-
-	if !reflect.DeepEqual(attestations[0], att) {
-		t.Error("Retrieved attestations did not match")
-	}
-}
-
-func TestRetrieveAttestations_PruneInvalidAtts(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	service := NewService(context.Background(), &Config{BeaconDB: beaconDB})
-
-	origAttestations := make([]*ethpb.Attestation, 140)
-	for i := 0; i < len(origAttestations); i++ {
-		origAttestations[i] = &ethpb.Attestation{
-			Data: &ethpb.AttestationData{
-				Slot:   uint64(i),
-				Source: &ethpb.Checkpoint{},
-				Target: &ethpb.Checkpoint{},
-			},
-			AggregationBits: bitfield.Bitlist{0b11},
-		}
-		if err := service.beaconDB.SaveAttestation(context.Background(), origAttestations[i]); err != nil {
-			t.Fatalf("Failed to save attestation: %v", err)
-		}
-	}
-
-	headBlockRoot := [32]byte{1, 2, 3}
-	if err := beaconDB.SaveState(context.Background(), &pb.BeaconState{
-		Slot: 200}, headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(context.Background(), headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-
-	attestations, err := service.AttestationPool(context.Background(), 200)
-	if err != nil {
-		t.Fatalf("Could not retrieve attestations: %v", err)
-	}
-
-	if len(attestations) != 0 {
-		t.Error("Incorrect pruned attestations")
-	}
-
-	// Verify the invalid attestations are deleted.
-	hash, err := hashutil.HashProto(origAttestations[1])
-	if err != nil {
-		t.Fatal(err)
-	}
-	if service.beaconDB.HasAttestation(context.Background(), hash) {
-		t.Error("Invalid attestation is not deleted")
-	}
-}
-
-func TestRemoveProcessedAttestations_Ok(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	s := NewService(context.Background(), &Config{BeaconDB: beaconDB})
-
-	attestations := make([]*ethpb.Attestation, 10)
-	for i := 0; i < len(attestations); i++ {
-		attestations[i] = &ethpb.Attestation{
-			Data: &ethpb.AttestationData{
-				Slot:   uint64(i),
-				Source: &ethpb.Checkpoint{},
-				Target: &ethpb.Checkpoint{},
-			},
-			AggregationBits: bitfield.Bitlist{0b11},
-		}
-		if err := s.beaconDB.SaveAttestation(context.Background(), attestations[i]); err != nil {
-			t.Fatalf("Failed to save attestation: %v", err)
-		}
-	}
-	headBlockRoot := [32]byte{1, 2, 3}
-	if err := beaconDB.SaveState(context.Background(), &pb.BeaconState{Slot: 15}, headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-	if err := beaconDB.SaveHeadBlockRoot(context.Background(), headBlockRoot); err != nil {
-		t.Fatal(err)
-	}
-
-	if err := s.removeAttestationsFromPool(context.Background(), attestations); err != nil {
-		t.Fatalf("Could not remove attestations: %v", err)
-	}
-
-	atts, err := s.AttestationPool(context.Background(), 15)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if len(atts) != 0 {
-		t.Errorf("Attestation pool should be empty but got a length of %d", len(atts))
-	}
-}
-
-func TestForkchoiceRetrieveAttestations_NotVoted(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	service := NewService(context.Background(), &Config{BeaconDB: beaconDB})
-	service.attestationPool = make(map[[32]byte]*dbpb.AttestationContainer)
-
-	aggBits := bitfield.NewBitlist(8)
-	aggBits.SetBitAt(1, true)
-	custodyBits := bitfield.NewBitlist(8)
-	att := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{},
-			Target: &ethpb.Checkpoint{},
-		},
-		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
-	}
-
-	r, _ := ssz.HashTreeRoot(att.Data)
-	service.attestationPool[r] = dbpb.NewContainerFromAttestations([]*ethpb.Attestation{att})
-
-	atts, err := service.AttestationPoolForForkchoice(context.Background())
-	if err != nil {
-		t.Fatalf("Could not retrieve attestations: %v", err)
-	}
-
-	if !reflect.DeepEqual(atts[0], att) {
-		t.Error("Did not receive wanted attestation")
-	}
-}
-
-func TestForkchoiceRetrieveAttestations_AlreadyVoted(t *testing.T) {
-	beaconDB := dbutil.SetupDB(t)
-	defer dbutil.TeardownDB(t, beaconDB)
-	service := NewService(context.Background(), &Config{BeaconDB: beaconDB})
-	service.attestationPool = make(map[[32]byte]*dbpb.AttestationContainer)
-
-	aggBits := bitfield.NewBitlist(8)
-	aggBits.SetBitAt(1, true)
-	custodyBits := bitfield.NewBitlist(8)
-	att := &ethpb.Attestation{
-		Data: &ethpb.AttestationData{
-			Source: &ethpb.Checkpoint{},
-			Target: &ethpb.Checkpoint{},
-		},
-		AggregationBits: aggBits,
-		CustodyBits:     custodyBits,
-	}
-
-	r, _ := ssz.HashTreeRoot(att.Data)
-	service.attestationPool[r] = dbpb.NewContainerFromAttestations([]*ethpb.Attestation{att})
-
-	_, err := service.AttestationPoolForForkchoice(context.Background())
-	if err != nil {
-		t.Fatalf("Could not retrieve attestations: %v", err)
-	}
-
-	atts, err := service.AttestationPoolForForkchoice(context.Background())
-	if err != nil {
-		t.Fatalf("Could not retrieve attestations: %v", err)
-	}
-
-	if len(atts) != 0 {
-		t.Errorf("Wanted att count 0, got %d", len(atts))
-	}
-}

beacon-chain/operations/attestations/BUILD.bazel

@@ -2,18 +2,42 @@ load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
 
 go_library(
     name = "go_default_library",
-    srcs = ["pool.go"],
+    srcs = [
+        "aggregate.go",
+        "log.go",
+        "pool.go",
+        "prepare_forkchoice.go",
+        "service.go",
+    ],
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/operations/attestations",
     visibility = ["//beacon-chain:__subpackages__"],
     deps = [
+        "//beacon-chain/core/helpers:go_default_library",
         "//beacon-chain/operations/attestations/kv:go_default_library",
+        "//shared/hashutil:go_default_library",
+        "//shared/params:go_default_library",
+        "@com_github_dgraph_io_ristretto//:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
+        "@com_github_prysmaticlabs_go_ssz//:go_default_library",
+        "@com_github_sirupsen_logrus//:go_default_library",
+        "@io_opencensus_go//trace:go_default_library",
     ],
 )
 
 go_test(
     name = "go_default_test",
-    srcs = ["pool_test.go"],
+    srcs = [
+        "aggregate_test.go",
+        "pool_test.go",
+        "prepare_forkchoice_test.go",
+        "service_test.go",
+    ],
     embed = [":go_default_library"],
-    deps = ["//beacon-chain/operations/attestations/kv:go_default_library"],
+    deps = [
+        "//beacon-chain/core/helpers:go_default_library",
+        "//beacon-chain/operations/attestations/kv:go_default_library",
+        "//shared/bls:go_default_library",
+        "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",
+        "@com_github_prysmaticlabs_go_bitfield//:go_default_library",
+    ],
 )

beacon-chain/operations/attestations/aggregate.go

@@ -0,0 +1,77 @@
+package attestations
+
+import (
+	"context"
+	"time"
+
+	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
+	"github.com/prysmaticlabs/go-ssz"
+	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
+	"github.com/prysmaticlabs/prysm/shared/params"
+	"go.opencensus.io/trace"
+)
+
+// Define time to aggregate the unaggregated attestations at 3 times per slot, this gives
+// enough confidence all the unaggregated attestations will be aggregated as aggregator requests.
+var timeToAggregate = time.Duration(params.BeaconConfig().SecondsPerSlot/3) * time.Second
+
+// This kicks off a routine to aggregate the unaggregated attestations from pool.
+func (s *Service) aggregateRoutine() {
+	ticker := time.NewTicker(timeToAggregate)
+	ctx := context.TODO()
+	for {
+		select {
+		case <-s.ctx.Done():
+			return
+		case <-ticker.C:
+			unaggregatedAtts := s.pool.UnaggregatedAttestations()
+			if err := s.aggregateAttestations(ctx, unaggregatedAtts); err != nil {
+				log.WithError(err).Error("Could not aggregate attestation")
+			}
+		}
+	}
+}
+
+// This aggregates the input attestations via AggregateAttestations helper
+// function.
+func (s *Service) aggregateAttestations(ctx context.Context, unaggregatedAtts []*ethpb.Attestation) error {
+	ctx, span := trace.StartSpan(ctx, "Operations.attestations.aggregateAttestations")
+	defer span.End()
+
+	unaggregatedAttsByRoot := make(map[[32]byte][]*ethpb.Attestation)
+
+	for _, att := range unaggregatedAtts {
+		attDataRoot, err := ssz.HashTreeRoot(att.Data)
+		if err != nil {
+			return err
+		}
+		unaggregatedAttsByRoot[attDataRoot] = append(unaggregatedAttsByRoot[attDataRoot], att)
+
+		if err := s.pool.DeleteUnaggregatedAttestation(att); err != nil {
+			return err
+		}
+	}
+
+	for _, atts := range unaggregatedAttsByRoot {
+		aggregatedAtts, err := helpers.AggregateAttestations(atts)
+		if err != nil {
+			return err
+		}
+		for _, att := range aggregatedAtts {
+			// In case of aggregation bit overlaps or there's only one
+			// unaggregated att in pool. Not every attestations will
+			// be aggregated.
+			if helpers.IsAggregated(att) {
+				if err := s.pool.SaveAggregatedAttestation(att); err != nil {
+					return err
+				}
+			} else {
+				if err := s.pool.SaveUnaggregatedAttestation(att); err != nil {
+					return err
+				}
+			}
+		}
+	}
+
+	return nil
+}