Revert Validator Reference Copy (#5701)

* revert all changes
* revert test
* unreachable code
* Merge refs/heads/master into revertValidatorRefCopy
* Merge refs/heads/master into revertValidatorRefCopy

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,59 @@
+---
+name: "\U0001F41EBug report"
+about: Report a bug or problem with running Prysm
+---
+<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
+
+Hellooo! 😄 
+
+To help us tend to your issue faster, please search our currently open issues before submitting a new one.
+Existing issues often contain information about workarounds, resolution, or progress updates.
+
+💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
+
+# 🐞 Bug Report
+
+### Description
+
+<!-- ✍️--> A clear and concise description of the problem...
+
+### Has this worked before in a previous version?
+
+<!-- Did this behavior use to work in the previous version? -->
+<!-- ✍️--> Yes, the previous version in which this bug was not present was: ....
+
+## 🔬 Minimal Reproduction
+
+<!--
+Please let us know how we can reproduce this issue. Include the exact method you used to run Prysm along with any flags used in your beacon chain and/or validator. Make sure you don't upload any confidential files or private keys.
+-->
+
+## 🔥 Error
+
+<pre><code>
+<!-- If the issue is accompanied by an error, please share the error logs with us below. If you have a lot of logs, place make a paste bin with your logs and share the link with us here: -->
+<!-- ✍️-->
+
+</code></pre>
+
+
+## 🌍  Your Environment
+
+**Operating System:**
+
+<pre>
+  <code>
+
+  </code>
+</pre>
+
+**What version of Prysm are you running? (Which release)**
+
+<pre>
+  <code>
+
+  </code>
+</pre>
+
+**Anything else relevant (validator index / public key)?**
+

.github/ISSUE_TEMPLATE/discussion.md

@@ -0,0 +1,21 @@
+---
+name: "\U0001F48EGeneral issue / discussion"
+about: Any other type of general issue or discussion
+
+---
+<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
+
+Hellooo! 😄 
+
+To help us tend to your issue faster, please search our currently open issues before submitting a new one.
+Existing issues often contain information about workarounds, resolution, or progress updates.
+
+💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
+
+# 💎 Issue
+
+### Background
+
+<!-- ✍️--> Context and background information on the discussion...
+
+### Description

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,27 @@
+---
+name: "\U0001F680Feature request"
+about: Suggest a feature for Prysm
+
+---
+<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
+
+Hellooo! 😄 
+
+To help us tend to your issue faster, please search our currently open issues before submitting a new one.
+Existing issues often contain information about workarounds, resolution, or progress updates.
+
+💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
+
+# 🚀 Feature Request
+
+### Description
+
+<!-- ✍️--> A clear and concise description of the problem or missing capability...
+
+### Describe the solution you'd like
+
+<!-- ✍️--> If you have a solution in mind, please describe it.
+
+### Describe alternatives you've considered
+
+<!-- ✍️--> Have you considered any alternative solutions or workarounds?

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,30 @@
+<!-- Thanks for sending a PR! Before submitting:
+
+1. If this is your first PR, please read CONTRIBUTING.md and sign the CLA
+   first. We cannot review code without a signed CLA.
+2. Please file an issue *first*. All features and most bug fixes should have
+   an associated issue with a design discussed and decided upon. Small bug
+   fixes and documentation improvements don't need issues.
+3. New features and bug fixes must have tests. Documentation may need to
+   be updated. If you're unsure what to update, send the PR, and we'll discuss
+   in review.
+4. Note that PRs updating dependencies and new Go versions are not accepted.
+   Please file an issue instead.
+-->
+
+**What type of PR is this?**
+
+> Uncomment one line below and remove others.
+>
+> Bug fix
+> Feature
+> Documentation
+> Other
+
+**What does this PR do? Why is it needed?**
+
+**Which issues(s) does this PR fix?**
+
+Fixes #
+
+**Other notes for review**

.github/workflows/dappnode-release-trigger.yml

@@ -0,0 +1,41 @@
+name: Update DAppNodePackages
+
+on:
+  push:
+    tags:
+    - '*'
+
+jobs:
+  dappnode-update-beacon-chain:
+    name: Trigger a beacon-chain release
+    runs-on: ubuntu-latest
+    steps:
+    - name: Get latest tag
+      id: get_tag
+      run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\//}
+    - name: Send dispatch event to DAppNodePackage-prysm-beacon-chain
+      env: 
+        DISPATCH_REPO: dappnode/DAppNodePackage-prysm-beacon-chain
+      run: |
+        curl -v -X POST -u "${{ secrets.PAT_GITHUB }}" \
+        -H "Accept: application/vnd.github.everest-preview+json" \
+        -H "Content-Type: application/json" \
+        --data '{"event_type":"new_release", "client_payload": { "tag":"${{ steps.get_tag.outputs.TAG }}"}}' \
+        https://api.github.com/repos/$DISPATCH_REPO/dispatches
+        
+  dappnode-update-validator:
+    name: Trigger a validator release
+    runs-on: ubuntu-latest
+    steps:
+    - name: Get latest tag
+      id: get_tag
+      run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\//}
+    - name: Send dispatch event to DAppNodePackage validator repository
+      env: 
+        DISPATCH_REPO: dappnode/DAppNodePackage-prysm-validator
+      run: |
+        curl -v -X POST -u "${{ secrets.PAT_GITHUB }}" \
+        -H "Accept: application/vnd.github.everest-preview+json" \
+        -H "Content-Type: application/json" \
+        --data '{"event_type":"new_release", "client_payload": { "tag":"${{ steps.get_tag.outputs.TAG }}"}}' \
+        https://api.github.com/repos/$DISPATCH_REPO/dispatches

BUILD.bazel

@@ -106,6 +106,7 @@ nogo(
         "//tools/analyzers/maligned:go_tool_library",
         "//tools/analyzers/roughtime:go_tool_library",
         "//tools/analyzers/errcheck:go_tool_library",
+        "//tools/analyzers/featureconfig:go_tool_library",
     ],
 )
 

PULL_REQUEST_TEMPLATE.md

@@ -1,13 +0,0 @@
-TODO(you): choose "part of" or "resolves" and the associated github issue #.
-
-[Part of|Resolves] #531 
-
----
-
-# Description
-
-**Write why you are making the changes in this pull request**
-
-**Write a summary of the changes you are making**
-
-**Link anything that would be helpful or relevant to the reviewers**

WORKSPACE

@@ -5,11 +5,11 @@ load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
 
 http_archive(
     name = "bazel_toolchains",
-    sha256 = "b5a8039df7119d618402472f3adff8a1bd0ae9d5e253f53fcc4c47122e91a3d2",
-    strip_prefix = "bazel-toolchains-2.1.1",
+    sha256 = "144290c4166bd67e76a54f96cd504ed86416ca3ca82030282760f0823c10be48",
+    strip_prefix = "bazel-toolchains-3.1.1",
     urls = [
-        "https://github.com/bazelbuild/bazel-toolchains/releases/download/2.1.1/bazel-toolchains-2.1.1.tar.gz",
-        "https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/2.1.1.tar.gz",
+        "https://github.com/bazelbuild/bazel-toolchains/releases/download/3.1.1/bazel-toolchains-3.1.1.tar.gz",
+        "https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/3.1.1.tar.gz",
     ],
 )
 
@@ -45,11 +45,17 @@ rbe_toolchains_config()
 
 http_archive(
     name = "bazel_skylib",
-    sha256 = "2ea8a5ed2b448baf4a6855d3ce049c4c452a6470b1efd1504fdb7c1c134d220a",
-    strip_prefix = "bazel-skylib-0.8.0",
-    url = "https://github.com/bazelbuild/bazel-skylib/archive/0.8.0.tar.gz",
+    sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
+    urls = [
+        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
+        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz",
+    ],
 )
 
+load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
+
+bazel_skylib_workspace()
+
 http_archive(
     name = "bazel_gazelle",
     sha256 = "d8c45ee70ec39a57e7a05e5027c32b1576cc7f16d9dd37135b0eddde45cf1b10",
@@ -75,19 +81,13 @@ http_archive(
 
 http_archive(
     name = "io_bazel_rules_go",
-    sha256 = "e6a6c016b0663e06fa5fccf1cd8152eab8aa8180c583ec20c872f4f9953a7ac5",
+    sha256 = "7b9bbe3ea1fccb46dcfa6c3f3e29ba7ec740d8733370e21cdc8937467b4a4349",
     urls = [
-        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.22.1/rules_go-v0.22.1.tar.gz",
-        "https://github.com/bazelbuild/rules_go/releases/download/v0.22.1/rules_go-v0.22.1.tar.gz",
+        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.22.4/rules_go-v0.22.4.tar.gz",
+        "https://github.com/bazelbuild/rules_go/releases/download/v0.22.4/rules_go-v0.22.4.tar.gz",
     ],
 )
 
-http_archive(
-    name = "build_bazel_rules_nodejs",
-    sha256 = "0942d188f4d0de6ddb743b9f6642a26ce1ad89f09c0035a9a5ca5ba9615c96aa",
-    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.38.1/rules_nodejs-0.38.1.tar.gz"],
-)
-
 git_repository(
     name = "graknlabs_bazel_distribution",
     commit = "962f3a7e56942430c0ec120c24f9e9f2a9c2ce1a",
@@ -252,7 +252,7 @@ go_repository(
 
 git_repository(
     name = "com_google_protobuf",
-    commit = "4cf5bfee9546101d98754d23ff378ff718ba8438",
+    commit = "4059c61f27eb1b06c4ee979546a238be792df0a4",
     remote = "https://github.com/protocolbuffers/protobuf",
     shallow_since = "1558721209 -0700",
 )
@@ -289,7 +289,7 @@ http_archive(
 
 go_repository(
     name = "com_github_ethereum_go_ethereum",
-    commit = "861ae1b1875c17d86a6a5d68118708ab2b099658",
+    commit = "0beb54b2147b3473a4c55e5ce6f02643ce403b14",
     importpath = "github.com/ethereum/go-ethereum",
     # Note: go-ethereum is not bazel-friendly with regards to cgo. We have a
     # a fork that has resolved these issues by disabling HID/USB support and
@@ -1110,8 +1110,9 @@ go_repository(
 
 go_repository(
     name = "grpc_ecosystem_grpc_gateway",
-    commit = "da7a886035e25b2f274f89b6f3c64bf70a9f6780",
     importpath = "github.com/grpc-ecosystem/grpc-gateway",
+    sum = "h1:IOPK2xMPP3aV6/NPt4jt//ELFo3Vv8sDVD8j3+tleDU=",
+    version = "v1.14.4",
 )
 
 go_repository(
@@ -1305,12 +1306,8 @@ go_repository(
 
 go_repository(
     name = "com_github_prysmaticlabs_ethereumapis",
-    commit = "6607cc86ddb7c78acfe3b1f0dfb115489a96d46d",
+    commit = "ba9042096e9fc49606279513d3e24e5e8cdbd5a0",
     importpath = "github.com/prysmaticlabs/ethereumapis",
-    patch_args = ["-p1"],
-    patches = [
-        "//third_party:com_github_prysmaticlabs_ethereumapis-tags.patch",
-    ],
 )
 
 go_repository(
@@ -1508,74 +1505,89 @@ go_repository(
 
 go_repository(
     name = "com_github_wealdtech_go_eth2_wallet",
-    commit = "6970d62e60d86fdae3c3e510e800e8a60d755a7d",
     importpath = "github.com/wealdtech/go-eth2-wallet",
+    sum = "h1:Hna/w4EKBJIs86VprIq7ez063A6kwk31d/O3Gs+MpYc=",
+    version = "v1.9.3",
 )
 
 go_repository(
-    name = "com_github_wealdtech_go_eth2_wallet_hd",
-    commit = "ce0a252a01c621687e9786a64899cfbfe802ba73",
-    importpath = "github.com/wealdtech/go-eth2-wallet-hd",
+    name = "com_github_wealdtech_go_eth2_wallet_hd_v2",
+    importpath = "github.com/wealdtech/go-eth2-wallet-hd/v2",
+    sum = "h1:kiCvdexK3zRC2GwZHSHq+hS+irVNtMs5pNADyumeeRM=",
+    version = "v2.0.1",
 )
 
 go_repository(
-    name = "com_github_wealdtech_go_eth2_wallet_nd",
-    commit = "12c8c41cdbd16797ff292e27f58e126bb89e9706",
-    importpath = "github.com/wealdtech/go-eth2-wallet-nd",
+    name = "com_github_wealdtech_go_eth2_wallet_nd_v2",
+    importpath = "github.com/wealdtech/go-eth2-wallet-nd/v2",
+    sum = "h1:4lB6GY5oHQn5xwn/Sxm1e9SeVCaxa7q/0hqXUQYUNwU=",
+    version = "v2.0.1",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_eth2_wallet_store_filesystem",
-    commit = "1eea6a48d75380047d2ebe7c8c4bd8985bcfdeca",
     importpath = "github.com/wealdtech/go-eth2-wallet-store-filesystem",
+    sum = "h1:Lc6wVTjIYeD+2hLAIzq1SugTWR527vEX4tEr5v3zxJc=",
+    version = "v1.7.2",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_eth2_wallet_store_s3",
-    commit = "1c821b5161f7bb0b3efa2030eff687eea5e70e53",
     importpath = "github.com/wealdtech/go-eth2-wallet-store-s3",
+    sum = "h1:xzyQDxbe5nr7xG0ByevTV2S8qkeOZvvjp+leBJcpxXQ=",
+    version = "v1.6.2",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_eth2_wallet_encryptor_keystorev4",
-    commit = "0c11c07b9544eb662210fadded94f40f309d8c8f",
     importpath = "github.com/wealdtech/go-eth2-wallet-encryptor-keystorev4",
+    sum = "h1:IcpS4VpXhYz+TVupB5n6C6IQzaKwG+Rc8nvgCa/da4c=",
+    version = "v1.0.0",
 )
 
 go_repository(
-    name = "com_github_wealdtech_go_eth2_wallet_types",
-    commit = "af67d8101be61e7c4dd8126d2b3eba20cff5dab2",
-    importpath = "github.com/wealdtech/go-eth2-wallet-types",
+    name = "com_github_wealdtech_go_eth2_wallet_types_v2",
+    importpath = "github.com/wealdtech/go-eth2-wallet-types/v2",
+    sum = "h1:Ct3RrNJTapBiG2GxVl53Kfgy96f0GEUV7bediTu91u8=",
+    version = "v2.0.1",
 )
 
 go_repository(
-    name = "com_github_wealdtech_go_eth2_types",
-    commit = "f9c31ddf180537dd5712d5998a3d56c45864d71f",
-    importpath = "github.com/wealdtech/go-eth2-types",
+    name = "com_github_wealdtech_go_eth2_types_v2",
+    build_directives = [
+        "gazelle:resolve go github.com/herumi/bls-eth-go-binary/bls @herumi_bls_eth_go_binary//:go_default_library",
+    ],
+    importpath = "github.com/wealdtech/go-eth2-types/v2",
+    sum = "h1:qfmgaCBkH2N11LHCXsRWYz7OOxc+1QXrKHlS9yDnFsw=",
+    version = "v2.3.0",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_eth2_util",
-    commit = "326ebb1755651131bb8f4506ea9a23be6d9ad1dd",
     importpath = "github.com/wealdtech/go-eth2-util",
+    sum = "h1:MyM16V7Qhd9q2ZaRa0WteBg2bWb8UplIKjZr8aeBZP0=",
+    version = "v1.1.4",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_ecodec",
-    commit = "7473d835445a3490e61a5fcf48fe4e9755a37957",
     importpath = "github.com/wealdtech/go-ecodec",
+    sum = "h1:yggrTSckcPJRaxxOxQF7FPm21kgE8WA6+f5jdq5Kr8o=",
+    version = "v1.1.0",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_bytesutil",
-    commit = "e564d0ade555b9f97494f0f669196ddcc6bc531d",
     importpath = "github.com/wealdtech/go-bytesutil",
+    sum = "h1:ocEg3Ke2GkZ4vQw5lp46rmO+pfqCCTgq35gqOy8JKVc=",
+    version = "v1.1.1",
 )
 
 go_repository(
     name = "com_github_wealdtech_go_indexer",
-    commit = "334862c32b1e3a5c6738a2618f5c0a8ebeb8cd51",
     importpath = "github.com/wealdtech/go-indexer",
+    sum = "h1:/S4rfWQbSOnnYmwnvuTVatDibZ8o1s9bmTCHO16XINg=",
+    version = "v1.0.0",
 )
 
 go_repository(
@@ -1643,8 +1655,8 @@ go_repository(
     name = "com_github_wealdtech_eth2_signer_api",
     build_file_proto_mode = "disable_global",
     importpath = "github.com/wealdtech/eth2-signer-api",
-    sum = "h1:fqJYjKwG/FeUAJYYiZblIP6agiz3WWB+Hxpw85Fnr5I=",
-    version = "v1.0.1",
+    sum = "h1:Fs0GfrdhboBKW7zaMvIvUHJaOB1ibpAmRG3lkB53in4=",
+    version = "v1.3.0",
 )
 
 go_repository(
@@ -1671,3 +1683,9 @@ go_repository(
     sum = "h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=",
     version = "v0.2.0",
 )
+
+go_repository(
+    name = "com_github_golang_gddo",
+    commit = "3c2cc9a6329d9842b3bbdaf307a8110d740cf94c",
+    importpath = "github.com/golang/gddo",
+)

beacon-chain/BUILD.bazel

@@ -20,6 +20,7 @@ go_library(
         "//shared/featureconfig:go_default_library",
         "//shared/logutil:go_default_library",
         "//shared/version:go_default_library",
+        "@com_github_ethereum_go_ethereum//log:go_default_library",
         "@com_github_ipfs_go_log//:go_default_library",
         "@com_github_joonix_log//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",
@@ -57,6 +58,7 @@ go_image(
         "//shared/featureconfig:go_default_library",
         "//shared/logutil:go_default_library",
         "//shared/version:go_default_library",
+        "@com_github_ethereum_go_ethereum//log:go_default_library",
         "@com_github_ipfs_go_log//:go_default_library",
         "@com_github_joonix_log//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",

beacon-chain/blockchain/chain_info.go

@@ -11,6 +11,7 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	"github.com/prysmaticlabs/prysm/beacon-chain/state"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
+	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 )
 
@@ -19,6 +20,7 @@ import (
 type ChainInfoFetcher interface {
 	HeadFetcher
 	FinalizationFetcher
+	GenesisFetcher
 }
 
 // TimeFetcher retrieves the Eth2 data that's related to time.
@@ -27,6 +29,11 @@ type TimeFetcher interface {
 	CurrentSlot() uint64
 }
 
+// GenesisFetcher retrieves the eth2 data related to its genesis.
+type GenesisFetcher interface {
+	GenesisValidatorRoot() [32]byte
+}
+
 // HeadFetcher defines a common interface for methods in blockchain service which
 // directly retrieves head related data.
 type HeadFetcher interface {
@@ -36,6 +43,7 @@ type HeadFetcher interface {
 	HeadState(ctx context.Context) (*state.BeaconState, error)
 	HeadValidatorsIndices(epoch uint64) ([]uint64, error)
 	HeadSeed(epoch uint64) ([32]byte, error)
+	HeadGenesisValidatorRoot() [32]byte
 }
 
 // ForkFetcher retrieves the current fork information of the Ethereum beacon chain.
@@ -173,11 +181,29 @@ func (s *Service) HeadSeed(epoch uint64) ([32]byte, error) {
 	return helpers.Seed(s.headState(), epoch, params.BeaconConfig().DomainBeaconAttester)
 }
 
+// HeadGenesisValidatorRoot returns genesis validator root of the head state.
+func (s *Service) HeadGenesisValidatorRoot() [32]byte {
+	if !s.hasHeadState() {
+		return [32]byte{}
+	}
+
+	return s.headGenesisValidatorRoot()
+}
+
 // GenesisTime returns the genesis time of beacon chain.
 func (s *Service) GenesisTime() time.Time {
 	return s.genesisTime
 }
 
+// GenesisValidatorRoot returns the genesis validator
+// root of the chain.
+func (s *Service) GenesisValidatorRoot() [32]byte {
+	if !s.hasHeadState() {
+		return [32]byte{}
+	}
+	return bytesutil.ToBytes32(s.head.state.GenesisValidatorRoot())
+}
+
 // CurrentFork retrieves the latest fork information of the beacon chain.
 func (s *Service) CurrentFork() *pb.Fork {
 	if !s.hasHeadState() {

beacon-chain/blockchain/chain_info_norace_test.go

@@ -17,7 +17,7 @@ func TestHeadSlot_DataRace(t *testing.T) {
 		beaconDB: db,
 	}
 	go func() {
-		if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+		if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 			t.Fatal(err)
 		}
 	}()
@@ -33,7 +33,7 @@ func TestHeadRoot_DataRace(t *testing.T) {
 		stateGen: stategen.New(db, cache.NewStateSummaryCache()),
 	}
 	go func() {
-		if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+		if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 			t.Fatal(err)
 		}
 	}()
@@ -51,7 +51,7 @@ func TestHeadBlock_DataRace(t *testing.T) {
 		stateGen: stategen.New(db, cache.NewStateSummaryCache()),
 	}
 	go func() {
-		if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+		if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 			t.Fatal(err)
 		}
 	}()
@@ -68,7 +68,7 @@ func TestHeadState_DataRace(t *testing.T) {
 		stateGen: stategen.New(db, cache.NewStateSummaryCache()),
 	}
 	go func() {
-		if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+		if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 			t.Fatal(err)
 		}
 	}()

beacon-chain/blockchain/chain_info_test.go

@@ -201,3 +201,20 @@ func TestCurrentFork_CanRetrieve(t *testing.T) {
 		t.Error("Received incorrect fork version")
 	}
 }
+
+func TestGenesisValidatorRoot_CanRetrieve(t *testing.T) {
+	// Should not panic if head state is nil.
+	c := &Service{}
+	if c.GenesisValidatorRoot() != [32]byte{} {
+		t.Error("Did not get correct genesis validator root")
+	}
+
+	s, err := state.InitializeFromProto(&pb.BeaconState{GenesisValidatorsRoot: []byte{'a'}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	c.head = &head{state: s}
+	if c.GenesisValidatorRoot() != [32]byte{'a'} {
+		t.Error("Did not get correct genesis validator root")
+	}
+}

beacon-chain/blockchain/head.go

@@ -59,7 +59,7 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
 
 	// If the head state is not available, just return nil.
 	// There's nothing to cache
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		if !s.stateGen.StateSummaryExists(ctx, headRoot) {
 			return nil
 		}
@@ -81,7 +81,7 @@ func (s *Service) saveHead(ctx context.Context, headRoot [32]byte) error {
 
 	// Get the new head state from cached state or DB.
 	var newHeadState *state.BeaconState
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		newHeadState, err = s.stateGen.StateByRoot(ctx, headRoot)
 		if err != nil {
 			return errors.Wrap(err, "could not retrieve head state in DB")
@@ -121,7 +121,7 @@ func (s *Service) saveHeadNoDB(ctx context.Context, b *ethpb.SignedBeaconBlock,
 
 	var headState *state.BeaconState
 	var err error
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		headState, err = s.stateGen.StateByRoot(ctx, r)
 		if err != nil {
 			return errors.Wrap(err, "could not retrieve head state in DB")
@@ -202,6 +202,14 @@ func (s *Service) headState() *state.BeaconState {
 	return s.head.state.Copy()
 }
 
+// This returns the genesis validator root of the head state.
+func (s *Service) headGenesisValidatorRoot() [32]byte {
+	s.headLock.RLock()
+	defer s.headLock.RUnlock()
+
+	return bytesutil.ToBytes32(s.head.state.GenesisValidatorRoot())
+}
+
 // Returns true if head state exists.
 func (s *Service) hasHeadState() bool {
 	s.headLock.RLock()

beacon-chain/blockchain/init_sync_process_block.go

@@ -170,6 +170,12 @@ func (s *Service) generateState(ctx context.Context, startRoot [32]byte, endRoot
 		return nil, err
 	}
 	if preState == nil {
+		if !s.stateGen.HasState(ctx, startRoot) {
+			if err := s.beaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
+				return nil, errors.Wrap(err, "could not save initial sync blocks")
+			}
+			s.clearInitSyncBlocks()
+		}
 		preState, err = s.stateGen.StateByRoot(ctx, startRoot)
 		if err != nil {
 			return nil, err

beacon-chain/blockchain/log.go

@@ -1,6 +1,7 @@
 package blockchain
 
 import (
+	"encoding/hex"
 	"fmt"
 
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
@@ -15,7 +16,6 @@ var log = logrus.WithField("prefix", "blockchain")
 // logs state transition related data every slot.
 func logStateTransitionData(b *ethpb.BeaconBlock) {
 	log.WithFields(logrus.Fields{
-		"slot":              b.Slot,
 		"attestations":      len(b.Body.Attestations),
 		"deposits":          len(b.Body.Deposits),
 		"attesterSlashings": len(b.Body.AttesterSlashings),
@@ -24,6 +24,16 @@ func logStateTransitionData(b *ethpb.BeaconBlock) {
 	}).Info("Finished applying state transition")
 }
 
+func logBlockSyncStatus(block *ethpb.BeaconBlock, blockRoot [32]byte, finalized *ethpb.Checkpoint) {
+	log.WithFields(logrus.Fields{
+		"slot":           block.Slot,
+		"block":          fmt.Sprintf("0x%s...", hex.EncodeToString(blockRoot[:])[:8]),
+		"epoch":          helpers.SlotToEpoch(block.Slot),
+		"finalizedEpoch": finalized.Epoch,
+		"finalizedRoot":  fmt.Sprintf("0x%s...", hex.EncodeToString(finalized.Root[:])[:8]),
+	}).Info("Synced new block")
+}
+
 func logEpochData(beaconState *stateTrie.BeaconState) {
 	log.WithFields(logrus.Fields{
 		"epoch":                  helpers.CurrentEpoch(beaconState),

beacon-chain/blockchain/metrics.go

@@ -19,6 +19,10 @@ var (
 		Name: "beacon_head_slot",
 		Help: "Slot of the head block of the beacon chain",
 	})
+	clockTimeSlot = promauto.NewGauge(prometheus.GaugeOpts{
+		Name: "beacon_clock_time_slot",
+		Help: "The current slot based on the genesis time and current clock",
+	})
 	competingBlks = promauto.NewCounter(prometheus.CounterOpts{
 		Name: "competing_blocks",
 		Help: "The # of blocks received and processed from a competing chain",
@@ -82,8 +86,9 @@ var (
 )
 
 // reportSlotMetrics reports slot related metrics.
-func reportSlotMetrics(currentSlot uint64, headSlot uint64, finalizedCheckpoint *ethpb.Checkpoint) {
-	beaconSlot.Set(float64(currentSlot))
+func reportSlotMetrics(stateSlot uint64, headSlot uint64, clockSlot uint64, finalizedCheckpoint *ethpb.Checkpoint) {
+	clockTimeSlot.Set(float64(clockSlot))
+	beaconSlot.Set(float64(stateSlot))
 	beaconHeadSlot.Set(float64(headSlot))
 	if finalizedCheckpoint != nil {
 		headFinalizedEpoch.Set(float64(finalizedCheckpoint.Epoch))

beacon-chain/blockchain/process_attestation.go

@@ -71,6 +71,16 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) ([]ui
 	ctx, span := trace.StartSpan(ctx, "blockchain.onAttestation")
 	defer span.End()
 
+	if a == nil {
+		return nil, errors.New("nil attestation")
+	}
+	if a.Data == nil {
+		return nil, errors.New("nil attestation.Data field")
+	}
+	if a.Data.Target == nil {
+		return nil, errors.New("nil attestation.Data.Target field")
+	}
+
 	tgt := stateTrie.CopyCheckpoint(a.Data.Target)
 	tgtSlot := helpers.StartSlot(tgt.Epoch)
 
@@ -128,12 +138,6 @@ func (s *Service) onAttestation(ctx context.Context, a *ethpb.Attestation) ([]ui
 	if indexedAtt.AttestingIndices == nil {
 		return nil, errors.New("nil attesting indices")
 	}
-	if a.Data == nil {
-		return nil, errors.New("nil att data")
-	}
-	if a.Data.Target == nil {
-		return nil, errors.New("nil att target")
-	}
 
 	// Update forkchoice store with the new attestation for updating weight.
 	s.forkChoiceStore.ProcessAttestation(ctx, indexedAtt.AttestingIndices, bytesutil.ToBytes32(a.Data.BeaconBlockRoot), a.Data.Target.Epoch)

beacon-chain/blockchain/process_attestation_helpers.go

@@ -31,48 +31,78 @@ func (s *Service) getAttPreState(ctx context.Context, c *ethpb.Checkpoint) (*sta
 		return cachedState, nil
 	}
 
-	var baseState *stateTrie.BeaconState
-	if !featureconfig.Get().DisableNewStateMgmt {
-		baseState, err = s.stateGen.StateByRoot(ctx, bytesutil.ToBytes32(c.Root))
+	if featureconfig.Get().NewStateMgmt {
+		if !s.stateGen.HasState(ctx, bytesutil.ToBytes32(c.Root)) {
+			if err := s.beaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
+				return nil, errors.Wrap(err, "could not save initial sync blocks")
+			}
+			s.clearInitSyncBlocks()
+		}
+
+		baseState, err := s.stateGen.StateByRoot(ctx, bytesutil.ToBytes32(c.Root))
 		if err != nil {
 			return nil, errors.Wrapf(err, "could not get pre state for slot %d", helpers.StartSlot(c.Epoch))
 		}
-	} else {
-		if featureconfig.Get().CheckHeadState {
-			headRoot, err := s.HeadRoot(ctx)
+
+		if helpers.StartSlot(c.Epoch) > baseState.Slot() {
+			baseState = baseState.Copy()
+			baseState, err = state.ProcessSlots(ctx, baseState, helpers.StartSlot(c.Epoch))
 			if err != nil {
-				return nil, errors.Wrapf(err, "could not get head root")
-			}
-			if bytes.Equal(headRoot, c.Root) {
-				st, err := s.HeadState(ctx)
-				if err != nil {
-					return nil, errors.Wrapf(err, "could not get head state")
-				}
-				if err := s.checkpointState.AddCheckpointState(&cache.CheckpointState{
-					Checkpoint: c,
-					State:      st.Copy(),
-				}); err != nil {
-					return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
-				}
-				return st, nil
+				return nil, errors.Wrapf(err, "could not process slots up to %d", helpers.StartSlot(c.Epoch))
 			}
 		}
 
-		baseState, err = s.beaconDB.State(ctx, bytesutil.ToBytes32(c.Root))
+		if err := s.checkpointState.AddCheckpointState(&cache.CheckpointState{
+			Checkpoint: c,
+			State:      baseState,
+		}); err != nil {
+			return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
+		}
+
+		return baseState, nil
+	}
+
+	if featureconfig.Get().CheckHeadState {
+		headRoot, err := s.HeadRoot(ctx)
 		if err != nil {
-			return nil, errors.Wrapf(err, "could not get pre state for slot %d", helpers.StartSlot(c.Epoch))
+			return nil, errors.Wrapf(err, "could not get head root")
+		}
+		if bytes.Equal(headRoot, c.Root) {
+			st, err := s.HeadState(ctx)
+			if err != nil {
+				return nil, errors.Wrapf(err, "could not get head state")
+			}
+			if err := s.checkpointState.AddCheckpointState(&cache.CheckpointState{
+				Checkpoint: c,
+				State:      st.Copy(),
+			}); err != nil {
+				return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
+			}
+			return st, nil
 		}
 	}
 
+	baseState, err := s.beaconDB.State(ctx, bytesutil.ToBytes32(c.Root))
+	if err != nil {
+		return nil, errors.Wrapf(err, "could not get pre state for slot %d", helpers.StartSlot(c.Epoch))
+	}
 	if baseState == nil {
 		return nil, fmt.Errorf("pre state of target block %d does not exist", helpers.StartSlot(c.Epoch))
 	}
 
 	if helpers.StartSlot(c.Epoch) > baseState.Slot() {
-		baseState, err = state.ProcessSlots(ctx, baseState, helpers.StartSlot(c.Epoch))
+		savedState := baseState.Copy()
+		savedState, err = state.ProcessSlots(ctx, savedState, helpers.StartSlot(c.Epoch))
 		if err != nil {
 			return nil, errors.Wrapf(err, "could not process slots up to %d", helpers.StartSlot(c.Epoch))
 		}
+		if err := s.checkpointState.AddCheckpointState(&cache.CheckpointState{
+			Checkpoint: c,
+			State:      savedState.Copy(),
+		}); err != nil {
+			return nil, errors.Wrap(err, "could not saved checkpoint state to cache")
+		}
+		return savedState, nil
 	}
 
 	if err := s.checkpointState.AddCheckpointState(&cache.CheckpointState{
@@ -128,7 +158,13 @@ func (s *Service) verifyAttestation(ctx context.Context, baseState *stateTrie.Be
 			// different seeds.
 			var aState *stateTrie.BeaconState
 			var err error
-			if !featureconfig.Get().DisableNewStateMgmt {
+			if featureconfig.Get().NewStateMgmt {
+				if !s.stateGen.HasState(ctx, bytesutil.ToBytes32(a.Data.BeaconBlockRoot)) {
+					if err := s.beaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
+						return nil, errors.Wrap(err, "could not save initial sync blocks")
+					}
+					s.clearInitSyncBlocks()
+				}
 				aState, err = s.stateGen.StateByRoot(ctx, bytesutil.ToBytes32(a.Data.BeaconBlockRoot))
 				if err != nil {
 					return nil, err

beacon-chain/blockchain/process_attestation_test.go

@@ -114,7 +114,7 @@ func TestStore_OnAttestation(t *testing.T) {
 			a:             &ethpb.Attestation{Data: &ethpb.AttestationData{Target: &ethpb.Checkpoint{Root: BlkWithOutStateRoot[:]}}},
 			s:             &pb.BeaconState{},
 			wantErr:       true,
-			wantErrString: "could not get pre state for slot 0: unknown boundary state",
+			wantErrString: "pre state of target block 0 does not exist",
 		},
 		{
 			name: "process attestation doesn't match current epoch",
@@ -124,13 +124,34 @@ func TestStore_OnAttestation(t *testing.T) {
 			wantErr:       true,
 			wantErrString: "does not match current epoch",
 		},
+		{
+			name:          "process nil field (a.Target) in attestation",
+			a:             nil,
+			s:             &pb.BeaconState{},
+			wantErr:       true,
+			wantErrString: "nil attestation",
+		},
+		{
+			name:          "process nil field (a.Data) in attestation",
+			a:             &ethpb.Attestation{},
+			s:             &pb.BeaconState{},
+			wantErr:       true,
+			wantErrString: "nil attestation.Data field",
+		},
+		{
+			name:          "process nil field (a.Target) in attestation",
+			a:             &ethpb.Attestation{Data: &ethpb.AttestationData{}},
+			s:             &pb.BeaconState{},
+			wantErr:       true,
+			wantErrString: "nil attestation.Data.Target field",
+		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			_, err := service.onAttestation(ctx, tt.a)
 			if tt.wantErr {
-				if !strings.Contains(err.Error(), tt.wantErrString) {
+				if err == nil || !strings.Contains(err.Error(), tt.wantErrString) {
 					t.Errorf("Store.onAttestation() error = %v, wantErr = %v", err, tt.wantErrString)
 				}
 			} else {

beacon-chain/blockchain/process_block.go

@@ -77,7 +77,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock)
 	log.WithFields(logrus.Fields{
 		"slot": b.Slot,
 		"root": fmt.Sprintf("0x%s...", hex.EncodeToString(root[:])[:8]),
-	}).Info("Executing state transition on block")
+	}).Debug("Executing state transition on block")
 
 	postState, err := state.ExecuteStateTransition(ctx, preState, signed)
 	if err != nil {
@@ -92,7 +92,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock)
 		return nil, errors.Wrapf(err, "could not insert block %d to fork choice store", b.Slot)
 	}
 
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		if err := s.stateGen.SaveState(ctx, root, postState); err != nil {
 			return nil, errors.Wrap(err, "could not save state")
 		}
@@ -122,7 +122,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock)
 			return nil, errors.Wrap(err, "could not save finalized checkpoint")
 		}
 
-		if featureconfig.Get().DisableNewStateMgmt {
+		if !featureconfig.Get().NewStateMgmt {
 			startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
 			endSlot := helpers.StartSlot(s.finalizedCheckpt.Epoch)
 			if endSlot > startSlot {
@@ -147,7 +147,7 @@ func (s *Service) onBlock(ctx context.Context, signed *ethpb.SignedBeaconBlock)
 			return nil, errors.Wrap(err, "could not save new justified")
 		}
 
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			fRoot := bytesutil.ToBytes32(postState.FinalizedCheckpoint().Root)
 			fBlock, err := s.beaconDB.Block(ctx, fRoot)
 			if err != nil {
@@ -233,7 +233,7 @@ func (s *Service) onBlockInitialSyncStateTransition(ctx context.Context, signed
 		return errors.Wrapf(err, "could not insert block %d to fork choice store", b.Slot)
 	}
 
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		if err := s.stateGen.SaveState(ctx, root, postState); err != nil {
 			return errors.Wrap(err, "could not save state")
 		}
@@ -268,7 +268,7 @@ func (s *Service) onBlockInitialSyncStateTransition(ctx context.Context, signed
 
 	// Update finalized check point. Prune the block cache and helper caches on every new finalized epoch.
 	if postState.FinalizedCheckpointEpoch() > s.finalizedCheckpt.Epoch {
-		if featureconfig.Get().DisableNewStateMgmt {
+		if !featureconfig.Get().NewStateMgmt {
 			startSlot := helpers.StartSlot(s.prevFinalizedCheckpt.Epoch)
 			endSlot := helpers.StartSlot(s.finalizedCheckpt.Epoch)
 			if endSlot > startSlot {
@@ -301,7 +301,7 @@ func (s *Service) onBlockInitialSyncStateTransition(ctx context.Context, signed
 			return errors.Wrap(err, "could not save new justified")
 		}
 
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			fRoot := bytesutil.ToBytes32(postState.FinalizedCheckpoint().Root)
 			fBlock, err := s.beaconDB.Block(ctx, fRoot)
 			if err != nil {
@@ -313,7 +313,7 @@ func (s *Service) onBlockInitialSyncStateTransition(ctx context.Context, signed
 		}
 	}
 
-	if featureconfig.Get().DisableNewStateMgmt {
+	if !featureconfig.Get().NewStateMgmt {
 		numOfStates := len(s.boundaryRoots)
 		if numOfStates > initialSyncCacheSize {
 			if err = s.persistCachedStates(ctx, numOfStates); err != nil {
@@ -338,7 +338,7 @@ func (s *Service) onBlockInitialSyncStateTransition(ctx context.Context, signed
 			return err
 		}
 
-		if featureconfig.Get().DisableNewStateMgmt && helpers.IsEpochStart(postState.Slot()) {
+		if !featureconfig.Get().NewStateMgmt && helpers.IsEpochStart(postState.Slot()) {
 			if err := s.beaconDB.SaveState(ctx, postState, root); err != nil {
 				return errors.Wrap(err, "could not save state")
 			}

beacon-chain/blockchain/process_block_helpers.go

@@ -21,7 +21,12 @@ import (
 
 // CurrentSlot returns the current slot based on time.
 func (s *Service) CurrentSlot() uint64 {
-	return uint64(roughtime.Now().Unix()-s.genesisTime.Unix()) / params.BeaconConfig().SecondsPerSlot
+	now := roughtime.Now().Unix()
+	genesis := s.genesisTime.Unix()
+	if now < genesis {
+		return 0
+	}
+	return uint64(now-genesis) / params.BeaconConfig().SecondsPerSlot
 }
 
 // getBlockPreState returns the pre state of an incoming block. It uses the parent root of the block
@@ -60,10 +65,19 @@ func (s *Service) verifyBlkPreState(ctx context.Context, b *ethpb.BeaconBlock) (
 	ctx, span := trace.StartSpan(ctx, "chainService.verifyBlkPreState")
 	defer span.End()
 
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		parentRoot := bytesutil.ToBytes32(b.ParentRoot)
-		if !s.stateGen.StateSummaryExists(ctx, parentRoot) {
-			return nil, errors.New("provided block root does not have block saved in the db")
+		// Loosen the check to HasBlock because state summary gets saved in batches
+		// during initial syncing. There's no risk given a state summary object is just a
+		// a subset of the block object.
+		if !s.stateGen.StateSummaryExists(ctx, parentRoot) && !s.beaconDB.HasBlock(ctx, parentRoot) {
+			return nil, errors.New("could not reconstruct parent state")
+		}
+		if !s.stateGen.HasState(ctx, parentRoot) {
+			if err := s.beaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
+				return nil, errors.Wrap(err, "could not save initial sync blocks")
+			}
+			s.clearInitSyncBlocks()
 		}
 		preState, err := s.stateGen.StateByRoot(ctx, parentRoot)
 		if err != nil {
@@ -196,7 +210,7 @@ func (s *Service) rmStatesOlderThanLastFinalized(ctx context.Context, startSlot
 	}
 
 	if err := s.beaconDB.DeleteStates(ctx, roots); err != nil {
-		return err
+		log.Warnf("Could not delete states: %v", err)
 	}
 
 	return nil
@@ -269,7 +283,7 @@ func (s *Service) updateJustified(ctx context.Context, state *stateTrie.BeaconSt
 		s.justifiedCheckpt = cpt
 	}
 
-	if featureconfig.Get().DisableNewStateMgmt {
+	if !featureconfig.Get().NewStateMgmt {
 		justifiedRoot := bytesutil.ToBytes32(cpt.Root)
 
 		justifiedState := s.initSyncState[justifiedRoot]

beacon-chain/blockchain/process_block_test.go

@@ -19,7 +19,9 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
+	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 	"github.com/prysmaticlabs/prysm/shared/params"
+	"github.com/prysmaticlabs/prysm/shared/roughtime"
 	"github.com/prysmaticlabs/prysm/shared/testutil"
 )
 
@@ -299,6 +301,9 @@ func TestShouldUpdateJustified_ReturnFalse(t *testing.T) {
 }
 
 func TestCachedPreState_CanGetFromStateSummary(t *testing.T) {
+	resetCfg := featureconfig.InitWithReset(&featureconfig.Flags{NewStateMgmt: true})
+	defer resetCfg()
+
 	ctx := context.Background()
 	db := testDB.SetupDB(t)
 	defer testDB.TeardownDB(t, db)
@@ -338,6 +343,8 @@ func TestCachedPreState_CanGetFromDB(t *testing.T) {
 	ctx := context.Background()
 	db := testDB.SetupDB(t)
 	defer testDB.TeardownDB(t, db)
+	resetCfg := featureconfig.InitWithReset(&featureconfig.Flags{NewStateMgmt: true})
+	defer resetCfg()
 
 	cfg := &Config{
 		BeaconDB: db,
@@ -353,7 +360,7 @@ func TestCachedPreState_CanGetFromDB(t *testing.T) {
 
 	service.finalizedCheckpt = &ethpb.Checkpoint{Root: r[:]}
 	_, err = service.verifyBlkPreState(ctx, b)
-	wanted := "provided block root does not have block saved in the db"
+	wanted := "could not reconstruct parent state"
 	if err.Error() != wanted {
 		t.Error("Did not get wanted error")
 	}
@@ -761,3 +768,13 @@ func blockTree1(db db.Database, genesisRoot []byte) ([][]byte, error) {
 	}
 	return [][]byte{r0[:], r1[:], nil, r3[:], r4[:], r5[:], r6[:], r7[:], r8[:]}, nil
 }
+
+func TestCurrentSlot_HandlesOverflow(t *testing.T) {
+	svc := Service{genesisTime: roughtime.Now().Add(1 * time.Hour)}
+
+	slot := svc.CurrentSlot()
+
+	if slot != 0 {
+		t.Fatalf("Expected slot to be 0, got %d", slot)
+	}
+}

beacon-chain/blockchain/receive_attestation.go

@@ -60,7 +60,7 @@ func (s *Service) ReceiveAttestationNoPubsub(ctx context.Context, att *ethpb.Att
 func (s *Service) IsValidAttestation(ctx context.Context, att *ethpb.Attestation) bool {
 	baseState, err := s.getAttPreState(ctx, att.Data.Target)
 	if err != nil {
-		log.WithError(err).Error("Failed to validate attestation")
+		log.WithError(err).Error("Failed to get attestation pre state")
 		return false
 	}
 
@@ -91,7 +91,7 @@ func (s *Service) processAttestation(subscribedToStateEvents chan struct{}) {
 			atts := s.attPool.ForkchoiceAttestations()
 			for _, a := range atts {
 				var hasState bool
-				if !featureconfig.Get().DisableNewStateMgmt {
+				if featureconfig.Get().NewStateMgmt {
 					hasState = s.stateGen.StateSummaryExists(ctx, bytesutil.ToBytes32(a.Data.BeaconBlockRoot))
 				} else {
 					hasState = s.beaconDB.HasState(ctx, bytesutil.ToBytes32(a.Data.BeaconBlockRoot)) && s.beaconDB.HasState(ctx, bytesutil.ToBytes32(a.Data.Target.Root))

beacon-chain/blockchain/receive_block.go

@@ -116,7 +116,10 @@ func (s *Service) ReceiveBlockNoPubsub(ctx context.Context, block *ethpb.SignedB
 	})
 
 	// Reports on block and fork choice metrics.
-	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.finalizedCheckpt)
+	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.CurrentSlot(), s.finalizedCheckpt)
+
+	// Log block sync status.
+	logBlockSyncStatus(blockCopy.Block, root, s.finalizedCheckpt)
 
 	// Log state transition data.
 	logStateTransitionData(blockCopy.Block)
@@ -166,7 +169,10 @@ func (s *Service) ReceiveBlockNoPubsubForkchoice(ctx context.Context, block *eth
 	})
 
 	// Reports on block and fork choice metrics.
-	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.finalizedCheckpt)
+	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.CurrentSlot(), s.finalizedCheckpt)
+
+	// Log block sync status.
+	logBlockSyncStatus(blockCopy.Block, root, s.finalizedCheckpt)
 
 	// Log state transition data.
 	logStateTransitionData(blockCopy.Block)
@@ -222,7 +228,7 @@ func (s *Service) ReceiveBlockNoVerify(ctx context.Context, block *ethpb.SignedB
 	})
 
 	// Reports on blockCopy and fork choice metrics.
-	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.finalizedCheckpt)
+	reportSlotMetrics(blockCopy.Block.Slot, s.headSlot(), s.CurrentSlot(), s.finalizedCheckpt)
 
 	// Log state transition data.
 	log.WithFields(logrus.Fields{

beacon-chain/blockchain/service.go

@@ -1,6 +1,6 @@
-// Package blockchain defines the life-cycle and status of the beacon chain
-// as well as the Ethereum Serenity beacon chain fork-choice rule based on
-// Casper Proof of Stake finality.
+// Package blockchain defines the life-cycle of the blockchain at the core of
+// eth2, including processing of new blocks and attestations using casper
+// proof of stake.
 package blockchain
 
 import (
@@ -140,7 +140,7 @@ func (s *Service) Start() {
 	}
 
 	if beaconState == nil {
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			beaconState, err = s.stateGen.StateByRoot(ctx, bytesutil.ToBytes32(cp.Root))
 			if err != nil {
 				log.Fatalf("Could not fetch beacon state by root: %v", err)
@@ -181,7 +181,7 @@ func (s *Service) Start() {
 		s.prevFinalizedCheckpt = stateTrie.CopyCheckpoint(finalizedCheckpoint)
 		s.resumeForkChoice(justifiedCheckpoint, finalizedCheckpoint)
 
-		if featureconfig.Get().DisableNewStateMgmt {
+		if !featureconfig.Get().NewStateMgmt {
 			if finalizedCheckpoint.Epoch > 1 {
 				if err := s.pruneGarbageState(ctx, helpers.StartSlot(finalizedCheckpoint.Epoch)-params.BeaconConfig().SlotsPerEpoch); err != nil {
 					log.WithError(err).Warn("Could not prune old states")
@@ -327,7 +327,7 @@ func (s *Service) saveGenesisData(ctx context.Context, genesisState *stateTrie.B
 	if err := s.beaconDB.SaveBlock(ctx, genesisBlk); err != nil {
 		return errors.Wrap(err, "could not save genesis block")
 	}
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		if err := s.stateGen.SaveState(ctx, genesisBlkRoot, genesisState); err != nil {
 			return errors.Wrap(err, "could not save genesis state")
 		}
@@ -415,7 +415,7 @@ func (s *Service) initializeChainInfo(ctx context.Context) error {
 	}
 	finalizedRoot := bytesutil.ToBytes32(finalized.Root)
 	var finalizedState *stateTrie.BeaconState
-	if !featureconfig.Get().DisableNewStateMgmt {
+	if featureconfig.Get().NewStateMgmt {
 		finalizedRoot = s.beaconDB.LastArchivedIndexRoot(ctx)
 		finalizedState, err = s.stateGen.Resume(ctx)
 		if err != nil {

beacon-chain/blockchain/service_norace_test.go

@@ -21,11 +21,11 @@ func TestChainService_SaveHead_DataRace(t *testing.T) {
 		beaconDB: db,
 	}
 	go func() {
-		if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+		if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 			t.Fatal(err)
 		}
 	}()
-	if err := s.saveHead(context.Background(), [32]byte{}, ); err != nil {
+	if err := s.saveHead(context.Background(), [32]byte{}); err != nil {
 		t.Fatal(err)
 	}
 }

beacon-chain/blockchain/testing/mock.go

@@ -1,3 +1,5 @@
+// Package testing includes useful mocks for writing unit
+// tests which depend on logic from the blockchain package.
 package testing
 
 import (
@@ -32,6 +34,7 @@ type ChainService struct {
 	BlocksReceived              []*ethpb.SignedBeaconBlock
 	Balance                     *precompute.Balance
 	Genesis                     time.Time
+	ValidatorsRoot              [32]byte
 	Fork                        *pb.Fork
 	DB                          db.Database
 	stateNotifier               statefeed.Notifier
@@ -217,6 +220,11 @@ func (ms *ChainService) GenesisTime() time.Time {
 	return ms.Genesis
 }
 
+// GenesisValidatorRoot mocks the same method in the chain service.
+func (ms *ChainService) GenesisValidatorRoot() [32]byte {
+	return ms.ValidatorsRoot
+}
+
 // CurrentSlot mocks the same method in the chain service.
 func (ms *ChainService) CurrentSlot() uint64 {
 	return uint64(time.Now().Unix()-ms.Genesis.Unix()) / params.BeaconConfig().SecondsPerSlot
@@ -239,3 +247,8 @@ func (ms *ChainService) ClearCachedStates() {}
 func (ms *ChainService) HasInitSyncBlock(root [32]byte) bool {
 	return false
 }
+
+// HeadGenesisValidatorRoot mocks HeadGenesisValidatorRoot method in chain service.
+func (ms *ChainService) HeadGenesisValidatorRoot() [32]byte {
+	return [32]byte{}
+}

beacon-chain/cache/BUILD.bazel

@@ -8,6 +8,7 @@ go_library(
         "committee.go",
         "committee_ids.go",
         "common.go",
+        "doc.go",
         "eth1_data.go",
         "hot_state_cache.go",
         "skip_slot_cache.go",

beacon-chain/cache/checkpoint_state.go

@@ -93,7 +93,8 @@ func (c *CheckpointStateCache) StateByCheckpoint(cp *ethpb.Checkpoint) (*stateTr
 		return nil, ErrNotCheckpointState
 	}
 
-	return info.State.Copy(), nil
+	// Copy here is unnecessary since the return will only be used to verify attestation signature.
+	return info.State, nil
 }
 
 // AddCheckpointState adds CheckpointState object to the cache. This method also trims the least

beacon-chain/cache/checkpoint_state_test.go

@@ -5,12 +5,12 @@ import (
 	"testing"
 
 	"github.com/gogo/protobuf/proto"
-	"github.com/prysmaticlabs/prysm/shared/params"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	stateTrie "github.com/prysmaticlabs/prysm/beacon-chain/state"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/hashutil"
+	"github.com/prysmaticlabs/prysm/shared/params"
 )
 
 func TestCheckpointStateCacheKeyFn_OK(t *testing.T) {

beacon-chain/cache/committee_ids.go

@@ -56,7 +56,7 @@ func (c *committeeIDs) GetAttesterCommitteeIDs(slot uint64) []uint64 {
 		return nil
 	}
 	if v, ok := val.([]uint64); ok {
-	   return v
+		return v
 	}
 	return nil
 }

beacon-chain/cache/depositcache/deposits_cache.go

@@ -1,3 +1,7 @@
+// Package depositcache is the source of validator deposits maintained
+// in-memory by the beacon node – deposits processed from the
+// eth1 powchain are then stored in this cache to be accessed by
+// any other service during a beacon node's runtime.
 package depositcache
 
 import (
@@ -35,12 +39,11 @@ type DepositFetcher interface {
 // stores all the deposit related data that is required by the beacon-node.
 type DepositCache struct {
 	// Beacon chain deposits in memory.
-	pendingDeposits       []*dbpb.DepositContainer
-	deposits              []*dbpb.DepositContainer
-	depositsLock          sync.RWMutex
-	chainStartDeposits    []*ethpb.Deposit
-	chainstartPubkeys     map[string]bool
-	chainstartPubkeysLock sync.RWMutex
+	pendingDeposits    []*dbpb.DepositContainer
+	deposits           []*dbpb.DepositContainer
+	depositsLock       sync.RWMutex
+	chainStartDeposits []*ethpb.Deposit
+	chainStartPubkeys  map[string]bool
 }
 
 // NewDepositCache instantiates a new deposit cache
@@ -48,7 +51,7 @@ func NewDepositCache() *DepositCache {
 	return &DepositCache{
 		pendingDeposits:    []*dbpb.DepositContainer{},
 		deposits:           []*dbpb.DepositContainer{},
-		chainstartPubkeys:  make(map[string]bool),
+		chainStartPubkeys:  make(map[string]bool),
 		chainStartDeposits: make([]*ethpb.Deposit, 0),
 	}
 }
@@ -102,21 +105,17 @@ func (dc *DepositCache) AllDepositContainers(ctx context.Context) []*dbpb.Deposi
 func (dc *DepositCache) MarkPubkeyForChainstart(ctx context.Context, pubkey string) {
 	ctx, span := trace.StartSpan(ctx, "DepositsCache.MarkPubkeyForChainstart")
 	defer span.End()
-	dc.chainstartPubkeysLock.Lock()
-	defer dc.chainstartPubkeysLock.Unlock()
-	dc.chainstartPubkeys[pubkey] = true
+	dc.chainStartPubkeys[pubkey] = true
 }
 
 // PubkeyInChainstart returns bool for whether the pubkey passed in has deposited.
 func (dc *DepositCache) PubkeyInChainstart(ctx context.Context, pubkey string) bool {
 	ctx, span := trace.StartSpan(ctx, "DepositsCache.PubkeyInChainstart")
 	defer span.End()
-	dc.chainstartPubkeysLock.Lock()
-	defer dc.chainstartPubkeysLock.Unlock()
-	if dc.chainstartPubkeys != nil {
-		return dc.chainstartPubkeys[pubkey]
+	if dc.chainStartPubkeys != nil {
+		return dc.chainStartPubkeys[pubkey]
 	}
-	dc.chainstartPubkeys = make(map[string]bool)
+	dc.chainStartPubkeys = make(map[string]bool)
 	return false
 }
 

beacon-chain/cache/doc.go

@@ -0,0 +1,5 @@
+// Package cache includes all important caches for the runtime
+// of an eth2 beacon node, ensuring the node does not spend
+// resources computing duplicate operations such as committee
+// calculations for validators during the same epoch, etc.
+package cache

beacon-chain/cache/feature_flag_test.go

@@ -1,9 +1,14 @@
 package cache
 
-import "github.com/prysmaticlabs/prysm/shared/featureconfig"
+import (
+	"os"
+	"testing"
 
-func init() {
-	featureconfig.Init(&featureconfig.Flags{
-		EnableEth1DataVoteCache: true,
-	})
+	"github.com/prysmaticlabs/prysm/shared/featureconfig"
+)
+
+func TestMain(m *testing.M) {
+	resetCfg := featureconfig.InitWithReset(&featureconfig.Flags{EnableEth1DataVoteCache: true})
+	defer resetCfg()
+	os.Exit(m.Run())
 }

beacon-chain/core/blocks/block.go

@@ -1,6 +1,5 @@
-// Package blocks contains block processing libraries. These libraries
-// process and verify block specific messages such as PoW receipt root,
-// RANDAO, validator deposits, exits and slashing proofs.
+// Package blocks contains block processing libraries according to
+// the eth2spec.
 package blocks
 
 import (

beacon-chain/core/blocks/block_operations.go

@@ -379,7 +379,7 @@ func ProcessRandaoNoVerify(
 	for i, x := range blockRandaoReveal {
 		latestMixSlice[i] ^= x
 	}
-	if err := beaconState.UpdateRandaoMixesAtIndex(latestMixSlice, currentEpoch%latestMixesLength); err != nil {
+	if err := beaconState.UpdateRandaoMixesAtIndex(currentEpoch%latestMixesLength, latestMixSlice); err != nil {
 		return nil, err
 	}
 	return beaconState, nil
@@ -987,7 +987,7 @@ func ProcessDeposit(
 		depositSig := deposit.Data.Signature
 		if err := verifyDepositDataSigningRoot(deposit.Data, pubKey, depositSig, domain); err != nil {
 			// Ignore this error as in the spec pseudo code.
-			log.Errorf("Skipping deposit: could not verify deposit data signature: %v", err)
+			log.Debugf("Skipping deposit: could not verify deposit data signature: %v", err)
 			return beaconState, nil
 		}
 

beacon-chain/core/blocks/block_operations_fuzz_test.go

@@ -25,12 +25,8 @@ func TestFuzzProcessAttestationNoVerify_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(att)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if _, err = ProcessAttestationNoVerify(ctx, s, att); err != nil {
-			t.Log(err)
-		}
+		_, err = ProcessAttestationNoVerify(ctx, s, att)
+		_ = err
 	}
 }
 
@@ -44,12 +40,8 @@ func TestFuzzProcessBlockHeader_10000(t *testing.T) {
 		fuzzer.Fuzz(block)
 
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if _, err = ProcessBlockHeader(s, block); err != nil {
-			t.Log(err)
-		}
+		_, err = ProcessBlockHeader(s, block)
+		_ = err
 	}
 }
 
@@ -70,13 +62,9 @@ func TestFuzzverifyDepositDataSigningRoot_10000(t *testing.T) {
 		fuzzer.Fuzz(&p)
 		fuzzer.Fuzz(&s)
 		fuzzer.Fuzz(&d)
-		if err := verifySignature(ba, pubkey[:], sig[:], domain[:]); err != nil {
-			t.Log(err)
-		}
-		if err := verifySignature(ba, p, s, d); err != nil {
-			t.Log(err)
-		}
-
+		err := verifySignature(ba, pubkey[:], sig[:], domain[:])
+		err = verifySignature(ba, p, s, d)
+		_ = err
 	}
 }
 
@@ -117,12 +105,8 @@ func TestFuzzEth1DataHasEnoughSupport_10000(t *testing.T) {
 		s, err := beaconstate.InitializeFromProto(&ethereum_beacon_p2p_v1.BeaconState{
 			Eth1DataVotes: stateVotes,
 		})
-		if err != nil {
-			t.Log(err)
-		}
-		if _, err := Eth1DataHasEnoughSupport(s, eth1data); err != nil {
-			t.Log(err)
-		}
+		_, err = Eth1DataHasEnoughSupport(s, eth1data)
+		_ = err
 	}
 
 }
@@ -136,12 +120,8 @@ func TestFuzzProcessBlockHeaderNoVerify_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(block)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if _, err = ProcessBlockHeaderNoVerify(s, block); err != nil {
-			t.Log(err)
-		}
+		_, err = ProcessBlockHeaderNoVerify(s, block)
+		_ = err
 	}
 }
 
@@ -154,9 +134,6 @@ func TestFuzzProcessRandao_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessRandao(s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -173,9 +150,6 @@ func TestFuzzProcessRandaoNoVerify_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessRandaoNoVerify(s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -192,9 +166,6 @@ func TestFuzzProcessProposerSlashings_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessProposerSlashings(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -210,12 +181,8 @@ func TestFuzzVerifyProposerSlashing_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(proposerSlashing)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if err := VerifyProposerSlashing(s, proposerSlashing); err != nil {
-			t.Log(err)
-		}
+		err = VerifyProposerSlashing(s, proposerSlashing)
+		_ = err
 	}
 }
 
@@ -228,9 +195,6 @@ func TestFuzzProcessAttesterSlashings_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessAttesterSlashings(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -247,12 +211,8 @@ func TestFuzzVerifyAttesterSlashing_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(attesterSlashing)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if err := VerifyAttesterSlashing(ctx, s, attesterSlashing); err != nil {
-			t.Log(err)
-		}
+		err = VerifyAttesterSlashing(ctx, s, attesterSlashing)
+		_ = err
 	}
 }
 
@@ -287,9 +247,6 @@ func TestFuzzProcessAttestations_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessAttestations(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -306,9 +263,6 @@ func TestFuzzProcessAttestationsNoVerify_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessAttestationsNoVerify(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -325,9 +279,6 @@ func TestFuzzProcessAttestation_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(attestation)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessAttestation(ctx, s, attestation)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, attestation)
@@ -344,12 +295,8 @@ func TestFuzzVerifyIndexedAttestationn_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(idxAttestation)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if err := VerifyIndexedAttestation(ctx, s, idxAttestation); err != nil {
-			t.Log(err)
-		}
+		err = VerifyIndexedAttestation(ctx, s, idxAttestation)
+		_ = err
 	}
 }
 
@@ -362,12 +309,8 @@ func TestFuzzVerifyAttestation_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(attestation)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if err := VerifyAttestation(ctx, s, attestation); err != nil {
-			t.Log(err)
-		}
+		err = VerifyAttestation(ctx, s, attestation)
+		_ = err
 	}
 }
 
@@ -380,9 +323,6 @@ func TestFuzzProcessDeposits_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessDeposits(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -400,9 +340,6 @@ func TestFuzzProcessPreGenesisDeposit_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(deposit)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessPreGenesisDeposit(ctx, s, deposit)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, deposit)
@@ -419,9 +356,6 @@ func TestFuzzProcessDeposit_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(deposit)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessDeposit(s, deposit)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, deposit)
@@ -437,12 +371,8 @@ func TestFuzzverifyDeposit_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(deposit)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
-		if err := verifyDeposit(s, deposit); err != nil {
-			t.Log(err)
-		}
+		err = verifyDeposit(s, deposit)
+		_ = err
 	}
 }
 
@@ -455,9 +385,6 @@ func TestFuzzProcessVoluntaryExits_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessVoluntaryExits(ctx, s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -473,9 +400,6 @@ func TestFuzzProcessVoluntaryExitsNoVerify_10000(t *testing.T) {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(blockBody)
 		s, err := beaconstate.InitializeFromProtoUnsafe(state)
-		if err != nil {
-			t.Log(err)
-		}
 		r, err := ProcessVoluntaryExitsNoVerify(s, blockBody)
 		if err != nil && r != nil {
 			t.Fatalf("return value should be nil on err. found: %v on error: %v for state: %v and block: %v", r, err, state, blockBody)
@@ -495,8 +419,7 @@ func TestFuzzVerifyExit_10000(t *testing.T) {
 		fuzzer.Fuzz(val)
 		fuzzer.Fuzz(fork)
 		fuzzer.Fuzz(&slot)
-		if err := VerifyExit(val, slot, fork, ve, params.BeaconConfig().ZeroHash[:]); err != nil {
-			t.Log(err)
-		}
+		err := VerifyExit(val, slot, fork, ve, params.BeaconConfig().ZeroHash[:])
+		_ = err
 	}
 }

beacon-chain/core/blocks/spectest/block_processing_test.go

@@ -1,3 +1,5 @@
+// Package spectest contains all comformity specification tests
+// for block processing according to the eth2 spec.
 package spectest
 
 import (

beacon-chain/core/epoch/epoch_processing.go

@@ -1,6 +1,6 @@
-// Package epoch contains epoch processing libraries. These libraries
-// process new balance for the validators, justify and finalize new
-// check points, shuffle and reassign validators to different slots and
+// Package epoch contains epoch processing libraries according to spec, able to
+// process new balance for validators, justify and finalize new
+// check points, and shuffle validators to different slots and
 // shards.
 package epoch
 
@@ -293,7 +293,7 @@ func ProcessFinalUpdates(state *stateTrie.BeaconState) (*stateTrie.BeaconState,
 	if err != nil {
 		return nil, err
 	}
-	if err := state.UpdateRandaoMixesAtIndex(mix, nextEpoch%randaoMixLength); err != nil {
+	if err := state.UpdateRandaoMixesAtIndex(nextEpoch%randaoMixLength, mix); err != nil {
 		return nil, err
 	}
 

beacon-chain/core/epoch/epoch_processing_fuzz_test.go

@@ -19,8 +19,5 @@ func TestFuzzFinalUpdates_10000(t *testing.T) {
 			t.Fatal(err)
 		}
 		_, err = ProcessFinalUpdates(s)
-		if err != nil {
-			t.Log(err)
-		}
 	}
 }

beacon-chain/core/epoch/precompute/new.go

@@ -1,3 +1,6 @@
+// Package precompute provides gathering of nicely-structured
+// data important to feed into epoch processing, such as attesting
+// records and balances, for faster computation.
 package precompute
 
 import (

beacon-chain/core/epoch/spectest/registry_mainnet_test.go

@@ -1,3 +1,5 @@
+// Package spectest contains all conformity specification tests
+// for epoch processing according to the eth2 spec.
 package spectest
 
 import (

beacon-chain/core/feed/block/events.go

@@ -1,3 +1,5 @@
+// Package block contains types for block-specific events fired
+// during the runtime of a beacon node.
 package block
 
 import (

beacon-chain/core/feed/event.go

@@ -1,3 +1,5 @@
+// Package feed defines event feed types for inter-service communication
+// during a beacon node's runtime.
 package feed
 
 // How to add a new event to the feed:

beacon-chain/core/feed/operation/events.go

@@ -1,3 +1,6 @@
+// Package operation contains types for block operation-specific events fired
+// during the runtime of a beacon node such as attestations, voluntary
+// exits, and slashings.
 package operation
 
 import (

beacon-chain/core/feed/state/events.go

@@ -1,3 +1,6 @@
+// Package state contains types for state operation-specific events fired
+// during the runtime of a beacon node such state initialization, state updates,
+// and chain start.
 package state
 
 import "time"
@@ -9,6 +12,8 @@ const (
 	ChainStarted
 	// Initialized is sent when the internal beacon node's state is ready to be accessed.
 	Initialized
+	// Synced is sent when the beacon node has completed syncing and is ready to participate in the network.
+	Synced
 )
 
 // BlockProcessedData is the data sent with BlockProcessed events.
@@ -27,6 +32,12 @@ type ChainStartedData struct {
 	StartTime time.Time
 }
 
+// SyncedData is the data sent with Synced events.
+type SyncedData struct {
+	// StartTime is the time at which the chain started.
+	StartTime time.Time
+}
+
 // InitializedData is the data sent with Initialized events.
 type InitializedData struct {
 	// StartTime is the time at which the chain started.

beacon-chain/core/helpers/BUILD.bazel

@@ -16,14 +16,15 @@ go_library(
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/core/helpers",
     visibility = [
         "//beacon-chain:__subpackages__",
+        "//endtoend/evaluators:__pkg__",
         "//shared/benchutil/benchmark_files:__subpackages__",
-        "//shared/testutil:__pkg__",
-        "//shared/keystore:__pkg__",
         "//shared/interop:__pkg__",
+        "//shared/keystore:__pkg__",
+        "//shared/p2putils:__pkg__",
+        "//shared/testutil:__pkg__",
         "//slasher:__subpackages__",
         "//tools:__subpackages__",
         "//validator:__subpackages__",
-        "//endtoend/evaluators:__pkg__",
     ],
     deps = [
         "//beacon-chain/cache:go_default_library",
@@ -64,7 +65,6 @@ go_test(
         "//shared/attestationutil:go_default_library",
         "//shared/bls:go_default_library",
         "//shared/bytesutil:go_default_library",
-        "//shared/featureconfig:go_default_library",
         "//shared/hashutil:go_default_library",
         "//shared/params:go_default_library",
         "//shared/sliceutil:go_default_library",

beacon-chain/core/helpers/committee.go

@@ -1,4 +1,5 @@
-// Package helpers contains helper functions outlined in ETH2.0 spec beacon chain spec
+// Package helpers contains helper functions outlined in the eth2 beacon chain spec, such as
+// computing committees, randao, rewards/penalties, and more.
 package helpers
 
 import (
@@ -372,11 +373,10 @@ func precomputeProposerIndices(state *stateTrie.BeaconState, activeIndices []uin
 		return nil, errors.Wrap(err, "could not generate seed")
 	}
 	slot := StartSlot(e)
-	vals := state.Validators()
 	for i := uint64(0); i < params.BeaconConfig().SlotsPerEpoch; i++ {
 		seedWithSlot := append(seed[:], bytesutil.Bytes8(slot+i)...)
 		seedWithSlotHash := hashFunc(seedWithSlot)
-		index, err := ComputeProposerIndex(vals, activeIndices, seedWithSlotHash)
+		index, err := ComputeProposerIndex(state, activeIndices, seedWithSlotHash)
 		if err != nil {
 			return nil, err
 		}

beacon-chain/core/helpers/committee_test.go

@@ -12,7 +12,6 @@ import (
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
 	"github.com/prysmaticlabs/prysm/shared/attestationutil"
 	"github.com/prysmaticlabs/prysm/shared/bytesutil"
-	"github.com/prysmaticlabs/prysm/shared/featureconfig"
 	"github.com/prysmaticlabs/prysm/shared/hashutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/sliceutil"
@@ -434,10 +433,6 @@ func TestShuffledIndices_ShuffleRightLength(t *testing.T) {
 
 func TestUpdateCommitteeCache_CanUpdate(t *testing.T) {
 	ClearCache()
-	c := featureconfig.Get()
-	featureconfig.Init(c)
-	defer featureconfig.Init(nil)
-
 	validatorCount := int(params.BeaconConfig().MinGenesisActiveValidatorCount)
 	validators := make([]*ethpb.Validator, validatorCount)
 	indices := make([]uint64, validatorCount)
@@ -679,10 +674,6 @@ func BenchmarkComputeCommittee4000000_WithOutCache(b *testing.B) {
 }
 
 func TestBeaconCommitteeFromState_UpdateCacheForPreviousEpoch(t *testing.T) {
-	c := featureconfig.Get()
-	featureconfig.Init(c)
-	defer featureconfig.Init(nil)
-
 	committeeSize := uint64(16)
 	validators := make([]*ethpb.Validator, committeeSize*params.BeaconConfig().SlotsPerEpoch)
 	for i := 0; i < len(validators); i++ {
@@ -752,7 +743,7 @@ func TestPrecomputeProposerIndices_Ok(t *testing.T) {
 	for i := uint64(0); i < params.BeaconConfig().SlotsPerEpoch; i++ {
 		seedWithSlot := append(seed[:], bytesutil.Bytes8(i)...)
 		seedWithSlotHash := hashutil.Hash(seedWithSlot)
-		index, err := ComputeProposerIndex(state.Validators(), indices, seedWithSlotHash)
+		index, err := ComputeProposerIndex(state, indices, seedWithSlotHash)
 		if err != nil {
 			t.Fatal(err)
 		}

beacon-chain/core/helpers/signing_root_test.go

@@ -76,11 +76,8 @@ func TestFuzzverifySigningRoot_10000(t *testing.T) {
 		fuzzer.Fuzz(&p)
 		fuzzer.Fuzz(&s)
 		fuzzer.Fuzz(&d)
-		if err := VerifySigningRoot(state, pubkey[:], sig[:], domain[:]); err != nil {
-			t.Log(err)
-		}
-		if err := VerifySigningRoot(state, p, s, d); err != nil {
-			t.Log(err)
-		}
+		err := VerifySigningRoot(state, pubkey[:], sig[:], domain[:])
+		err = VerifySigningRoot(state, p, s, d)
+		_ = err
 	}
 }

beacon-chain/core/helpers/spectest/shuffle_yaml_test.go

@@ -1,3 +1,5 @@
+// Package spectest contains all conformity specification tests
+// for validator shuffling logic according to the eth2 beacon spec.
 package spectest
 
 import (

beacon-chain/core/helpers/validators.go

@@ -179,11 +179,63 @@ func BeaconProposerIndex(state *stateTrie.BeaconState) (uint64, error) {
 		return 0, errors.Wrap(err, "could not update committee cache")
 	}
 
-	return ComputeProposerIndex(state.Validators(), indices, seedWithSlotHash)
+	return ComputeProposerIndex(state, indices, seedWithSlotHash)
 }
 
 // ComputeProposerIndex returns the index sampled by effective balance, which is used to calculate proposer.
 //
+//	This method is more efficient than ComputeProposerIndexWithValidators as it uses the read only validator
+//	abstraction to retrieve validator related data. Whereas the other method requires a whole copy of the validator
+//	set.
+//
+// Spec pseudocode definition:
+//  def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Hash) -> ValidatorIndex:
+//    """
+//    Return from ``indices`` a random index sampled by effective balance.
+//    """
+//    assert len(indices) > 0
+//    MAX_RANDOM_BYTE = 2**8 - 1
+//    i = 0
+//    while True:
+//        candidate_index = indices[compute_shuffled_index(ValidatorIndex(i % len(indices)), len(indices), seed)]
+//        random_byte = hash(seed + int_to_bytes(i // 32, length=8))[i % 32]
+//        effective_balance = state.validators[candidate_index].effective_balance
+//        if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
+//            return ValidatorIndex(candidate_index)
+//        i += 1
+func ComputeProposerIndex(bState *stateTrie.BeaconState, activeIndices []uint64, seed [32]byte) (uint64, error) {
+	length := uint64(len(activeIndices))
+	if length == 0 {
+		return 0, errors.New("empty active indices list")
+	}
+	maxRandomByte := uint64(1<<8 - 1)
+	hashFunc := hashutil.CustomSHA256Hasher()
+
+	for i := uint64(0); ; i++ {
+		candidateIndex, err := ComputeShuffledIndex(i%length, length, seed, true /* shuffle */)
+		if err != nil {
+			return 0, err
+		}
+		candidateIndex = activeIndices[candidateIndex]
+		if int(candidateIndex) >= bState.NumValidators() {
+			return 0, errors.New("active index out of range")
+		}
+		b := append(seed[:], bytesutil.Bytes8(i/32)...)
+		randomByte := hashFunc(b)[i%32]
+		v, err := bState.ValidatorAtIndexReadOnly(candidateIndex)
+		if err != nil {
+			return 0, nil
+		}
+		effectiveBal := v.EffectiveBalance()
+
+		if effectiveBal*maxRandomByte >= params.BeaconConfig().MaxEffectiveBalance*uint64(randomByte) {
+			return candidateIndex, nil
+		}
+	}
+}
+
+// ComputeProposerIndexWithValidators returns the index sampled by effective balance, which is used to calculate proposer.
+//
 // Note: This method signature deviates slightly from the spec recommended definition. The full
 // state object is not required to compute the proposer index.
 //
@@ -202,7 +254,8 @@ func BeaconProposerIndex(state *stateTrie.BeaconState) (uint64, error) {
 //        if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
 //            return ValidatorIndex(candidate_index)
 //        i += 1
-func ComputeProposerIndex(validators []*ethpb.Validator, activeIndices []uint64, seed [32]byte) (uint64, error) {
+// Deprecated: Prefer using the beacon state with ComputeProposerIndex to avoid an unnecessary copy of the validator set.
+func ComputeProposerIndexWithValidators(validators []*ethpb.Validator, activeIndices []uint64, seed [32]byte) (uint64, error) {
 	length := uint64(len(activeIndices))
 	if length == 0 {
 		return 0, errors.New("empty active indices list")

beacon-chain/core/helpers/validators_test.go

@@ -5,6 +5,8 @@ import (
 	"reflect"
 	"testing"
 
+	"github.com/prysmaticlabs/prysm/shared/hashutil"
+
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	beaconstate "github.com/prysmaticlabs/prysm/beacon-chain/state"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
@@ -184,6 +186,62 @@ func TestBeaconProposerIndex_OK(t *testing.T) {
 	}
 }
 
+func TestComputeProposerIndex_Compatibility(t *testing.T) {
+	validators := make([]*ethpb.Validator, params.BeaconConfig().MinGenesisActiveValidatorCount)
+	for i := 0; i < len(validators); i++ {
+		validators[i] = &ethpb.Validator{
+			ExitEpoch: params.BeaconConfig().FarFutureEpoch,
+		}
+	}
+
+	state, err := beaconstate.InitializeFromProto(&pb.BeaconState{
+		Validators:  validators,
+		RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	indices, err := ActiveValidatorIndices(state, 0)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var proposerIndices []uint64
+	seed, err := Seed(state, 0, params.BeaconConfig().DomainBeaconProposer)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for i := uint64(0); i < params.BeaconConfig().SlotsPerEpoch; i++ {
+		seedWithSlot := append(seed[:], bytesutil.Bytes8(i)...)
+		seedWithSlotHash := hashutil.Hash(seedWithSlot)
+		index, err := ComputeProposerIndex(state, indices, seedWithSlotHash)
+		if err != nil {
+			t.Fatal(err)
+		}
+		proposerIndices = append(proposerIndices, index)
+	}
+
+	var wantedProposerIndices []uint64
+	seed, err = Seed(state, 0, params.BeaconConfig().DomainBeaconProposer)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for i := uint64(0); i < params.BeaconConfig().SlotsPerEpoch; i++ {
+		seedWithSlot := append(seed[:], bytesutil.Bytes8(i)...)
+		seedWithSlotHash := hashutil.Hash(seedWithSlot)
+		index, err := ComputeProposerIndexWithValidators(state.Validators(), indices, seedWithSlotHash)
+		if err != nil {
+			t.Fatal(err)
+		}
+		wantedProposerIndices = append(wantedProposerIndices, index)
+	}
+
+	if !reflect.DeepEqual(wantedProposerIndices, proposerIndices) {
+		t.Error("Wanted proposer indices from ComputeProposerIndexWithValidators does not match")
+	}
+}
+
 func TestDelayedActivationExitEpoch_OK(t *testing.T) {
 	epoch := uint64(9999)
 	got := ActivationExitEpoch(epoch)
@@ -541,7 +599,13 @@ func TestComputeProposerIndex(t *testing.T) {
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			got, err := ComputeProposerIndex(tt.args.validators, tt.args.indices, tt.args.seed)
+			bState := &pb.BeaconState{Validators: tt.args.validators}
+			stTrie, err := beaconstate.InitializeFromProtoUnsafe(bState)
+			if err != nil {
+				t.Error(err)
+				return
+			}
+			got, err := ComputeProposerIndex(stTrie, tt.args.indices, tt.args.seed)
 			if (err != nil) != tt.wantErr {
 				t.Errorf("ComputeProposerIndex() error = %v, wantErr %v", err, tt.wantErr)
 				return

beacon-chain/core/state/BUILD.bazel

@@ -12,11 +12,12 @@ go_library(
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/core/state",
     visibility = [
         "//beacon-chain:__subpackages__",
+        "//endtoend:__pkg__",
         "//shared/interop:__pkg__",
         "//shared/testutil:__pkg__",
         "//tools/benchmark-files-gen:__pkg__",
         "//tools/genesis-state-gen:__pkg__",
-        "//endtoend:__pkg__",
+        "//tools/pcli:__pkg__",
     ],
     deps = [
         "//beacon-chain/cache:go_default_library",

beacon-chain/core/state/interop/log.go

@@ -1,3 +1,6 @@
+// Package interop contains useful utilities for persisting
+// ssz-encoded states and blocks to disk during each state
+// transition for development purposes.
 package interop
 
 import (

beacon-chain/core/state/spectest/slot_processing_mainnet_test.go

@@ -1,3 +1,5 @@
+// Package spectest contains all conformity specification tests
+// for slot processing logic according to the eth2 beacon spec.
 package spectest
 
 import (

beacon-chain/core/state/state.go

@@ -1,6 +1,6 @@
 // Package state implements the whole state transition
-// function which consists of per slot, per-epoch transitions.
-// It also bootstraps the genesis beacon state for slot 0.
+// function which consists of per slot, per-epoch transitions, and
+// bootstrapping the genesis state according to the eth2 spec.
 package state
 
 import (

beacon-chain/core/state/stateutils/validator_index_map.go

@@ -1,3 +1,6 @@
+// Package stateutils contains useful tools for faster computation
+// of state transitions using maps to represent validators instead
+// of slices.
 package stateutils
 
 import (

beacon-chain/core/state/transition_fuzz_test.go

@@ -179,9 +179,8 @@ func TestFuzzverifyOperationLengths_10000(t *testing.T) {
 	for i := 0; i < 10000; i++ {
 		fuzzer.Fuzz(state)
 		fuzzer.Fuzz(bb)
-		if err := verifyOperationLengths(state, bb); err != nil {
-			t.Log(err)
-		}
+		err := verifyOperationLengths(state, bb)
+		_ = err
 	}
 }
 

beacon-chain/db/BUILD.bazel

@@ -13,12 +13,12 @@ go_library(
         "alias.go",
         "http_backup_handler.go",
     ] + select({
-        "//conditions:default": [
-            "db_kafka_wrapped.go",
-        ],
         ":kafka_disabled": [
             "db.go",
         ],
+        "//conditions:default": [
+            "db_kafka_wrapped.go",
+        ],
     }),
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/db",
     visibility = [

beacon-chain/db/db.go

@@ -1,3 +1,5 @@
+// Package db defines the ability to create a new database
+// for an eth2 beacon node.
 package db
 
 import (

beacon-chain/db/filters/filter.go

@@ -1,5 +1,5 @@
 // Package filters specifies utilities for building a set of data attribute
-// filters to be used  when filtering data through database queries in practice.
+// filters to be used when filtering data through database queries in practice.
 // For example, one can specify a filter query for data by start epoch + end epoch + shard
 // for attestations, build a filter as follows, and respond to it accordingly:
 //

beacon-chain/db/iface/interface.go

@@ -1,4 +1,6 @@
-// Package iface exists to prevent circular dependencies when implementing the database interface.
+// Package iface defines the actual database interface used
+// by a Prysm beacon node, also containing useful, scoped interfaces such as
+// a ReadOnlyDatabase.
 package iface
 
 import (
@@ -130,4 +132,7 @@ type Database interface {
 
 	// Backup and restore methods
 	Backup(ctx context.Context) error
+
+	// HistoricalStatesDeleted verifies historical states exist in DB.
+	HistoricalStatesDeleted(ctx context.Context) error
 }

beacon-chain/db/kafka/BUILD.bazel

@@ -7,6 +7,7 @@ go_library(
         "passthrough.go",
     ],
     importpath = "github.com/prysmaticlabs/prysm/beacon-chain/db/kafka",
+    tags = ["manual"],
     visibility = ["//beacon-chain/db:__pkg__"],
     deps = [
         "//beacon-chain/db/filters:go_default_library",

beacon-chain/db/kafka/export_wrapper.go

@@ -1,3 +1,5 @@
+// Package kafka defines an implementation of Database interface
+// which exports streaming data using Kafka for data analysis.
 package kafka
 
 import (

beacon-chain/db/kafka/passthrough.go

@@ -357,3 +357,8 @@ func (e Exporter) SaveLastArchivedIndex(ctx context.Context, index uint64) error
 func (e Exporter) LastArchivedIndex(ctx context.Context) (uint64, error) {
 	return e.db.LastArchivedIndex(ctx)
 }
+
+// HistoricalStatesDeleted -- passthrough
+func (e Exporter) HistoricalStatesDeleted(ctx context.Context) error {
+	return e.db.HistoricalStatesDeleted(ctx)
+}

beacon-chain/db/kv/BUILD.bazel

@@ -36,7 +36,6 @@ go_library(
         "//proto/beacon/db:go_default_library",
         "//proto/beacon/p2p/v1:go_default_library",
         "//shared/bytesutil:go_default_library",
-        "//shared/cmd:go_default_library",
         "//shared/featureconfig:go_default_library",
         "//shared/params:go_default_library",
         "//shared/sliceutil:go_default_library",
@@ -46,6 +45,7 @@ go_library(
         "@com_github_ferranbt_fastssz//:go_default_library",
         "@com_github_gogo_protobuf//proto:go_default_library",
         "@com_github_golang_snappy//:go_default_library",
+        "@com_github_hashicorp_golang_lru//:go_default_library",
         "@com_github_pkg_errors//:go_default_library",
         "@com_github_prometheus_client_golang//prometheus:go_default_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_default_library",

beacon-chain/db/kv/blocks.go

@@ -263,7 +263,7 @@ func (k *Store) SaveHeadBlockRoot(ctx context.Context, blockRoot [32]byte) error
 	ctx, span := trace.StartSpan(ctx, "BeaconDB.SaveHeadBlockRoot")
 	defer span.End()
 	return k.db.Update(func(tx *bolt.Tx) error {
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			hasStateSummaryInCache := k.stateSummaryCache.Has(blockRoot)
 			hasStateSummaryInDB := tx.Bucket(stateSummaryBucket).Get(blockRoot[:]) != nil
 			hasStateInDB := tx.Bucket(stateBucket).Get(blockRoot[:]) != nil

beacon-chain/db/kv/check_historical_state.go

@@ -4,15 +4,16 @@ import (
 	"context"
 
 	"github.com/pkg/errors"
-	"github.com/prysmaticlabs/prysm/shared/cmd"
 	"github.com/prysmaticlabs/prysm/shared/featureconfig"
+	log "github.com/sirupsen/logrus"
 	bolt "go.etcd.io/bbolt"
 )
 
 var historicalStateDeletedKey = []byte("historical-states-deleted")
 
-func (kv *Store) ensureNewStateServiceCompatible(ctx context.Context) error {
-	if featureconfig.Get().DisableNewStateMgmt {
+// HistoricalStatesDeleted verifies historical states exist in DB.
+func (kv *Store) HistoricalStatesDeleted(ctx context.Context) error {
+	if !featureconfig.Get().NewStateMgmt {
 		return kv.db.Update(func(tx *bolt.Tx) error {
 			bkt := tx.Bucket(newStateServiceCompatibleBucket)
 			return bkt.Put(historicalStateDeletedKey, []byte{0x01})
@@ -29,22 +30,8 @@ func (kv *Store) ensureNewStateServiceCompatible(ctx context.Context) error {
 		return err
 	}
 
-	regenHistoricalStatesConfirmed := false
-	var err error
 	if historicalStateDeleted {
-		actionText := "--disable-new-state-mgmt was used. To proceed without the flag, the db will need " +
-			"to generate and save historical states. This process may take a while, - do you want to proceed? (Y/N)"
-		deniedText := "Historical states will not be generated. Please continue use --disable-new-state-mgmt"
-
-		regenHistoricalStatesConfirmed, err = cmd.ConfirmAction(actionText, deniedText)
-		if err != nil {
-			return err
-		}
-
-		if !regenHistoricalStatesConfirmed {
-			return errors.New("exiting... please use --disable-new-state-mgmt")
-		}
-
+		log.Warn("Regenerating and saving historical states. This may take a while.")
 		if err := kv.regenHistoricalStates(ctx); err != nil {
 			return errors.Wrap(err, "could not regenerate historical states, please retry")
 		}

beacon-chain/db/kv/checkpoint.go

@@ -65,7 +65,7 @@ func (k *Store) SaveJustifiedCheckpoint(ctx context.Context, checkpoint *ethpb.C
 	}
 	return k.db.Update(func(tx *bolt.Tx) error {
 		bucket := tx.Bucket(checkpointBucket)
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			hasStateSummaryInDB := tx.Bucket(stateSummaryBucket).Get(checkpoint.Root) != nil
 			hasStateSummaryInCache := k.stateSummaryCache.Has(bytesutil.ToBytes32(checkpoint.Root))
 			hasStateInDB := tx.Bucket(stateBucket).Get(checkpoint.Root) != nil
@@ -96,7 +96,7 @@ func (k *Store) SaveFinalizedCheckpoint(ctx context.Context, checkpoint *ethpb.C
 	}
 	return k.db.Update(func(tx *bolt.Tx) error {
 		bucket := tx.Bucket(checkpointBucket)
-		if !featureconfig.Get().DisableNewStateMgmt {
+		if featureconfig.Get().NewStateMgmt {
 			hasStateSummaryInDB := tx.Bucket(stateSummaryBucket).Get(checkpoint.Root) != nil
 			hasStateSummaryInCache := k.stateSummaryCache.Has(bytesutil.ToBytes32(checkpoint.Root))
 			hasStateInDB := tx.Bucket(stateBucket).Get(checkpoint.Root) != nil

beacon-chain/db/kv/kv.go

@@ -1,7 +1,8 @@
+// Package kv defines a bolt-db, key-value store implementation
+// of the Database interface defined by a Prysm beacon node.
 package kv
 
 import (
-	"context"
 	"os"
 	"path"
 	"sync"
@@ -120,10 +121,6 @@ func NewKVStore(dirPath string, stateSummaryCache *cache.StateSummaryCache) (*St
 		return nil, err
 	}
 
-	if err := kv.ensureNewStateServiceCompatible(context.Background()); err != nil {
-		return nil, err
-	}
-
 	err = prometheus.Register(createBoltCollector(kv.db))
 
 	return kv, err

beacon-chain/db/kv/regen_historical_states.go

@@ -3,7 +3,9 @@ package kv
 import (
 	"context"
 	"fmt"
+	"runtime"
 
+	lru "github.com/hashicorp/golang-lru"
 	"github.com/pkg/errors"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	"github.com/prysmaticlabs/go-ssz"
@@ -11,11 +13,17 @@ import (
 	transition "github.com/prysmaticlabs/prysm/beacon-chain/core/state"
 	"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
 	stateTrie "github.com/prysmaticlabs/prysm/beacon-chain/state"
+	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	log "github.com/sirupsen/logrus"
 	"go.opencensus.io/trace"
 )
 
+// Using max possible size to avoid using DB to save and retrieve pre state (slow)
+// The size is 80 because block at slot 43772 built on top of block at slot 43693.
+// That is the worst case.
+const historicalStatesSize = 80
+
 func (kv *Store) regenHistoricalStates(ctx context.Context) error {
 	ctx, span := trace.StartSpan(ctx, "db.regenHistoricalStates")
 	defer span.End()
@@ -54,7 +62,17 @@ func (kv *Store) regenHistoricalStates(ctx context.Context) error {
 	if err != nil {
 		return err
 	}
+
+	cacheState, err := lru.New(historicalStatesSize)
+	if err != nil {
+		return err
+	}
 	for i := lastArchivedIndex; i <= lastSavedBlockArchivedIndex; i++ {
+		// This is an expensive operation, so we check if the context was canceled
+		// at any point in the iteration.
+		if err := ctx.Err(); err != nil {
+			return err
+		}
 		targetSlot := startSlot + slotsPerArchivedPoint
 		filter := filters.NewFilter().SetStartSlot(startSlot + 1).SetEndSlot(targetSlot)
 		blocks, err := kv.Blocks(ctx, filter)
@@ -68,23 +86,44 @@ func (kv *Store) regenHistoricalStates(ctx context.Context) error {
 				if blocks[i].Block.Slot == 0 {
 					continue
 				}
-				currentState, err = regenHistoricalStateTransition(ctx, currentState, blocks[i])
+
+				var preState *stateTrie.BeaconState
+				item, ok := cacheState.Get(bytesutil.ToBytes32(blocks[i].Block.ParentRoot))
+				if !ok {
+					preState, err = kv.State(ctx, bytesutil.ToBytes32(blocks[i].Block.ParentRoot))
+					if err != nil {
+						return err
+					}
+				} else {
+					preState = item.(*stateTrie.BeaconState).Copy()
+				}
+				if preState == nil {
+					return errors.New("pre state can't be nil")
+				}
+
+				currentState, err = regenHistoricalStateTransition(ctx, preState.Copy(), blocks[i])
+				if err != nil {
+					return errors.Wrap(err, "could not regenerate historical state transition")
+				}
+
+				r, err := ssz.HashTreeRoot(blocks[i].Block)
 				if err != nil {
 					return err
 				}
+				cacheState.Add(r, currentState)
 			}
 		}
 		if targetSlot > currentState.Slot() {
 			currentState, err = regenHistoricalStateProcessSlots(ctx, currentState, targetSlot)
 			if err != nil {
-				return err
+				return errors.Wrap(err, "could not regenerate historical process slot")
 			}
 		}
 
 		if len(blocks) > 0 {
 			// Save the historical root, state and highest index to the DB.
 			if helpers.IsEpochStart(currentState.Slot()) && currentState.Slot()%slotsPerArchivedPoint == 0 && blocks[len(blocks)-1].Block.Slot&slotsPerArchivedPoint == 0 {
-				if err := kv.saveArchivedInfo(ctx, currentState, blocks, i); err != nil {
+				if err := kv.saveArchivedInfo(ctx, currentState.Copy(), blocks, i); err != nil {
 					return err
 				}
 				log.WithFields(log.Fields{
@@ -94,6 +133,13 @@ func (kv *Store) regenHistoricalStates(ctx context.Context) error {
 		}
 		startSlot += slotsPerArchivedPoint
 	}
+
+	// Flush the cache, the cached states never be used again.
+	cacheState.Purge()
+
+	// Manually garbage collect as previous cache will never be used again.
+	runtime.GC()
+
 	return nil
 }
 

beacon-chain/db/testing/setup_db.go

@@ -1,3 +1,5 @@
+// Package testing allows for spinning up a real bolt-db
+// instance for unit tests throughout the Prysm repo.
 package testing
 
 import (

beacon-chain/flags/base.go

@@ -1,3 +1,5 @@
+// Package flags defines beacon-node specific runtime flags for
+// setting important values such as ports, eth1 endpoints, and more.
 package flags
 
 import (
@@ -42,6 +44,12 @@ var (
 		Usage: "Max number of items returned per page in RPC responses for paginated endpoints.",
 		Value: 500,
 	}
+	// MonitoringPortFlag defines the http port used to serve prometheus metrics.
+	MonitoringPortFlag = &cli.Int64Flag{
+		Name:  "monitoring-port",
+		Usage: "Port used to listening and respond metrics for prometheus.",
+		Value: 8080,
+	}
 	// CertFlag defines a flag for the node's TLS certificate.
 	CertFlag = &cli.StringFlag{
 		Name:  "tls-cert",
@@ -110,4 +118,15 @@ var (
 		Name:  "disable-discv5",
 		Usage: "Does not run the discoveryV5 dht.",
 	}
+	// BlockBatchLimit specifies the requested block batch size.
+	BlockBatchLimit = &cli.IntFlag{
+		Name:  "block-batch-limit",
+		Usage: "The amount of blocks the local peer is bounded to request and respond to in a batch.",
+		Value: 64,
+	}
+	// EnableDebugRPCEndpoints as /v1/beacon/state.
+	EnableDebugRPCEndpoints = &cli.BoolFlag{
+		Name:  "enable-debug-rpc-endpoints",
+		Usage: "Enables the debug rpc service, containing utility endpoints such as /eth/v1alpha1/beacon/state. Requires --new-state-mgmt",
+	}
 )

beacon-chain/flags/config.go

@@ -18,6 +18,7 @@ type GlobalFlags struct {
 	MinimumSyncPeers                  int
 	MaxPageSize                       int
 	DeploymentBlock                   int
+	BlockBatchLimit                   int
 }
 
 var globalConfig *GlobalFlags
@@ -57,6 +58,7 @@ func ConfigureGlobalFlags(ctx *cli.Context) {
 	if ctx.Bool(DisableDiscv5.Name) {
 		cfg.DisableDiscv5 = true
 	}
+	cfg.BlockBatchLimit = ctx.Int(BlockBatchLimit.Name)
 	cfg.MaxPageSize = ctx.Int(RPCMaxPageSize.Name)
 	cfg.DeploymentBlock = ctx.Int(ContractDeploymentBlock.Name)
 	configureMinimumPeers(ctx, cfg)

beacon-chain/forkchoice/protoarray/doc.go

@@ -1,7 +1,7 @@
 /*
 Package protoarray implements proto array fork choice as outlined:
 https://github.com/protolambda/lmd-ghost#array-based-stateful-dag-proto_array
-This was motivated by the following light house implementation:
+This was motivated by the the original implementation by Sigma Prime here:
 https://github.com/sigp/lighthouse/pull/804
 */
 package protoarray

beacon-chain/gateway/BUILD.bazel

@@ -16,6 +16,7 @@ go_library(
     ],
     deps = [
         "//shared:go_default_library",
+        "//proto/beacon/rpc/v1:go_grpc_gateway_library",
         "@com_github_prysmaticlabs_ethereumapis//eth/v1alpha1:go_grpc_gateway_library",
         "@com_github_rs_cors//:go_default_library",
         "@com_github_sirupsen_logrus//:go_default_library",

beacon-chain/gateway/gateway.go

@@ -1,3 +1,5 @@
+// Package gateway defines a gRPC gateway to serve HTTP-JSON
+// traffic as a proxy and forward it to a beacon node's gRPC service.
 package gateway
 
 import (
@@ -9,6 +11,7 @@ import (
 
 	gwruntime "github.com/grpc-ecosystem/grpc-gateway/runtime"
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1_gateway"
+	pbrpc "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1_gateway"
 	"github.com/prysmaticlabs/prysm/shared"
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/connectivity"
@@ -19,16 +22,16 @@ var _ = shared.Service(&Gateway{})
 // Gateway is the gRPC gateway to serve HTTP JSON traffic as a proxy and forward
 // it to the beacon-chain gRPC server.
 type Gateway struct {
-	conn           *grpc.ClientConn
-	ctx            context.Context
-	cancel         context.CancelFunc
-	gatewayAddr    string
-	remoteAddr     string
-	server         *http.Server
-	mux            *http.ServeMux
-	allowedOrigins []string
-
-	startFailure error
+	conn                    *grpc.ClientConn
+	ctx                     context.Context
+	cancel                  context.CancelFunc
+	gatewayAddr             string
+	remoteAddr              string
+	server                  *http.Server
+	mux                     *http.ServeMux
+	allowedOrigins          []string
+	startFailure            error
+	enableDebugRPCEndpoints bool
 }
 
 // Start the gateway service. This serves the HTTP JSON traffic on the specified
@@ -48,12 +51,21 @@ func (g *Gateway) Start() {
 
 	g.conn = conn
 
-	gwmux := gwruntime.NewServeMux(gwruntime.WithMarshalerOption(gwruntime.MIMEWildcard, &gwruntime.JSONPb{OrigName: false, EmitDefaults: true}))
-	for _, f := range []func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error{
+	gwmux := gwruntime.NewServeMux(
+		gwruntime.WithMarshalerOption(
+			gwruntime.MIMEWildcard,
+			&gwruntime.JSONPb{OrigName: false, EmitDefaults: true},
+		),
+	)
+	handlers := []func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error{
 		ethpb.RegisterNodeHandler,
 		ethpb.RegisterBeaconChainHandler,
 		ethpb.RegisterBeaconNodeValidatorHandler,
-	} {
+	}
+	if g.enableDebugRPCEndpoints {
+		handlers = append(handlers, pbrpc.RegisterDebugHandler)
+	}
+	for _, f := range handlers {
 		if err := f(ctx, gwmux, conn); err != nil {
 			log.WithError(err).Error("Failed to start gateway")
 			g.startFailure = err
@@ -106,17 +118,25 @@ func (g *Gateway) Stop() error {
 
 // New returns a new gateway server which translates HTTP into gRPC.
 // Accepts a context and optional http.ServeMux.
-func New(ctx context.Context, remoteAddress, gatewayAddress string, mux *http.ServeMux, allowedOrigins []string) *Gateway {
+func New(
+	ctx context.Context,
+	remoteAddress,
+	gatewayAddress string,
+	mux *http.ServeMux,
+	allowedOrigins []string,
+	enableDebugRPCEndpoints bool,
+) *Gateway {
 	if mux == nil {
 		mux = http.NewServeMux()
 	}
 
 	return &Gateway{
-		remoteAddr:     remoteAddress,
-		gatewayAddr:    gatewayAddress,
-		ctx:            ctx,
-		mux:            mux,
-		allowedOrigins: allowedOrigins,
+		remoteAddr:              remoteAddress,
+		gatewayAddr:             gatewayAddress,
+		ctx:                     ctx,
+		mux:                     mux,
+		allowedOrigins:          allowedOrigins,
+		enableDebugRPCEndpoints: enableDebugRPCEndpoints,
 	}
 }
 

beacon-chain/gateway/server/main.go

@@ -1,3 +1,5 @@
+// Package main allows for creation of an HTTP-JSON to gRPC
+// gateway as a binary go process.
 package main
 
 import (
@@ -14,10 +16,11 @@ import (
 )
 
 var (
-	beaconRPC      = flag.String("beacon-rpc", "localhost:4000", "Beacon chain gRPC endpoint")
-	port           = flag.Int("port", 8000, "Port to serve on")
-	debug          = flag.Bool("debug", false, "Enable debug logging")
-	allowedOrigins = flag.String("corsdomain", "", "A comma separated list of CORS domains to allow.")
+	beaconRPC               = flag.String("beacon-rpc", "localhost:4000", "Beacon chain gRPC endpoint")
+	port                    = flag.Int("port", 8000, "Port to serve on")
+	debug                   = flag.Bool("debug", false, "Enable debug logging")
+	allowedOrigins          = flag.String("corsdomain", "", "A comma separated list of CORS domains to allow")
+	enableDebugRPCEndpoints = flag.Bool("enable-debug-rpc-endpoints", false, "Enable debug rpc endpoints such as /eth/v1alpha1/beacon/state")
 )
 
 func init() {
@@ -33,7 +36,14 @@ func main() {
 	}
 
 	mux := http.NewServeMux()
-	gw := gateway.New(context.Background(), *beaconRPC, fmt.Sprintf("0.0.0.0:%d", *port), mux, strings.Split(*allowedOrigins, ","))
+	gw := gateway.New(
+		context.Background(),
+		*beaconRPC,
+		fmt.Sprintf("0.0.0.0:%d", *port),
+		mux,
+		strings.Split(*allowedOrigins, ","),
+		*enableDebugRPCEndpoints,
+	)
 	mux.HandleFunc("/swagger/", gateway.SwaggerServer())
 	mux.HandleFunc("/healthz", healthzServer(gw))
 	gw.Start()

beacon-chain/interop-cold-start/service.go

@@ -1,3 +1,6 @@
+// Package interopcoldstart allows for spinning up a deterministic
+// local chain without the need for eth1 deposits useful for
+// local client development and interoperability testing.
 package interopcoldstart
 
 import (

beacon-chain/main.go

@@ -1,4 +1,4 @@
-// Package beacon-chain defines all the utilities needed for a beacon chain node.
+// Package beacon-chain defines the entire runtime of an eth2 beacon node.
 package main
 
 import (
@@ -7,6 +7,7 @@ import (
 	"runtime"
 	runtimeDebug "runtime/debug"
 
+	gethlog "github.com/ethereum/go-ethereum/log"
 	golog "github.com/ipfs/go-log"
 	joonix "github.com/joonix/log"
 	"github.com/prysmaticlabs/prysm/beacon-chain/flags"
@@ -39,6 +40,7 @@ var appFlags = []cli.Flag{
 	flags.SetGCPercent,
 	flags.UnsafeSync,
 	flags.DisableDiscv5,
+	flags.BlockBatchLimit,
 	flags.InteropMockEth1DataVotesFlag,
 	flags.InteropGenesisStateFlag,
 	flags.InteropNumValidatorsFlag,
@@ -48,6 +50,7 @@ var appFlags = []cli.Flag{
 	flags.ArchiveBlocksFlag,
 	flags.ArchiveAttestationsFlag,
 	flags.SlotsPerArchivedPoint,
+	flags.EnableDebugRPCEndpoints,
 	cmd.BootstrapNode,
 	cmd.NoDiscovery,
 	cmd.StaticPeers,
@@ -62,13 +65,14 @@ var appFlags = []cli.Flag{
 	cmd.P2PMetadata,
 	cmd.P2PWhitelist,
 	cmd.P2PEncoding,
+	cmd.P2PPubsub,
 	cmd.DataDirFlag,
 	cmd.VerbosityFlag,
 	cmd.EnableTracingFlag,
 	cmd.TracingProcessNameFlag,
 	cmd.TracingEndpointFlag,
 	cmd.TraceSampleFractionFlag,
-	cmd.MonitoringPortFlag,
+	flags.MonitoringPortFlag,
 	cmd.DisableMonitoringFlag,
 	cmd.ClearDB,
 	cmd.ForceClearDB,
@@ -167,7 +171,12 @@ func startNode(ctx *cli.Context) error {
 	}
 	logrus.SetLevel(level)
 	if level == logrus.TraceLevel {
+		// libp2p specific logging.
 		golog.SetAllLoggers(gologging.DEBUG)
+		// Geth specific logging.
+		glogger := gethlog.NewGlogHandler(gethlog.StreamHandler(os.Stderr, gethlog.TerminalFormat(true)))
+		glogger.Verbosity(gethlog.LvlTrace)
+		gethlog.Root().SetHandler(glogger)
 	}
 
 	beacon, err := node.NewBeaconNode(ctx)

beacon-chain/node/node.go

@@ -1,4 +1,6 @@
-// Package node defines the services that a beacon chain node would perform.
+// Package node is the main service which launches a beacon node and manages
+// the lifecycle of all its associated services at runtime, such as p2p, RPC, sync,
+// gracefully closing them if the process ends.
 package node
 
 import (
@@ -59,7 +61,9 @@ const testSkipPowFlag = "test-skip-pow"
 // full PoS node. It handles the lifecycle of the entire system and registers
 // services to a service registry.
 type BeaconNode struct {
-	ctx               *cli.Context
+	cliCtx            *cli.Context
+	ctx               context.Context
+	cancel            context.CancelFunc
 	services          *shared.ServiceRegistry
 	lock              sync.RWMutex
 	stop              chan struct{} // Channel to wait for termination notifications.
@@ -78,23 +82,26 @@ type BeaconNode struct {
 
 // NewBeaconNode creates a new node instance, sets up configuration options, and registers
 // every required service to the node.
-func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
+func NewBeaconNode(cliCtx *cli.Context) (*BeaconNode, error) {
 	if err := tracing.Setup(
 		"beacon-chain", // service name
-		ctx.String(cmd.TracingProcessNameFlag.Name),
-		ctx.String(cmd.TracingEndpointFlag.Name),
-		ctx.Float64(cmd.TraceSampleFractionFlag.Name),
-		ctx.Bool(cmd.EnableTracingFlag.Name),
+		cliCtx.String(cmd.TracingProcessNameFlag.Name),
+		cliCtx.String(cmd.TracingEndpointFlag.Name),
+		cliCtx.Float64(cmd.TraceSampleFractionFlag.Name),
+		cliCtx.Bool(cmd.EnableTracingFlag.Name),
 	); err != nil {
 		return nil, err
 	}
 
-	featureconfig.ConfigureBeaconChain(ctx)
-	flags.ConfigureGlobalFlags(ctx)
+	featureconfig.ConfigureBeaconChain(cliCtx)
+	flags.ConfigureGlobalFlags(cliCtx)
 	registry := shared.NewServiceRegistry()
 
+	ctx, cancel := context.WithCancel(cliCtx)
 	beacon := &BeaconNode{
+		cliCtx:            cliCtx,
 		ctx:               ctx,
+		cancel:            cancel,
 		services:          registry,
 		stop:              make(chan struct{}),
 		stateFeed:         new(event.Feed),
@@ -106,17 +113,17 @@ func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
 		stateSummaryCache: cache.NewStateSummaryCache(),
 	}
 
-	if err := beacon.startDB(ctx); err != nil {
+	if err := beacon.startDB(cliCtx); err != nil {
 		return nil, err
 	}
 
 	beacon.startStateGen()
 
-	if err := beacon.registerP2P(ctx); err != nil {
+	if err := beacon.registerP2P(cliCtx); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerPOWChainService(ctx); err != nil {
+	if err := beacon.registerPOWChainService(); err != nil {
 		return nil, err
 	}
 
@@ -124,38 +131,38 @@ func NewBeaconNode(ctx *cli.Context) (*BeaconNode, error) {
 		return nil, err
 	}
 
-	if err := beacon.registerInteropServices(ctx); err != nil {
+	if err := beacon.registerInteropServices(); err != nil {
 		return nil, err
 	}
 
 	beacon.startForkChoice()
 
-	if err := beacon.registerBlockchainService(ctx); err != nil {
+	if err := beacon.registerBlockchainService(); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerInitialSyncService(ctx); err != nil {
+	if err := beacon.registerInitialSyncService(); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerSyncService(ctx); err != nil {
+	if err := beacon.registerSyncService(); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerRPCService(ctx); err != nil {
+	if err := beacon.registerRPCService(); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerGRPCGateway(ctx); err != nil {
+	if err := beacon.registerGRPCGateway(); err != nil {
 		return nil, err
 	}
 
-	if err := beacon.registerArchiverService(ctx); err != nil {
+	if err := beacon.registerArchiverService(); err != nil {
 		return nil, err
 	}
 
-	if !ctx.Bool(cmd.DisableMonitoringFlag.Name) {
-		if err := beacon.registerPrometheusService(ctx); err != nil {
+	if !cliCtx.Bool(cmd.DisableMonitoringFlag.Name) {
+		if err := beacon.registerPrometheusService(); err != nil {
 			return nil, err
 		}
 	}
@@ -197,7 +204,7 @@ func (b *BeaconNode) Start() {
 		defer signal.Stop(sigc)
 		<-sigc
 		log.Info("Got interrupt, shutting down...")
-		debug.Exit(b.ctx) // Ensure trace and CPU profile data are flushed.
+		debug.Exit(b.cliCtx) // Ensure trace and CPU profile data are flushed.
 		go b.Close()
 		for i := 10; i > 0; i-- {
 			<-sigc
@@ -218,6 +225,7 @@ func (b *BeaconNode) Close() {
 	defer b.lock.Unlock()
 
 	log.Info("Stopping beacon node")
+	b.cancel() // Cancel the beacon node struct's context.
 	b.services.StopAll()
 	if err := b.db.Close(); err != nil {
 		log.Errorf("Failed to close database: %v", err)
@@ -230,11 +238,11 @@ func (b *BeaconNode) startForkChoice() {
 	b.forkChoiceStore = f
 }
 
-func (b *BeaconNode) startDB(ctx *cli.Context) error {
-	baseDir := ctx.String(cmd.DataDirFlag.Name)
+func (b *BeaconNode) startDB(cliCtx *cli.Context) error {
+	baseDir := cliCtx.String(cmd.DataDirFlag.Name)
 	dbPath := path.Join(baseDir, beaconChainDBName)
-	clearDB := ctx.Bool(cmd.ClearDB.Name)
-	forceClearDB := ctx.Bool(cmd.ForceClearDB.Name)
+	clearDB := cliCtx.Bool(cmd.ClearDB.Name)
+	forceClearDB := cliCtx.Bool(cmd.ForceClearDB.Name)
 
 	d, err := db.NewDB(dbPath, b.stateSummaryCache)
 	if err != nil {
@@ -259,7 +267,12 @@ func (b *BeaconNode) startDB(ctx *cli.Context) error {
 		if err != nil {
 			return err
 		}
+	} else {
+		if err := d.HistoricalStatesDeleted(b.ctx); err != nil {
+			return err
+		}
 	}
+
 	log.WithField("database-path", dbPath).Info("Checking DB")
 	b.db = d
 	b.depositCache = depositcache.NewDepositCache()
@@ -270,9 +283,9 @@ func (b *BeaconNode) startStateGen() {
 	b.stateGen = stategen.New(b.db, b.stateSummaryCache)
 }
 
-func (b *BeaconNode) registerP2P(ctx *cli.Context) error {
+func (b *BeaconNode) registerP2P(cliCtx *cli.Context) error {
 	// Bootnode ENR may be a filepath to an ENR file.
-	bootnodeAddrs := strings.Split(ctx.String(cmd.BootstrapNode.Name), ",")
+	bootnodeAddrs := strings.Split(cliCtx.String(cmd.BootstrapNode.Name), ",")
 	for i, addr := range bootnodeAddrs {
 		if filepath.Ext(addr) == ".enr" {
 			b, err := ioutil.ReadFile(addr)
@@ -283,30 +296,31 @@ func (b *BeaconNode) registerP2P(ctx *cli.Context) error {
 		}
 	}
 
-	datadir := ctx.String(cmd.DataDirFlag.Name)
+	datadir := cliCtx.String(cmd.DataDirFlag.Name)
 	if datadir == "" {
 		datadir = cmd.DefaultDataDir()
 	}
 
 	svc, err := p2p.NewService(&p2p.Config{
-		NoDiscovery:       ctx.Bool(cmd.NoDiscovery.Name),
-		StaticPeers:       sliceutil.SplitCommaSeparated(ctx.StringSlice(cmd.StaticPeers.Name)),
+		NoDiscovery:       cliCtx.Bool(cmd.NoDiscovery.Name),
+		StaticPeers:       sliceutil.SplitCommaSeparated(cliCtx.StringSlice(cmd.StaticPeers.Name)),
 		BootstrapNodeAddr: bootnodeAddrs,
-		RelayNodeAddr:     ctx.String(cmd.RelayNode.Name),
+		RelayNodeAddr:     cliCtx.String(cmd.RelayNode.Name),
 		DataDir:           datadir,
-		LocalIP:           ctx.String(cmd.P2PIP.Name),
-		HostAddress:       ctx.String(cmd.P2PHost.Name),
-		HostDNS:           ctx.String(cmd.P2PHostDNS.Name),
-		PrivateKey:        ctx.String(cmd.P2PPrivKey.Name),
-		MetaDataDir:       ctx.String(cmd.P2PMetadata.Name),
-		TCPPort:           ctx.Uint(cmd.P2PTCPPort.Name),
-		UDPPort:           ctx.Uint(cmd.P2PUDPPort.Name),
-		MaxPeers:          ctx.Uint(cmd.P2PMaxPeers.Name),
-		WhitelistCIDR:     ctx.String(cmd.P2PWhitelist.Name),
-		EnableUPnP:        ctx.Bool(cmd.EnableUPnPFlag.Name),
-		DisableDiscv5:     ctx.Bool(flags.DisableDiscv5.Name),
-		Encoding:          ctx.String(cmd.P2PEncoding.Name),
+		LocalIP:           cliCtx.String(cmd.P2PIP.Name),
+		HostAddress:       cliCtx.String(cmd.P2PHost.Name),
+		HostDNS:           cliCtx.String(cmd.P2PHostDNS.Name),
+		PrivateKey:        cliCtx.String(cmd.P2PPrivKey.Name),
+		MetaDataDir:       cliCtx.String(cmd.P2PMetadata.Name),
+		TCPPort:           cliCtx.Uint(cmd.P2PTCPPort.Name),
+		UDPPort:           cliCtx.Uint(cmd.P2PUDPPort.Name),
+		MaxPeers:          cliCtx.Uint(cmd.P2PMaxPeers.Name),
+		WhitelistCIDR:     cliCtx.String(cmd.P2PWhitelist.Name),
+		EnableUPnP:        cliCtx.Bool(cmd.EnableUPnPFlag.Name),
+		DisableDiscv5:     cliCtx.Bool(flags.DisableDiscv5.Name),
+		Encoding:          cliCtx.String(cmd.P2PEncoding.Name),
 		StateNotifier:     b,
+		PubSub:            cliCtx.String(cmd.P2PPubsub.Name),
 	})
 	if err != nil {
 		return err
@@ -314,7 +328,7 @@ func (b *BeaconNode) registerP2P(ctx *cli.Context) error {
 	return b.services.RegisterService(svc)
 }
 
-func (b *BeaconNode) fetchP2P(ctx *cli.Context) p2p.P2P {
+func (b *BeaconNode) fetchP2P() p2p.P2P {
 	var p *p2p.Service
 	if err := b.services.FetchService(&p); err != nil {
 		panic(err)
@@ -323,7 +337,7 @@ func (b *BeaconNode) fetchP2P(ctx *cli.Context) p2p.P2P {
 }
 
 func (b *BeaconNode) registerAttestationPool() error {
-	s, err := attestations.NewService(context.Background(), &attestations.Config{
+	s, err := attestations.NewService(b.ctx, &attestations.Config{
 		Pool: b.attestationPool,
 	})
 	if err != nil {
@@ -332,7 +346,7 @@ func (b *BeaconNode) registerAttestationPool() error {
 	return b.services.RegisterService(s)
 }
 
-func (b *BeaconNode) registerBlockchainService(ctx *cli.Context) error {
+func (b *BeaconNode) registerBlockchainService() error {
 	var web3Service *powchain.Service
 	if err := b.services.FetchService(&web3Service); err != nil {
 		return err
@@ -343,15 +357,15 @@ func (b *BeaconNode) registerBlockchainService(ctx *cli.Context) error {
 		return err
 	}
 
-	maxRoutines := ctx.Int64(cmd.MaxGoroutines.Name)
-	blockchainService, err := blockchain.NewService(context.Background(), &blockchain.Config{
+	maxRoutines := b.cliCtx.Int64(cmd.MaxGoroutines.Name)
+	blockchainService, err := blockchain.NewService(b.ctx, &blockchain.Config{
 		BeaconDB:          b.db,
 		DepositCache:      b.depositCache,
 		ChainStartFetcher: web3Service,
 		AttPool:           b.attestationPool,
 		ExitPool:          b.exitPool,
 		SlashingPool:      b.slashingsPool,
-		P2p:               b.fetchP2P(ctx),
+		P2p:               b.fetchP2P(),
 		MaxRoutines:       maxRoutines,
 		StateNotifier:     b,
 		ForkChoiceStore:   b.forkChoiceStore,
@@ -364,11 +378,11 @@ func (b *BeaconNode) registerBlockchainService(ctx *cli.Context) error {
 	return b.services.RegisterService(blockchainService)
 }
 
-func (b *BeaconNode) registerPOWChainService(cliCtx *cli.Context) error {
-	if cliCtx.Bool(testSkipPowFlag) {
+func (b *BeaconNode) registerPOWChainService() error {
+	if b.cliCtx.Bool(testSkipPowFlag) {
 		return b.services.RegisterService(&powchain.Service{})
 	}
-	depAddress := cliCtx.String(flags.DepositContractFlag.Name)
+	depAddress := b.cliCtx.String(flags.DepositContractFlag.Name)
 	if depAddress == "" {
 		log.Fatal(fmt.Sprintf("%s is required", flags.DepositContractFlag.Name))
 	}
@@ -377,25 +391,24 @@ func (b *BeaconNode) registerPOWChainService(cliCtx *cli.Context) error {
 		log.Fatalf("Invalid deposit contract address given: %s", depAddress)
 	}
 
-	ctx := context.Background()
 	cfg := &powchain.Web3ServiceConfig{
-		ETH1Endpoint:    cliCtx.String(flags.Web3ProviderFlag.Name),
-		HTTPEndPoint:    cliCtx.String(flags.HTTPWeb3ProviderFlag.Name),
+		ETH1Endpoint:    b.cliCtx.String(flags.Web3ProviderFlag.Name),
+		HTTPEndPoint:    b.cliCtx.String(flags.HTTPWeb3ProviderFlag.Name),
 		DepositContract: common.HexToAddress(depAddress),
 		BeaconDB:        b.db,
 		DepositCache:    b.depositCache,
 		StateNotifier:   b,
 	}
-	web3Service, err := powchain.NewService(ctx, cfg)
+	web3Service, err := powchain.NewService(b.ctx, cfg)
 	if err != nil {
 		return errors.Wrap(err, "could not register proof-of-work chain web3Service")
 	}
-	knownContract, err := b.db.DepositContractAddress(ctx)
+	knownContract, err := b.db.DepositContractAddress(b.ctx)
 	if err != nil {
 		return err
 	}
 	if len(knownContract) == 0 {
-		if err := b.db.SaveDepositContractAddress(ctx, cfg.DepositContract); err != nil {
+		if err := b.db.SaveDepositContractAddress(b.ctx, cfg.DepositContract); err != nil {
 			return errors.Wrap(err, "could not save deposit contract")
 		}
 	}
@@ -405,7 +418,7 @@ func (b *BeaconNode) registerPOWChainService(cliCtx *cli.Context) error {
 	return b.services.RegisterService(web3Service)
 }
 
-func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
+func (b *BeaconNode) registerSyncService() error {
 	var web3Service *powchain.Service
 	if err := b.services.FetchService(&web3Service); err != nil {
 		return err
@@ -433,7 +446,7 @@ func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
 
 	rs := prysmsync.NewRegularSync(&prysmsync.Config{
 		DB:                  b.db,
-		P2P:                 b.fetchP2P(ctx),
+		P2P:                 b.fetchP2P(),
 		Chain:               chainService,
 		InitialSync:         initSync,
 		StateNotifier:       b,
@@ -449,7 +462,7 @@ func (b *BeaconNode) registerSyncService(ctx *cli.Context) error {
 	return b.services.RegisterService(rs)
 }
 
-func (b *BeaconNode) registerInitialSyncService(ctx *cli.Context) error {
+func (b *BeaconNode) registerInitialSyncService() error {
 	var chainService *blockchain.Service
 	if err := b.services.FetchService(&chainService); err != nil {
 		return err
@@ -459,7 +472,7 @@ func (b *BeaconNode) registerInitialSyncService(ctx *cli.Context) error {
 		is := initialsyncold.NewInitialSync(&initialsyncold.Config{
 			DB:            b.db,
 			Chain:         chainService,
-			P2P:           b.fetchP2P(ctx),
+			P2P:           b.fetchP2P(),
 			StateNotifier: b,
 			BlockNotifier: b,
 		})
@@ -469,14 +482,14 @@ func (b *BeaconNode) registerInitialSyncService(ctx *cli.Context) error {
 	is := initialsync.NewInitialSync(&initialsync.Config{
 		DB:            b.db,
 		Chain:         chainService,
-		P2P:           b.fetchP2P(ctx),
+		P2P:           b.fetchP2P(),
 		StateNotifier: b,
 		BlockNotifier: b,
 	})
 	return b.services.RegisterService(is)
 }
 
-func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
+func (b *BeaconNode) registerRPCService() error {
 	var chainService *blockchain.Service
 	if err := b.services.FetchService(&chainService); err != nil {
 		return err
@@ -502,8 +515,8 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
 		syncService = initSyncTmp
 	}
 
-	genesisValidators := ctx.Uint64(flags.InteropNumValidatorsFlag.Name)
-	genesisStatePath := ctx.String(flags.InteropGenesisStateFlag.Name)
+	genesisValidators := b.cliCtx.Uint64(flags.InteropNumValidatorsFlag.Name)
+	genesisStatePath := b.cliCtx.String(flags.InteropGenesisStateFlag.Name)
 	var depositFetcher depositcache.DepositFetcher
 	var chainStartFetcher powchain.ChainStartFetcher
 	if genesisValidators > 0 || genesisStatePath != "" {
@@ -518,50 +531,53 @@ func (b *BeaconNode) registerRPCService(ctx *cli.Context) error {
 		chainStartFetcher = web3Service
 	}
 
-	host := ctx.String(flags.RPCHost.Name)
-	port := ctx.String(flags.RPCPort.Name)
-	cert := ctx.String(flags.CertFlag.Name)
-	key := ctx.String(flags.KeyFlag.Name)
-	slasherCert := ctx.String(flags.SlasherCertFlag.Name)
-	slasherProvider := ctx.String(flags.SlasherProviderFlag.Name)
-
-	mockEth1DataVotes := ctx.Bool(flags.InteropMockEth1DataVotesFlag.Name)
-	rpcService := rpc.NewService(context.Background(), &rpc.Config{
-		Host:                  host,
-		Port:                  port,
-		CertFlag:              cert,
-		KeyFlag:               key,
-		BeaconDB:              b.db,
-		Broadcaster:           b.fetchP2P(ctx),
-		PeersFetcher:          b.fetchP2P(ctx),
-		HeadFetcher:           chainService,
-		ForkFetcher:           chainService,
-		FinalizationFetcher:   chainService,
-		ParticipationFetcher:  chainService,
-		BlockReceiver:         chainService,
-		AttestationReceiver:   chainService,
-		GenesisTimeFetcher:    chainService,
-		AttestationsPool:      b.attestationPool,
-		ExitPool:              b.exitPool,
-		SlashingsPool:         b.slashingsPool,
-		POWChainService:       web3Service,
-		ChainStartFetcher:     chainStartFetcher,
-		MockEth1Votes:         mockEth1DataVotes,
-		SyncService:           syncService,
-		DepositFetcher:        depositFetcher,
-		PendingDepositFetcher: b.depositCache,
-		BlockNotifier:         b,
-		StateNotifier:         b,
-		OperationNotifier:     b,
-		SlasherCert:           slasherCert,
-		SlasherProvider:       slasherProvider,
-		StateGen:              b.stateGen,
+	host := b.cliCtx.String(flags.RPCHost.Name)
+	port := b.cliCtx.String(flags.RPCPort.Name)
+	cert := b.cliCtx.String(flags.CertFlag.Name)
+	key := b.cliCtx.String(flags.KeyFlag.Name)
+	slasherCert := b.cliCtx.String(flags.SlasherCertFlag.Name)
+	slasherProvider := b.cliCtx.String(flags.SlasherProviderFlag.Name)
+	mockEth1DataVotes := b.cliCtx.Bool(flags.InteropMockEth1DataVotesFlag.Name)
+	enableDebugRPCEndpoints := b.cliCtx.Bool(flags.EnableDebugRPCEndpoints.Name)
+	p2pService := b.fetchP2P()
+	rpcService := rpc.NewService(b.ctx, &rpc.Config{
+		Host:                    host,
+		Port:                    port,
+		CertFlag:                cert,
+		KeyFlag:                 key,
+		BeaconDB:                b.db,
+		Broadcaster:             p2pService,
+		PeersFetcher:            p2pService,
+		HeadFetcher:             chainService,
+		ForkFetcher:             chainService,
+		FinalizationFetcher:     chainService,
+		ParticipationFetcher:    chainService,
+		BlockReceiver:           chainService,
+		AttestationReceiver:     chainService,
+		GenesisTimeFetcher:      chainService,
+		GenesisFetcher:          chainService,
+		AttestationsPool:        b.attestationPool,
+		ExitPool:                b.exitPool,
+		SlashingsPool:           b.slashingsPool,
+		POWChainService:         web3Service,
+		ChainStartFetcher:       chainStartFetcher,
+		MockEth1Votes:           mockEth1DataVotes,
+		SyncService:             syncService,
+		DepositFetcher:          depositFetcher,
+		PendingDepositFetcher:   b.depositCache,
+		BlockNotifier:           b,
+		StateNotifier:           b,
+		OperationNotifier:       b,
+		SlasherCert:             slasherCert,
+		SlasherProvider:         slasherProvider,
+		StateGen:                b.stateGen,
+		EnableDebugRPCEndpoints: enableDebugRPCEndpoints,
 	})
 
 	return b.services.RegisterService(rpcService)
 }
 
-func (b *BeaconNode) registerPrometheusService(ctx *cli.Context) error {
+func (b *BeaconNode) registerPrometheusService() error {
 	var additionalHandlers []prometheus.Handler
 	var p *p2p.Service
 	if err := b.services.FetchService(&p); err != nil {
@@ -581,7 +597,7 @@ func (b *BeaconNode) registerPrometheusService(ctx *cli.Context) error {
 	additionalHandlers = append(additionalHandlers, prometheus.Handler{Path: "/tree", Handler: c.TreeHandler})
 
 	service := prometheus.NewPrometheusService(
-		fmt.Sprintf(":%d", ctx.Int64(cmd.MonitoringPortFlag.Name)),
+		fmt.Sprintf(":%d", b.cliCtx.Int64(flags.MonitoringPortFlag.Name)),
 		b.services,
 		additionalHandlers...,
 	)
@@ -590,24 +606,34 @@ func (b *BeaconNode) registerPrometheusService(ctx *cli.Context) error {
 	return b.services.RegisterService(service)
 }
 
-func (b *BeaconNode) registerGRPCGateway(ctx *cli.Context) error {
-	gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
+func (b *BeaconNode) registerGRPCGateway() error {
+	gatewayPort := b.cliCtx.Int(flags.GRPCGatewayPort.Name)
 	if gatewayPort > 0 {
-		selfAddress := fmt.Sprintf("127.0.0.1:%d", ctx.Int(flags.RPCPort.Name))
+		selfAddress := fmt.Sprintf("127.0.0.1:%d", b.cliCtx.Int(flags.RPCPort.Name))
 		gatewayAddress := fmt.Sprintf("0.0.0.0:%d", gatewayPort)
-		allowedOrigins := strings.Split(ctx.String(flags.GPRCGatewayCorsDomain.Name), ",")
-		return b.services.RegisterService(gateway.New(context.Background(), selfAddress, gatewayAddress, nil /*optional mux*/, allowedOrigins))
+		allowedOrigins := strings.Split(b.cliCtx.String(flags.GPRCGatewayCorsDomain.Name), ",")
+		enableDebugRPCEndpoints := b.cliCtx.Bool(flags.EnableDebugRPCEndpoints.Name)
+		return b.services.RegisterService(
+			gateway.New(
+				b.ctx,
+				selfAddress,
+				gatewayAddress,
+				nil, /*optional mux*/
+				allowedOrigins,
+				enableDebugRPCEndpoints,
+			),
+		)
 	}
 	return nil
 }
 
-func (b *BeaconNode) registerInteropServices(ctx *cli.Context) error {
-	genesisTime := ctx.Uint64(flags.InteropGenesisTimeFlag.Name)
-	genesisValidators := ctx.Uint64(flags.InteropNumValidatorsFlag.Name)
-	genesisStatePath := ctx.String(flags.InteropGenesisStateFlag.Name)
+func (b *BeaconNode) registerInteropServices() error {
+	genesisTime := b.cliCtx.Uint64(flags.InteropGenesisTimeFlag.Name)
+	genesisValidators := b.cliCtx.Uint64(flags.InteropNumValidatorsFlag.Name)
+	genesisStatePath := b.cliCtx.String(flags.InteropGenesisStateFlag.Name)
 
 	if genesisValidators > 0 || genesisStatePath != "" {
-		svc := interopcoldstart.NewColdStartService(context.Background(), &interopcoldstart.Config{
+		svc := interopcoldstart.NewColdStartService(b.ctx, &interopcoldstart.Config{
 			GenesisTime:   genesisTime,
 			NumValidators: genesisValidators,
 			BeaconDB:      b.db,
@@ -620,7 +646,7 @@ func (b *BeaconNode) registerInteropServices(ctx *cli.Context) error {
 	return nil
 }
 
-func (b *BeaconNode) registerArchiverService(ctx *cli.Context) error {
+func (b *BeaconNode) registerArchiverService() error {
 	if !flags.Get().EnableArchive {
 		return nil
 	}
@@ -628,7 +654,7 @@ func (b *BeaconNode) registerArchiverService(ctx *cli.Context) error {
 	if err := b.services.FetchService(&chainService); err != nil {
 		return err
 	}
-	svc := archiver.NewArchiverService(context.Background(), &archiver.Config{
+	svc := archiver.NewArchiverService(b.ctx, &archiver.Config{
 		BeaconDB:             b.db,
 		HeadFetcher:          chainService,
 		ParticipationFetcher: chainService,

beacon-chain/operations/attestations/kv/kv.go

@@ -1,3 +1,6 @@
+// Package kv includes a key-value store implementation
+// of an attestation cache used to satisfy important use-cases
+// such as aggregation in a beacon node runtime.
 package kv
 
 import (

beacon-chain/operations/attestations/service.go

@@ -1,3 +1,6 @@
+// Package attestations defines an attestation pool
+// service implementation which is used to manage the lifecycle
+// of aggregated, unaggregated, and fork-choice attestations.
 package attestations
 
 import (

beacon-chain/operations/slashings/doc.go

@@ -1,2 +1,5 @@
-// Package slashings defines the operations management of slashings.
+// Package slashings defines an in-memory pool of received
+// slashing events by the beacon node, handling their lifecycle
+// and performing integrity checks before serving them as objects
+// for validators to include in blocks.
 package slashings

beacon-chain/operations/voluntaryexits/doc.go

@@ -1,2 +1,5 @@
-// Package voluntaryexits defines the operations management of voluntary exits.
+// Package voluntaryexits defines an in-memory pool of received
+// voluntary exit events by the beacon node, handling their lifecycle
+// and performing integrity checks before serving them as objects
+// for validators to include in blocks.
 package voluntaryexits

beacon-chain/p2p/BUILD.bazel

@@ -43,6 +43,7 @@ go_library(
         "//shared/featureconfig:go_default_library",
         "//shared/hashutil:go_default_library",
         "//shared/iputils:go_default_library",
+        "//shared/p2putils:go_default_library",
         "//shared/params:go_default_library",
         "//shared/runutil:go_default_library",
         "//shared/sliceutil:go_default_library",
@@ -114,6 +115,7 @@ go_test(
         "//proto/beacon/p2p/v1:go_default_library",
         "//proto/testing:go_default_library",
         "//shared/iputils:go_default_library",
+        "//shared/p2putils:go_default_library",
         "//shared/params:go_default_library",
         "//shared/testutil:go_default_library",
         "@com_github_ethereum_go_ethereum//p2p/discover:go_default_library",

beacon-chain/p2p/broadcaster.go

@@ -22,7 +22,7 @@ var ErrMessageNotMapped = errors.New("message type is not mapped to a PubSub top
 func (s *Service) Broadcast(ctx context.Context, msg proto.Message) error {
 	ctx, span := trace.StartSpan(ctx, "p2p.Broadcast")
 	defer span.End()
-	forkDigest, err := s.ForkDigest()
+	forkDigest, err := s.forkDigest()
 	if err != nil {
 		return err
 	}
@@ -44,7 +44,7 @@ func (s *Service) Broadcast(ctx context.Context, msg proto.Message) error {
 	span.AddAttributes(trace.StringAttribute("topic", topic))
 
 	buf := new(bytes.Buffer)
-	if _, err := s.Encoding().Encode(buf, msg); err != nil {
+	if _, err := s.Encoding().EncodeGossip(buf, msg); err != nil {
 		err := errors.Wrap(err, "could not encode message")
 		traceutil.AnnotateError(span, err)
 		return err

beacon-chain/p2p/broadcaster_test.go

@@ -10,8 +10,8 @@ import (
 
 	"github.com/gogo/protobuf/proto"
 	eth "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
-	testpb "github.com/prysmaticlabs/prysm/proto/testing"
 	p2ptest "github.com/prysmaticlabs/prysm/beacon-chain/p2p/testing"
+	testpb "github.com/prysmaticlabs/prysm/proto/testing"
 	"github.com/prysmaticlabs/prysm/shared/testutil"
 )
 
@@ -29,6 +29,8 @@ func TestService_Broadcast(t *testing.T) {
 		cfg: &Config{
 			Encoding: "ssz",
 		},
+		genesisTime:           time.Now(),
+		genesisValidatorsRoot: []byte{'A'},
 	}
 
 	msg := &testpb.TestSimpleMessage{
@@ -38,7 +40,7 @@ func TestService_Broadcast(t *testing.T) {
 	topic := "/eth2/%x/testing"
 	// Set a test gossip mapping for testpb.TestSimpleMessage.
 	GossipTypeMapping[reflect.TypeOf(msg)] = topic
-	digest, err := p.ForkDigest()
+	digest, err := p.forkDigest()
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -67,7 +69,7 @@ func TestService_Broadcast(t *testing.T) {
 		}
 
 		result := &testpb.TestSimpleMessage{}
-		if err := p.Encoding().Decode(incomingMessage.Data, result); err != nil {
+		if err := p.Encoding().DecodeGossip(incomingMessage.Data, result); err != nil {
 			tt.Fatal(err)
 		}
 		if !proto.Equal(result, msg) {
@@ -85,7 +87,10 @@ func TestService_Broadcast(t *testing.T) {
 }
 
 func TestService_Broadcast_ReturnsErr_TopicNotMapped(t *testing.T) {
-	p := Service{}
+	p := Service{
+		genesisTime:           time.Now(),
+		genesisValidatorsRoot: []byte{'A'},
+	}
 	if err := p.Broadcast(context.Background(), &testpb.AddressBook{}); err != ErrMessageNotMapped {
 		t.Fatalf("Expected error %v, got %v", ErrMessageNotMapped, err)
 	}

beacon-chain/p2p/config.go

@@ -27,4 +27,5 @@ type Config struct {
 	WhitelistCIDR         string
 	Encoding              string
 	StateNotifier         statefeed.Notifier
+	PubSub                string
 }

beacon-chain/p2p/connmgr/connmgr.go

@@ -20,7 +20,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
 */
 
-// Package connmgr : This file is forked from github.com/libp2p/go-libp2p-core/connmgr/connmgr.go
+// Package connmgr is forked from github.com/libp2p/go-libp2p-core/connmgr/connmgr.go
 package connmgr
 
 import (

beacon-chain/p2p/discovery_test.go

@@ -53,7 +53,9 @@ func TestCreateListener(t *testing.T) {
 	port := 1024
 	ipAddr, pkey := createAddrAndPrivKey(t)
 	s := &Service{
-		cfg: &Config{UDPPort: uint(port)},
+		genesisTime:           time.Now(),
+		genesisValidatorsRoot: []byte{'A'},
+		cfg:                   &Config{UDPPort: uint(port)},
 	}
 	listener := s.createListener(ipAddr, pkey)
 	defer listener.Close()
@@ -130,7 +132,10 @@ func TestStartDiscV5_DiscoverAllPeers(t *testing.T) {
 func TestMultiAddrsConversion_InvalidIPAddr(t *testing.T) {
 	addr := net.ParseIP("invalidIP")
 	_, pkey := createAddrAndPrivKey(t)
-	s := &Service{}
+	s := &Service{
+		genesisTime:           time.Now(),
+		genesisValidatorsRoot: []byte{'A'},
+	}
 	node, err := s.createLocalNode(pkey, addr, 0, 0)
 	if err != nil {
 		t.Fatal(err)
@@ -149,6 +154,8 @@ func TestMultiAddrConversion_OK(t *testing.T) {
 			TCPPort: 0,
 			UDPPort: 0,
 		},
+		genesisTime:           time.Now(),
+		genesisValidatorsRoot: []byte{'A'},
 	}
 	listener := s.createListener(ipAddr, pkey)
 	defer listener.Close()

beacon-chain/p2p/encoder/BUILD.bazel

@@ -13,10 +13,11 @@ go_library(
         "//beacon-chain:__subpackages__",
     ],
     deps = [
+        "//shared/params:go_default_library",
         "@com_github_gogo_protobuf//proto:go_default_library",
         "@com_github_golang_snappy//:go_default_library",
+        "@com_github_pkg_errors//:go_default_library",
         "@com_github_prysmaticlabs_go_ssz//:go_default_library",
-        "@com_github_sirupsen_logrus//:go_default_library",
     ],
 )
 

beacon-chain/p2p/encoder/network_encoding.go

@@ -12,8 +12,10 @@ const (
 
 // NetworkEncoding represents an encoder compatible with Ethereum 2.0 p2p.
 type NetworkEncoding interface {
-	// Decodes to the provided message. The interface must be a pointer to the decoding destination.
+	// Decode to the provided message. The interface must be a pointer to the decoding destination.
 	Decode([]byte, interface{}) error
+	// DecodeGossip to the provided gossip message. The interface must be a pointer to the decoding destination.
+	DecodeGossip([]byte, interface{}) error
 	// DecodeWithLength a bytes from a reader with a varint length prefix. The interface must be a pointer to the
 	// decoding destination.
 	DecodeWithLength(io.Reader, interface{}) error
@@ -22,6 +24,8 @@ type NetworkEncoding interface {
 	DecodeWithMaxLength(io.Reader, interface{}, uint64) error
 	// Encode an arbitrary message to the provided writer. The interface must be a pointer object to encode.
 	Encode(io.Writer, interface{}) (int, error)
+	// EncodeGossip an arbitrary gossip message to the provided writer. The interface must be a pointer object to encode.
+	EncodeGossip(io.Writer, interface{}) (int, error)
 	// EncodeWithLength an arbitrary message to the provided writer with a varint length prefix. The interface must be
 	// a pointer object to encode.
 	EncodeWithLength(io.Writer, interface{}) (int, error)

beacon-chain/p2p/encoder/ssz.go

@@ -7,14 +7,18 @@ import (
 
 	"github.com/gogo/protobuf/proto"
 	"github.com/golang/snappy"
+	errors "github.com/pkg/errors"
 	"github.com/prysmaticlabs/go-ssz"
-	"github.com/sirupsen/logrus"
+	"github.com/prysmaticlabs/prysm/shared/params"
 )
 
 var _ = NetworkEncoding(&SszNetworkEncoder{})
 
 // MaxChunkSize allowed for decoding messages.
-const MaxChunkSize = uint64(1 << 20) // 1Mb
+var MaxChunkSize = params.BeaconNetworkConfig().MaxChunkSize // 1Mib
+
+// MaxGossipSize allowed for gossip messages.
+var MaxGossipSize = params.BeaconNetworkConfig().GossipMaxSize // 1 Mib
 
 // SszNetworkEncoder supports p2p networking encoding using SimpleSerialize
 // with snappy compression (if enabled).
@@ -41,6 +45,24 @@ func (e SszNetworkEncoder) Encode(w io.Writer, msg interface{}) (int, error) {
 	return w.Write(b)
 }
 
+// EncodeGossip the proto gossip message to the io.Writer.
+func (e SszNetworkEncoder) EncodeGossip(w io.Writer, msg interface{}) (int, error) {
+	if msg == nil {
+		return 0, nil
+	}
+	b, err := e.doEncode(msg)
+	if err != nil {
+		return 0, err
+	}
+	if len(b) > int(MaxGossipSize) {
+		return 0, errors.Errorf("gossip message exceeds max gossip size: %d bytes > %d bytes", len(b), MaxGossipSize)
+	}
+	if e.UseSnappyCompression {
+		b = snappy.Encode(nil /*dst*/, b)
+	}
+	return w.Write(b)
+}
+
 // EncodeWithLength the proto message to the io.Writer. This encoding prefixes the byte slice with a protobuf varint
 // to indicate the size of the message.
 func (e SszNetworkEncoder) EncodeWithLength(w io.Writer, msg interface{}) (int, error) {
@@ -105,6 +127,21 @@ func (e SszNetworkEncoder) Decode(b []byte, to interface{}) error {
 	return e.doDecode(b, to)
 }
 
+// DecodeGossip decodes the bytes to the protobuf gossip message provided.
+func (e SszNetworkEncoder) DecodeGossip(b []byte, to interface{}) error {
+	if e.UseSnappyCompression {
+		var err error
+		b, err = snappy.Decode(nil /*dst*/, b)
+		if err != nil {
+			return err
+		}
+	}
+	if len(b) > int(MaxGossipSize) {
+		return errors.Errorf("gossip message exceeds max gossip size: %d bytes > %d bytes", len(b), MaxGossipSize)
+	}
+	return e.doDecode(b, to)
+}
+
 // DecodeWithLength the bytes from io.Reader to the protobuf message provided.
 func (e SszNetworkEncoder) DecodeWithLength(r io.Reader, to interface{}) error {
 	return e.DecodeWithMaxLength(r, to, MaxChunkSize)
@@ -154,10 +191,9 @@ func (e SszNetworkEncoder) MaxLength(length int) int {
 // Writes a bytes value through a snappy buffered writer.
 func writeSnappyBuffer(w io.Writer, b []byte) (int, error) {
 	bufWriter := snappy.NewBufferedWriter(w)
-	defer func() {
-		if err := bufWriter.Close(); err != nil {
-			logrus.WithError(err).Error("Failed to close snappy buffered writer")
-		}
-	}()
-	return bufWriter.Write(b)
+	num, err := bufWriter.Write(b)
+	if err != nil {
+		return 0, err
+	}
+	return num, bufWriter.Close()
 }

beacon-chain/p2p/encoder/ssz_test.go

@@ -15,12 +15,14 @@ func TestSszNetworkEncoder_RoundTrip(t *testing.T) {
 	e := &encoder.SszNetworkEncoder{UseSnappyCompression: false}
 	testRoundTrip(t, e)
 	testRoundTripWithLength(t, e)
+	testRoundTripWithGossip(t, e)
 }
 
 func TestSszNetworkEncoder_RoundTrip_Snappy(t *testing.T) {
 	e := &encoder.SszNetworkEncoder{UseSnappyCompression: true}
 	testRoundTrip(t, e)
 	testRoundTripWithLength(t, e)
+	testRoundTripWithGossip(t, e)
 }
 
 func testRoundTrip(t *testing.T, e *encoder.SszNetworkEncoder) {
@@ -63,6 +65,26 @@ func testRoundTripWithLength(t *testing.T, e *encoder.SszNetworkEncoder) {
 	}
 }
 
+func testRoundTripWithGossip(t *testing.T, e *encoder.SszNetworkEncoder) {
+	buf := new(bytes.Buffer)
+	msg := &testpb.TestSimpleMessage{
+		Foo: []byte("fooooo"),
+		Bar: 9001,
+	}
+	_, err := e.EncodeGossip(buf, msg)
+	if err != nil {
+		t.Fatal(err)
+	}
+	decoded := &testpb.TestSimpleMessage{}
+	if err := e.DecodeGossip(buf.Bytes(), decoded); err != nil {
+		t.Fatal(err)
+	}
+	if !proto.Equal(decoded, msg) {
+		t.Logf("decoded=%+v\n", decoded)
+		t.Error("Decoded message is not the same as original")
+	}
+}
+
 func TestSszNetworkEncoder_EncodeWithMaxLength(t *testing.T) {
 	buf := new(bytes.Buffer)
 	msg := &testpb.TestSimpleMessage{

beacon-chain/p2p/fork.go

@@ -10,19 +10,19 @@ import (
 	"github.com/ethereum/go-ethereum/p2p/enr"
 	"github.com/pkg/errors"
 	"github.com/prysmaticlabs/go-ssz"
-	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
+	"github.com/prysmaticlabs/prysm/shared/p2putils"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/sirupsen/logrus"
 )
 
 // ENR key used for eth2-related fork data.
-const eth2ENRKey = "eth2"
+var eth2ENRKey = params.BeaconNetworkConfig().ETH2Key
 
 // ForkDigest returns the current fork digest of
 // the node.
-func (s *Service) ForkDigest() ([4]byte, error) {
-	return createForkDigest(s.genesisTime, s.genesisValidatorsRoot)
+func (s *Service) forkDigest() ([4]byte, error) {
+	return p2putils.CreateForkDigest(s.genesisTime, s.genesisValidatorsRoot)
 }
 
 // Compares fork ENRs between an incoming peer's record and our node's
@@ -72,35 +72,6 @@ func (s *Service) compareForkENR(record *enr.Record) error {
 	return nil
 }
 
-// Creates a fork digest from a genesis time and genesis
-// validators root, utilizing the current slot to determine
-// the active fork version in the node.
-func createForkDigest(
-	genesisTime time.Time,
-	genesisValidatorsRoot []byte,
-) ([4]byte, error) {
-	currentSlot := helpers.SlotsSince(genesisTime)
-	currentEpoch := helpers.SlotToEpoch(currentSlot)
-
-	// We retrieve a list of scheduled forks by epoch.
-	// We loop through the keys in this map to determine the current
-	// fork version based on the current, time-based epoch number
-	// since the genesis time.
-	currentForkVersion := params.BeaconConfig().GenesisForkVersion
-	scheduledForks := params.BeaconConfig().ForkVersionSchedule
-	for epoch, forkVersion := range scheduledForks {
-		if epoch <= currentEpoch {
-			currentForkVersion = forkVersion
-		}
-	}
-
-	digest, err := helpers.ComputeForkDigest(currentForkVersion, genesisValidatorsRoot)
-	if err != nil {
-		return [4]byte{}, err
-	}
-	return digest, nil
-}
-
 // Adds a fork entry as an ENR record under the eth2EnrKey for
 // the local node. The fork entry is an ssz-encoded enrForkID type
 // which takes into account the current fork version from the current
@@ -111,7 +82,7 @@ func addForkEntry(
 	genesisTime time.Time,
 	genesisValidatorsRoot []byte,
 ) (*enode.LocalNode, error) {
-	digest, err := createForkDigest(genesisTime, genesisValidatorsRoot)
+	digest, err := p2putils.CreateForkDigest(genesisTime, genesisValidatorsRoot)
 	if err != nil {
 		return nil, err
 	}

beacon-chain/p2p/fork_test.go

@@ -16,6 +16,7 @@ import (
 	"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
 	testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
+	"github.com/prysmaticlabs/prysm/shared/p2putils"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/testutil"
 	"github.com/sirupsen/logrus"
@@ -213,7 +214,7 @@ func TestDiscv5_AddRetrieveForkEntryENR(t *testing.T) {
 
 	genesisTime := time.Now()
 	genesisValidatorsRoot := make([]byte, 32)
-	digest, err := createForkDigest(genesisTime, make([]byte, 32))
+	digest, err := p2putils.CreateForkDigest(genesisTime, make([]byte, 32))
 	if err != nil {
 		t.Fatal(err)
 	}

beacon-chain/p2p/handshake.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"time"
 
 	"github.com/libp2p/go-libp2p-core/network"
 	"github.com/libp2p/go-libp2p-core/peer"
@@ -15,7 +14,8 @@ import (
 // AddConnectionHandler adds a callback function which handles the connection with a
 // newly added peer. It performs a handshake with that peer by sending a hello request
 // and validating the response from the peer.
-func (s *Service) AddConnectionHandler(reqFunc func(ctx context.Context, id peer.ID) error) {
+func (s *Service) AddConnectionHandler(reqFunc func(ctx context.Context, id peer.ID) error,
+	goodbyeFunc func(ctx context.Context, id peer.ID) error) {
 	s.host.Network().Notify(&network.NotifyBundle{
 		ConnectedF: func(net network.Network, conn network.Conn) {
 			log := log.WithField("peer", conn.RemotePeer().Pretty())
@@ -28,10 +28,15 @@ func (s *Service) AddConnectionHandler(reqFunc func(ctx context.Context, id peer
 			}
 			s.peers.Add(nil /* ENR */, conn.RemotePeer(), conn.RemoteMultiaddr(), conn.Stat().Direction)
 			if len(s.peers.Active()) >= int(s.cfg.MaxPeers) {
-				log.WithField("reason", "at peer limit").Trace("Ignoring connection request")
-				if err := s.Disconnect(conn.RemotePeer()); err != nil {
-					log.WithError(err).Error("Unable to disconnect from peer")
-				}
+				go func() {
+					log.WithField("reason", "at peer limit").Trace("Ignoring connection request")
+					if err := goodbyeFunc(context.Background(), conn.RemotePeer()); err != nil {
+						log.WithError(err).Trace("Unable to send goodbye message to peer")
+					}
+					if err := s.Disconnect(conn.RemotePeer()); err != nil {
+						log.WithError(err).Error("Unable to disconnect from peer")
+					}
+				}()
 				return
 			}
 			if s.peers.IsBad(conn.RemotePeer()) {
@@ -52,10 +57,8 @@ func (s *Service) AddConnectionHandler(reqFunc func(ctx context.Context, id peer
 					"activePeers": len(s.peers.Active()),
 				})
 				s.peers.SetConnectionState(conn.RemotePeer(), peers.PeerConnecting)
-				ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
-				defer cancel()
-				if err := reqFunc(ctx, conn.RemotePeer()); err != nil && err != io.EOF {
-					log.WithError(err).Debug("Handshake failed")
+				if err := reqFunc(context.Background(), conn.RemotePeer()); err != nil && err != io.EOF {
+					log.WithError(err).Trace("Handshake failed")
 					if err.Error() == "protocol not supported" {
 						// This is only to ensure the smooth running of our testnets. This will not be
 						// used in production.

beacon-chain/p2p/interfaces.go

@@ -37,14 +37,13 @@ type SetStreamHandler interface {
 
 // ConnectionHandler configures p2p to handle connections with a peer.
 type ConnectionHandler interface {
-	AddConnectionHandler(f func(ctx context.Context, id peer.ID) error)
+	AddConnectionHandler(f func(ctx context.Context, id peer.ID) error, g func(context.Context, peer.ID) error)
 	AddDisconnectionHandler(f func(ctx context.Context, id peer.ID) error)
 }
 
 // EncodingProvider provides p2p network encoding.
 type EncodingProvider interface {
 	Encoding() encoder.NetworkEncoding
-	ForkDigest() ([4]byte, error)
 }
 
 // PubSubProvider provides the p2p pubsub protocol.