Beacon state: copy on write for certain large fields (#4699)

* begin state service * begin on the state trie idea * created beacon state structure * add in the full clone getter * return by value instead * add all setters * new state setters are being completed * arrays roots exposed * close to finishing all these headerssss * functionality complete * added in proto benchmark test * test for compatibility * add test for compat * comments fixed * add clone * add clone * remove underlying copies * make it immutable * integrate it into chainservice * revert * wrap up comments for package * address all comments and godocs * address all comments * clone the pending attestation properly * properly clone remaining items * tests pass fixed bug * begin using it instead of head state * prevent nil pointer exceptions * begin using new struct in db * integrated new type into db package * add proper nil checks * using new state in archiver * refactored much of core * editing all the precompute functions * done with most core refactor * fixed up some bugs in the clone comparisons * append current epoch atts * add missing setters * add new setters * fix other core methods * fix up transition * main service and forkchoice * fix rpc * integrated to powchain * some more changes * fix build * improve processing of deposits * fix error * prevent panic * comment * fix process att * gaz * fix up att process * resolve existing review comments * resolve another batch of gh comments * resolve broken cpt state * revise testutil to use the new state * begin updating the state transition func to pass in more compartmentalized args * finish editing transition function to return errors * block operations pretty much done with refactor * state transition fully refactored * got epoch processing completed * fix build in fork choice * fixing more of the build * fix up broken sync package * it builds nowww it buildssss * revert registry changes * Recompute on Read (#4627) * compute on read * fix up eth1 data votes * looking into slashings bug introduced in core/ * able to advance more slots * add logging * can now sync with testnet yay * remove the leaves algorithm and other merkle imports * expose initialize unsafe funcs * Update beacon-chain/db/kv/state.go * lint Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> * More Optimizations for New State (#4641) * map optimization * more optimizations * use a custom hasher * comment * block operations optimizations * Update beacon-chain/state/types.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * fixed up various operations to use the validator index map access Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> * archiver tests pass * fixing cache tests * cache tests passing * edited validator tests * powchain tests passing * halfway thru sync tests * more sync test fixes * add in tests for state/ * working through rpc tests * assignments tests passed * almost done with rpc/beacon tests * resolved painful validator test * fixed up even more tests * resolve tests * fix build * reduce a randao mixes copy * fixes under //beacon-chain/blockchain/... * build //beacon-chain/core/... * fixes * Runtime Optimizations (#4648) * parallelize shuffling * clean up * lint * fix build * use callback to read from registry * fix array roots and size map * new improvements * reduce hash allocs * improved shuffling * terence's review * use different method * raul's comment * new array roots * remove clone in pre-compute * Update beacon-chain/state/types.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * raul's review * lint * fix build issues * fix visibility Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> * fix visibility * build works for all * fix blockchain test * fix a few tests * fix more tests * update validator in slashing * archiver passing * fixed rpc/validator * progress on core tests * resolve broken rpc tests * blockchain tests passed * fix up some tests in core * fix message diff * remove unnecessary save * Save validator after slashing * Update validators one by one * another update * fix everything * fix more precompute tests * fix blocks tests * more elegant fix * more helper fixes * change back ? * fix test * fix skip slot * fix test * reset caches * fix testutil * raceoff fixed * passing * Retrieve cached state in the beginning * lint * Fixed tests part 1 * Fixed rest of the tests * Minor changes to avoid copying, small refactor to reduce deplicated code * Handle att req for slot 0 * New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689) * Only populate merkle layers as needed, copy merkle layers on copy/clone. * use custom copy * Make maps of correct size * slightly fast, doesn't wait for lock Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Target root can't be 0x00 * Don't use cache for current slot (may not be the right fix) * fixed up tests * Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing... * Align with prev logic for process slots cachedState.Slot() < slot * Fix Initial Sync Flag (#4692) * fixes * fix up some test failures due to lack of nil checks * fix up some test failures due to lack of nil checks * fix up imports * revert some changes * imports Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> * resolving further conflicts * Better skip slot cache (#4694) * Return copy of skip slot cache state, disable skip slot cache on sync * fix * Fix pruning * copy on write method * gaz * fix tests * fix up issues with broken tests * remove extra update * remove debugging lines * gofmt Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Nishant Das <nish1993@hotmail.com> Co-authored-by: shayzluf <thezluf@gmail.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2026-01-09 13:28:01 -05:00 · 2020-01-31 23:23:34 -08:00
parent d32493d43b
commit c2fbb40909
6 changed files with 112 additions and 9 deletions
--- a/beacon-chain/core/epoch/epoch_processing.go
+++ b/beacon-chain/core/epoch/epoch_processing.go
@@ -84,9 +84,9 @@ func ProcessRegistryUpdates(state *stateTrie.BeaconState) (*stateTrie.BeaconStat
 		// Process the validators for activation eligibility.
 		if helpers.IsEligibleForActivationQueue(validator) {
 			validator.ActivationEligibilityEpoch = helpers.CurrentEpoch(state) + 1
-		}
-		if err := state.UpdateValidatorAtIndex(uint64(idx), validator); err != nil {
-			return nil, err
+			if err := state.UpdateValidatorAtIndex(uint64(idx), validator); err != nil {
+				return nil, err
+			}
 		}

 		// Process the validators for ejection.
--- a/beacon-chain/state/BUILD.bazel
+++ b/beacon-chain/state/BUILD.bazel
@@ -35,6 +35,7 @@ go_test(
    embed = [":go_default_library"],
    deps = [
        "//proto/beacon/p2p/v1:go_default_library",
+        "//shared/bytesutil:go_default_library",
        "//shared/interop:go_default_library",
        "//shared/params:go_default_library",
        "//shared/stateutil:go_default_library",
--- a/beacon-chain/state/getters.go
+++ b/beacon-chain/state/getters.go
@@ -244,6 +244,9 @@ func (b *BeaconState) Validators() []*ethpb.Validator {
 	res := make([]*ethpb.Validator, len(b.state.Validators))
 	for i := 0; i < len(res); i++ {
 		val := b.state.Validators[i]
+		if val == nil {
+			continue
+		}
 		pubKey := make([]byte, len(val.PublicKey))
 		copy(pubKey, val.PublicKey)
 		withdrawalCreds := make([]byte, len(val.WithdrawalCredentials))
--- a/beacon-chain/state/setters.go
+++ b/beacon-chain/state/setters.go
@@ -93,7 +93,12 @@ func (b *BeaconState) UpdateBlockRootAtIndex(idx uint64, blockRoot [32]byte) err
 	if len(b.state.BlockRoots) <= int(idx) {
 		return fmt.Errorf("invalid index provided %d", idx)
 	}
-	b.state.BlockRoots[idx] = blockRoot[:]
+
+	// Copy on write since this is a shared array.
+	r := b.BlockRoots()
+	r[idx] = blockRoot[:]
+	b.state.BlockRoots = r
+
 	b.lock.Lock()
 	b.markFieldAsDirty(blockRoots)
 	b.lock.Unlock()
@@ -116,7 +121,12 @@ func (b *BeaconState) UpdateStateRootAtIndex(idx uint64, stateRoot [32]byte) err
 	if len(b.state.StateRoots) <= int(idx) {
 		return errors.Errorf("invalid index provided %d", idx)
 	}
-	b.state.StateRoots[idx] = stateRoot[:]
+
+	// Copy on write since this is a shared array.
+	r := b.StateRoots()
+	r[idx] = stateRoot[:]
+	b.state.StateRoots = r
+
 	b.lock.Lock()
 	b.markFieldAsDirty(stateRoots)
 	b.lock.Unlock()
@@ -184,12 +194,15 @@ func (b *BeaconState) SetValidators(val []*ethpb.Validator) error {
 // ApplyToEveryValidator applies the provided callback function to each validator in the
 // validator registry.
 func (b *BeaconState) ApplyToEveryValidator(f func(idx int, val *ethpb.Validator) error) error {
-	for i, val := range b.state.Validators {
+	// Copy on write since this is a shared array.
+	v := b.Validators()
+	for i, val := range v {
 		err := f(i, val)
 		if err != nil {
 			return err
 		}
 	}
+	b.state.Validators = v
 	b.lock.Lock()
 	b.markFieldAsDirty(validators)
 	b.lock.Unlock()
@@ -202,7 +215,10 @@ func (b *BeaconState) UpdateValidatorAtIndex(idx uint64, val *ethpb.Validator) e
 	if len(b.state.Validators) <= int(idx) {
 		return errors.Errorf("invalid index provided %d", idx)
 	}
-	b.state.Validators[idx] = val
+	// Copy on write since this is a shared array.
+	v := b.Validators()
+	v[idx] = val
+	b.state.Validators = v
 	b.lock.Lock()
 	b.markFieldAsDirty(validators)
 	b.lock.Unlock()
@@ -257,7 +273,12 @@ func (b *BeaconState) UpdateRandaoMixesAtIndex(val []byte, idx uint64) error {
 	if len(b.state.RandaoMixes) <= int(idx) {
 		return errors.Errorf("invalid index provided %d", idx)
 	}
-	b.state.RandaoMixes[idx] = val
+
+	// Copy on write since this is a shared array.
+	mixes := b.RandaoMixes()
+	mixes[idx] = val
+	b.state.RandaoMixes = mixes
+
 	b.lock.Lock()
 	b.markFieldAsDirty(randaoMixes)
 	b.lock.Unlock()
--- a/beacon-chain/state/types.go
+++ b/beacon-chain/state/types.go
@@ -52,7 +52,37 @@ func (b *BeaconState) Copy() *BeaconState {
 	b.lock.RLock()
 	defer b.lock.RUnlock()
 	dst := &BeaconState{
-		state:       b.CloneInnerState(),
+		state: &pbp2p.BeaconState{
+			// Primitive types, safe to copy.
+			GenesisTime:      b.state.GenesisTime,
+			Slot:             b.state.Slot,
+			Eth1DepositIndex: b.state.Eth1DepositIndex,
+
+			// Large arrays, infrequently changed, constant size.
+			RandaoMixes: b.state.RandaoMixes,
+			StateRoots:  b.state.StateRoots,
+			BlockRoots:  b.state.BlockRoots,
+
+			// Large arrays, increases over time.
+			Validators: b.state.Validators,
+
+			// Potential candidates for copy-on-write.
+			Balances:                  b.Balances(),
+			HistoricalRoots:           b.HistoricalRoots(),
+			PreviousEpochAttestations: b.PreviousEpochAttestations(),
+			CurrentEpochAttestations:  b.CurrentEpochAttestations(),
+			Slashings:                 b.Slashings(),
+			Eth1DataVotes:             b.Eth1DataVotes(),
+
+			// Everything else, too small to be concerned about, constant size.
+			Fork:                        b.Fork(),
+			LatestBlockHeader:           b.LatestBlockHeader(),
+			Eth1Data:                    b.Eth1Data(),
+			JustificationBits:           b.JustificationBits(),
+			PreviousJustifiedCheckpoint: b.PreviousJustifiedCheckpoint(),
+			CurrentJustifiedCheckpoint:  b.CurrentJustifiedCheckpoint(),
+			FinalizedCheckpoint:         b.FinalizedCheckpoint(),
+		},
 		dirtyFields: make(map[fieldIndex]interface{}, 20),
 		valIdxMap:   make(map[[48]byte]uint64, len(b.valIdxMap)),
 	}
--- a/beacon-chain/state/types_test.go
+++ b/beacon-chain/state/types_test.go
@@ -1,6 +1,7 @@
 package state_test

 import (
+	"reflect"
 	"strconv"
 	"testing"

@@ -8,6 +9,7 @@ import (
 	ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
 	stateTrie "github.com/prysmaticlabs/prysm/beacon-chain/state"
 	pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
+	"github.com/prysmaticlabs/prysm/shared/bytesutil"
 	"github.com/prysmaticlabs/prysm/shared/interop"
 	"github.com/prysmaticlabs/prysm/shared/params"
 	"github.com/prysmaticlabs/prysm/shared/stateutil"
@@ -178,3 +180,49 @@ func cloneValidatorsManually(vals []*ethpb.Validator) []*ethpb.Validator {
 	}
 	return res
 }
+
+func TestBeaconState_ImmutabilityWithSharedResources(t *testing.T) {
+	params.UseMinimalConfig()
+	genesis := setupGenesisState(t, 64)
+	a, err := stateTrie.InitializeFromProto(genesis)
+	if err != nil {
+		t.Fatal(err)
+	}
+	b := a.Copy()
+
+	// Randao mixes
+	if !reflect.DeepEqual(a.RandaoMixes(), b.RandaoMixes()) {
+		t.Fatal("Test precondition failed, fields are not equal")
+	}
+	a.UpdateRandaoMixesAtIndex([]byte("foo"), 1)
+	if reflect.DeepEqual(a.RandaoMixes(), b.RandaoMixes()) {
+		t.Error("Expect a.RandaoMixes() to be different from b.RandaoMixes()")
+	}
+
+	// Validators
+	if !reflect.DeepEqual(a.Validators(), b.Validators()) {
+		t.Fatal("Test precondition failed, fields are not equal")
+	}
+	a.UpdateValidatorAtIndex(1, &ethpb.Validator{Slashed: true})
+	if reflect.DeepEqual(a.Validators(), b.Validators()) {
+		t.Error("Expect a.Validators() to be different from b.Validators()")
+	}
+
+	// State Roots
+	if !reflect.DeepEqual(a.StateRoots(), b.StateRoots()) {
+		t.Fatal("Test precondition failed, fields are not equal")
+	}
+	a.UpdateStateRootAtIndex(1, bytesutil.ToBytes32([]byte("foo")))
+	if reflect.DeepEqual(a.StateRoots(), b.StateRoots()) {
+		t.Fatal("Expected a.StateRoots() to be different from b.StateRoots()")
+	}
+
+	// Block Roots
+	if !reflect.DeepEqual(a.BlockRoots(), b.BlockRoots()) {
+		t.Fatal("Test precondition failed, fields are not equal")
+	}
+	a.UpdateBlockRootAtIndex(1, bytesutil.ToBytes32([]byte("foo")))
+	if reflect.DeepEqual(a.BlockRoots(), b.BlockRoots()) {
+		t.Fatal("Expected a.BlockRoots() to be different from b.BlockRoots()")
+	}
+}