Switch to lazy state balance cache (#9822)

* quick lazy balance cache proof of concept

* WIP refactoring to use lazy cache

* updating tests to use functional opts

* updating the rest of the tests, all passing

* use mock stategen where possible

reduces the number of test cases that require db setup

* rename test opt method for clear link

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* test assumption that zerohash is in db

* remove unused MockDB (mocking stategen instead)

* fix cache bug, switch to sync.Mutex

* improve test coverage for the state cache

* uncomment failing genesis test for discussion

* gofmt

* remove unused Service struct member

* cleanup unused func input

* combining type declaration in signature

* don't export the state cache constructor

* work around blockchain deps w/ new file

service_test brings in a ton of dependencies that make bazel rules
for blockchain complex, so just sticking these mocks in their own
file simplifies things.

* gofmt

* remove intentionally failing test

this test established that the zero root can't be used to look up the
state, resulting in a change in another PR to update stategen to use the
GenesisState db method instead when the zero root is detected.

* fixed error introduced by develop refresh

* fix import ordering

* appease deepsource

* remove unused function

* godoc comments on new requires/assert

* defensive constructor per terence's PR comment

* more differentiated balance cache metric names

Co-authored-by: kasey <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

testing/require/requires.go

@@ -71,3 +71,9 @@ func LogsDoNotContain(tb assertions.AssertionTestingTB, hook *test.Hook, want st
 func NotEmpty(tb assertions.AssertionTestingTB, obj interface{}, msg ...interface{}) {
 	assertions.NotEmpty(tb.Fatalf, obj, msg...)
 }
+
+// ErrorIs uses Errors.Is to recursively unwrap err looking for target in the chain.
+// If any error in the chain matches target, the assertion will pass.
+func ErrorIs(tb assertions.AssertionTestingTB, err, target error, msg ...interface{}) {
+	assertions.ErrorIs(tb.Fatalf, err, target, msg)
+}