* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Trigger Consolidation
* Finally have it working
* Fix Build
* Revert Change
* Fix Context
* Finally have consolidations working
* Get Evaluator Working
* Get Withdrawals Working
* Changelog
* Finally Passes
* New line
* Try again
* fmt
* fmt
* Fix Test
* Implement static analysis to prevent panics
* Add nopanic to nogo
* Fix violations and add exclusions
Fix violations and add exclusions for all
* Changelog fragment
* Use pass.Report instead of pass.Reportf
* Remove strings.ToLower for checking init method name
* Add exclusion for herumi init
* Move api/client/beacon template function to init and its own file
* Fix nopanic testcase
* wip electra e2e
* add Deneb state to `validatorsParticipating`
* Run bazel
* add Electra state to `validatorsParticipating`
* fixing some e2e issues
* more evaluator fixes and changelog
* adding in special condition to pass electra epoch participation
* fixing typo
* missed updating forks for e2e tests
* reverting change current release fork
* missed updating e2e config for test
* updating to latest devnet 5 to fix unit tests
* go mod tidy
* fixing branch, temporary will need to update geth version later
* update to goethereum v1.15.0
* changing changelog to reflect update in geth dependency
* fixing test failures
* adding fix for range request limit during transition period between forks
* enabling validator rest for Electra
* rolling back error message
* adding fixed change logs
* fixing dependencies based on nishant's comments, deps.bzl should be updated not workspace
* partially reverting incorrect change
* removing fixes from change log, handled in separate prs
* removing comment
* updating update fraction field to the corrected spec value from prague
* Update testing/endtoend/evaluators/fork.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
---------
Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* updating the goethereum dependency
* fixing dependencies
* reverting workspace
* more fixes, work in progress
* trying with upgraded geth version
* fixing deprecated functions except for the time related ones on eth1 distance due to time issues
* fixing time issues
* gaz
* fixing test and upgrading some dependencies and reverting others
* Disable cgo in hid, delete old vendored usb library
* changelog
* rolling back dependencies
* fixing go mod tidy
* Geth v1.13.6
* fix tests
* Add ping interval, set to 500ms for tests. This didnt work
* Update to v1.14.8
* Spread it out to different bootnodes
* Fix it
* Remove Memsize
* Update all out of date dependencies
* Fix geth body change
* Fix Test
* Fix Build
* Fix Tests
* Fix Tests Again
* Fix Tests Again
* Fix Tests
* Fix Test
* Copy USB Package for HID
* Push it up
* Finally fix all tests with felix's changes
* updating geth dependency
* Update go-ethereum to v1.14.11
* fixing import
* reverting blob change
* fixing Implicit memory aliasing in for loop.
* WIP changes
* wip getting a little further on e2e runs
* getting a little further
* getting a little further
* setting everything to capella
* more partial fixes
* more fixes but still WIP
* fixing access list transactions"
* some cleanup
* making configs dynamic
* reverting time
* skip lower bound in builder
* updating to geth v1.14.12
* fixing verify blob to pointer
* go mod tidy
* fixing linting
* missed removing another terminal difficulty item
* another missed update
* updating more dependencies to fix cicd
* fixing holiman dependency update
* downgrading geth to 1.14.11 due to p2p loop issue
* reverting builder middleware caused by downgrade
* fixing more rollback issues
* upgrading back to 1.14.12 after discussing with preston
* mod tidy
* gofmt
* partial review feedback
* trying to start e2e from bellatrix instead
* reverting some changes
---------
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Prepare for future fork boilerplate.
* Implement the Fulu fork boilerplate.
* `Upgraded state to <fork> log`: Move from debug to info.
Rationale:
This log is the only one notifying the user a new fork happened.
A new fork is always a little bit stressful for a node operator.
Having at least one log indicating the client switched fork is something useful.
* Update testing/util/helpers.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Fix Radek's comment.
* Remove Electra struct type aliasing.
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Use slot to determine version
* gaz
* solve cyclic dependency
* Radek' review
* unit test
* gaz
* use require instead of assert
* fix test
* fix test
* fix TestGetAggregateAttestation
* fix ListAttestations test
* James' review
* Radek' review
* add extra checks to GetAttesterSlashingsV2
* fix matchingAtts
* improve tests + fix
* fix
* stop appending all non electra atts
* more tests
* changelog
* revert TestProduceSyncCommitteeContribution changes
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
* Remove Feature Flag From Prater (#12082)
* Use Epoch boundary cache to retrieve balances (#12083)
* Use Epoch boundary cache to retrieve balances
* save boundary states before inserting to forkchoice
* move up last block save
* remove boundary checks on balances
* fix ordering
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* chore: fix unfound image
* fix link
---------
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* wip changes
* wip changes refactoring deposit functions
* wip on deposit functions
* wip changes to fix deposit processing
* more wip function logic
* gaz
* wip refactoring deposit changes
* removing circlular dependency and other small fix
* fixing circular dependencies
* fixing validators file
* fixing more tests
* fixing unit tests
* wip deposit packing
* refactoring packing
* changing packing code some more
* fixing packing change
* updating more tests
* removing comment
* fixing transition code for eip6110
* including inserts for validator index cache
* adding tests and placeholder test
* moving deposit related unit tests over
* adding in test for electra proposer packing
* spec test wip
* moving cache saving to the correct spot
* eip-6110: deposit spectests
* adding deposit receipt spec tests
* reverting altair operations in this pr and will be handled separately
* fixing renames but need to refactor process deposits
* gaz
* fixing names
* fixing linting
* fixing unit test
* fixing a test and updating test util
* bal never used
* fixing more tests
* fixing one more test
* addressing feedback
* refactoring process deposits to be part of their own fork instead of in blocks package
* adding new tests to cover functions and removing redundancies
* removing comment based on feedback
* resolving easy deepsource issue
* refactoring for appropriate aliasing and readability
* reverting some changes to simplify diff
* small edit to comment
* fixing linting
* fixing merge changes and review comments
* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* partial review comments
* addressing slack feedback
* moving function from deposits to validator.go
* adding in batch verification and improving logs
---------
Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* `EpochFromString`: Use already defined `Uint64FromString` function.
* `Test_uint64FromString` => `Test_FromString`
This test function tests more functions than `Uint64FromString`.
* Slashing protection history: Remove unreachable code.
The function `NewKVStore` creates, via `kv.UpdatePublicKeysBuckets`,
a new item in the `proposal-history-bucket-interchange`.
IMO there is no real reason to prefer `proposal` than `attestation`
as a prefix for this bucket, but this is the way it is done right now
and renaming the bucket will probably be backward incompatible.
An `attestedPublicKey` cannot exist without
the corresponding `proposedPublicKey`.
Thus, the `else` portion of code removed in this commit is not reachable.
We raise an error if we get there.
This is also probably the reason why the removed `else` portion was not
tested.
* `NewKVStore`: Switch items in `createBuckets`.
So the order corresponds to `schema.go`
* `slashableAttestationCheck`: Fix comments and logs.
* `ValidatorClient.db`: Use `iface.ValidatorDB`.
* BoltDB database: Implement `GraffitiFileHash`.
* Filesystem database: Creates `db.go`.
This file defines the following structs:
- `Store`
- `Graffiti`
- `Configuration`
- `ValidatorSlashingProtection`
This files implements the following public functions:
- `NewStore`
- `Close`
- `Backup`
- `DatabasePath`
- `ClearDB`
- `UpdatePublicKeysBuckets`
This files implements the following private functions:
- `slashingProtectionDirPath`
- `configurationFilePath`
- `configuration`
- `saveConfiguration`
- `validatorSlashingProtection`
- `saveValidatorSlashingProtection`
- `publicKeys`
* Filesystem database: Creates `genesis.go`.
This file defines the following public functions:
- `GenesisValidatorsRoot`
- `SaveGenesisValidatorsRoot`
* Filesystem database: Creates `graffiti.go`.
This file defines the following public functions:
- `SaveGraffitiOrderedIndex`
- `GraffitiOrderedIndex`
* Filesystem database: Creates `migration.go`.
This file defines the following public functions:
- `RunUpMigrations`
- `RunDownMigrations`
* Filesystem database: Creates proposer_settings.go.
This file defines the following public functions:
- `ProposerSettings`
- `ProposerSettingsExists`
- `SaveProposerSettings`
* Filesystem database: Creates `attester_protection.go`.
This file defines the following public functions:
- `EIPImportBlacklistedPublicKeys`
- `SaveEIPImportBlacklistedPublicKeys`
- `SigningRootAtTargetEpoch`
- `LowestSignedTargetEpoch`
- `LowestSignedSourceEpoch`
- `AttestedPublicKeys`
- `CheckSlashableAttestation`
- `SaveAttestationForPubKey`
- `SaveAttestationsForPubKey`
- `AttestationHistoryForPubKey`
* Filesystem database: Creates `proposer_protection.go`.
This file defines the following public functions:
- `HighestSignedProposal`
- `LowestSignedProposal`
- `ProposalHistoryForPubKey`
- `ProposalHistoryForSlot`
- `ProposedPublicKeys`
* Ensure that the filesystem store implements the `ValidatorDB` interface.
* `slashableAttestationCheck`: Check the database type.
* `slashableProposalCheck`: Check the database type.
* `slashableAttestationCheck`: Allow usage of minimal slashing protection.
* `slashableProposalCheck`: Allow usage of minimal slashing protection.
* `ImportStandardProtectionJSON`: Check the database type.
* `ImportStandardProtectionJSON`: Allow usage of min slashing protection.
* Implement `RecursiveDirFind`.
* Implement minimal<->complete DB conversion.
3 public functions are implemented:
- `IsCompleteDatabaseExisting`
- `IsMinimalDatabaseExisting`
- `ConvertDatabase`
* `setupDB`: Add `isSlashingProtectionMinimal` argument.
The feature addition is located in `validator/node/node_test.go`.
The rest of this commit consists in minimal slashing protection testing.
* `setupWithKey`: Add `isSlashingProtectionMinimal` argument.
The feature addition is located in `validator/client/propose_test.go`.
The rest of this commit consists in tests wrapping.
* `setup`: Add `isSlashingProtectionMinimal` argument.
The added feature is located in the `validator/client/propose_test.go`
file.
The rest of this commit consists in tests wrapping.
* `initializeFromCLI` and `initializeForWeb`: Factorize db init.
* Add `convert-complete-to-minimal` command.
* Creates `--enable-minimal-slashing-protection` flag.
* `importSlashingProtectionJSON`: Check database type.
* `exportSlashingProtectionJSON`: Check database type.
* `TestClearDB`: Test with minimal slashing protection.
* KeyManager: Test with minimal slashing protection.
* RPC: KeyManager: Test with minimal slashing protection.
* `convert-complete-to-minimal`: Change option names.
Options were:
- `--source` (for source data directory), and
- `--target` (for target data directory)
However, since this command deals with slashing protection, which has
source (epochs) and target (epochs), the initial option names may confuse
the user.
In this commit:
`--source` ==> `--source-data-dir`
`--target` ==> `--target-data-dir`
* Set `SlashableAttestationCheck` as an iface method.
And delete `CheckSlashableAttestation` from iface.
* Move helpers functions in a more general directory.
No functional change.
* Extract common structs out of `kv`.
==> `filesystem` does not depend anymore on `kv`.
==> `iface` does not depend anymore on `kv`.
==> `slashing-protection` does not depend anymore on `kv`.
* Move `ValidateMetadata` in `validator/helpers`.
* `ValidateMetadata`: Test with mock.
This way, we can:
- Avoid any circular import for tests.
- Implement once for all `iface.ValidatorDB` implementations
the `ValidateMetadata`function.
- Have tests (and coverage) of `ValidateMetadata`in
its own package.
The ideal solution would have been to implement `ValidateMetadata` as
a method with the `iface.ValidatorDB`receiver.
Unfortunately, golang does not allow that.
* `iface.ValidatorDB`: Implement ImportStandardProtectionJSON.
The whole purpose of this commit is to avoid the `switch validatorDB.(type)`
in `ImportStandardProtectionJSON`.
* `iface.ValidatorDB`: Implement `SlashableProposalCheck`.
* Remove now useless `slashableProposalCheck`.
* Delete useless `ImportStandardProtectionJSON`.
* `file.Exists`: Detect directories and return an error.
Before, `Exists` was only able to detect if a file exists.
Now, this function takes an extra `File` or `Directory` argument.
It detects either if a file or a directory exists.
Before, if an error was returned by `os.Stat`, the the file was
considered as non existing.
Now, it is treated as a real error.
* Replace `os.Stat` by `file.Exists`.
* Remove `Is{Complete,Minimal}DatabaseExisting`.
* `publicKeys`: Add log if unexpected file found.
* Move `{Source,Target}DataDirFlag`in `db.go`.
* `failedAttLocalProtectionErr`: `var`==> `const`
* `signingRoot`: `32`==> `fieldparams.RootLength`.
* `validatorClientData`==> `validator-client-data`.
To be consistent with `slashing-protection`.
* Add progress bars for `import` and `convert`.
* `parseBlocksForUniquePublicKeys`: Move in `db/kv`.
* helpers: Remove unused `initializeProgressBar` function.
* First take at updating everything to v5
* Patch gRPC gateway to use prysm v5
Fix patch
* Update go ssz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Define `cli.App` without mutation.
No functional change.
* `usage.go`: Clean `appHelpTemplate`.
No functional change is added.
Modifications consist in adding prefix/suffix `-` to improve readability of
the template without adding new lines in template inference.
We now see some inconsistencies of the template:
- `if .App.Version` is around the `AUTHOR` section.
- `if .App.Copyright` is around both `COPYRIGHT` and `VERSION` sections.
- `if len .App.Authors` is around nothing.
* `usage.go`: Surround version and author correctly.
* `usage.go`: `AUTHOR` ==> `AUTHORS`
* `usage.go`: `GLOBAL` --> `global`.
* `--grpc-max-msg-size`: Remove double default.
* VC: Standardize help message.
- Flags help begin with a capital letter and end with a period.
- If a flag help begins with a verb, it is conjugated.
- Expermitemtal, danger etc... mentions are between parenthesis.
* VC help message: Wrap too long lines.
* scaffolding for verification package
* WIP blob verification methods
* lock wrapper for safer forkchoice sharing
* more solid cache and verification designs; adding tests
* more test coverage, adding missing cache files
* clearer func name
* remove forkchoice borrower (it's in another PR)
* revert temporary interface experiment
* lint
* nishant feedback
* add comments with spec text to all verifications
* some comments on public methods
* invert confusing verification name
* deep source
* remove cache from ProposerCache + gaz
* more consistently early return on error paths
* messed up the test with the wrong config value
* terence naming feedback
* tests on BeginsAt
* lint
* deep source...
* name errors after failure, not expectation
* deep sooource
* check len()==0 instead of nil so empty lists work
* update test for EIP-7044
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Fix exported names that start with a package name
* A few more renames
* Fix exported names that start with a package name
* A few more renames
* Radek's feedback
* Fix conflict
* fix keymanager test
* Fix comments
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* in progress...
* in progress...
* remove log
* log root
* Revert "Auxiliary commit to revert individual files from f12a609ea2a2f1e87e97321f3a717cd324b5ae97"
This reverts commit 5ae35edb6477d8d0ea4e94b273efc6590484da85.
* cleanup
* remove log
* remove whitespace
* remove logs
* more stuff
* copy
* always rebuild trie
* revert
* add state
* init state
* fix all
* uintptr
* move slice to new package
* lock in `Detach`
* remove constraint
* reorder
* blockroots and stateroots
* fill roots in empty()
* fix hasher
* implement slice for balances and inactivity scores
* detach in setters
* Revert "implement slice for balances and inactivity scores"
This reverts commit 59eb9df8d7.
# Conflicts:
# beacon-chain/state/state-native/setters_validator.go
* use counter to track states
* typos
* rename interface
* balances
* gauge
* some improvements
* first try with map
* fix
* inactivity scores in progress
* fix test
# Conflicts:
# beacon-chain/state/state-native/helpers_test.go
* test fixes
* ToProto fix
* copy roots
* validators
* build fixes
* fix bug in `ToProto`
* fix fuzz test
* fix bug in slice getters
* fix state equality checks
* make tests pass
* make tests pass
* more test updates
* Revert "Auxiliary commit to revert individual files from 34e7344bff08a589e6341bb1829e3cb74159e878"
This reverts commit ecd64efa8917f37ca41460e0356ff007fe55dd9d.
* Revert "make tests pass"
This reverts commit 0cf00f19ee.
* Revert "make tests pass"
This reverts commit 521b65e1d2.
* pass tests
* deepequal identifiable types
* Deflake `cloners_test.go`
* feature flag for block roots
* feature flag
* remove recursive locks
* reduce complexity of rootSelector
* fix randao mixes root
* some fixes
* revisit tests
* revert change to field trie helpers
* initialize field map for tests
* remove whitespace
* initialize roots with proper length
* more fixes
* out of bounds message fix
* optimize length calculation
* remove call to Len in PubkeyAtIndex
* don't log deposits
* unit tests
* unit tests
* fix
* comments
* test fixes
* id
* remove Enumerator interface
* review feedback
* simplify field trie
* bring back fieldtrie package
* fix bazel file
* use handle32ByteArrays for root computation
* fix locks
* metrics
* bzl
* simplify some things
* use htr in state test
* remove code from require package
* gzl
* more htr
* Fuzzing of the multi-value slice
* assert values
* getter optimizations
* use At when reading from validators
* Nishant's review
* restore safe copy
* remove empty line
* build fix
* restore how we get root at index for deafult mode
* more review comments
* optimize default behavior
* simplify Slice calls
* test fix
* remove unnecessary package
* remove unused setter
* make fieldMap unexported
* some improvements in state package
* call `Slice` instead of manually copying
* unlock in ReadFromEveryValidator
* Potuz's comments
* lock the state when reading from all validators
# Conflicts:
# beacon-chain/state/state-native/getters_validator.go
* add back preston's changes
* add index
---------
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* update github actions
* use quotes or it is go 1.2
lol
* Update gosec
* Update gosec
* Update go lint
* fix gosec violations
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* refactor initialization to blocking startup method
* require genesisSetter in blockchain, fix tests
* work-around gazelle weirdness
* fix dep gazelle ignores
* only call SetGenesis once
* fix typo
* validator test setup and fix to return right error
* move waitForChainStart to Start
* wire up sync Service.genesisWaiter
* fix p2p genesisWaiter plumbing
* remove extra clock type, integrate into genesis
and rename
* use time.Now when no Nower is specified
* remove unused ClockSetter
* simplify rpc context checking
* fix typo
* use clock everywhere in sync; [32]byte val root
* don't use DeepEqual to compare [32]byte and []byte
* don't use clock in init sync, not wired up yet
* use clock waiter in blockchain as well
* use cancelable contexts in tests with goroutines
* missed a reference to WithClockSetter
* Update beacon-chain/startup/genesis.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/blockchain/service_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* more clear docs
* doc for NewClock
* move clock typedef to more logical file name
* adding documentation
* gaz
* fixes for capella
* reducing test raciness
* fix races in committee cache tests
* lint
* add tests on Duration slot math helper
* startup package test coverage
* fix bad merge
* set non-zero genesis time in tests that call Start
* happy deepsource, happy me-epsource
* replace Synced event with channel
* remove unused error
* remove accidental wip commit
* gaz!
* remove unused event constants
* remove sync statefeed subscription to fix deadlock
* remove state notifier
* fix build
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* prysmctl using the same genesis func as e2e
* add whitespace to genesis.json for readability
* fix typo in fork name
* don't require validator count if deposits given
* add gosec exception
* the other nosec :(
* appease deepsource
* fix comments on renamed public value/func
---------
Co-authored-by: kasey <kasey@users.noreply.github.com>
