<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Bug fix
**What does this PR do? Why is it needed?**
validator fallbacks shouldn't work on nodes that are syncing as many of
the tasks validators perform require the node to be fully synced.
- 206 or any other code is interpreted as "not ready"
- 200 interpreted as "ready"
**Which issues(s) does this PR fix?**
continuation of https://github.com/OffchainLabs/prysm/pull/15401
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
* Remove Beacon API endpoints that were deprecated in Electra
* changelog <3
* build fix
* remove more stuff
* fix post-submit e2e and remove structs
* list endpoints in the changelog
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* removing ssz-only flag
* gaz
* reverting other uses of sszonly
* gaz
* adding kasey and radek's suggestions
* update changelog
* adding test
* radek advice with new headers and tests
* adding logs and fixing comments
* adding logs and fixing comments
* gaz
* Update validator/client/beacon-api/rest_handler_client.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's comments
* adding another failing case based on radek's suggestion
* another unit test
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Convert genesis times from seconds to time.Time
* Fixing failed forkchoice tests in a new commit so it doesn't get worse
Fixing failed spectest tests in a new commit so it doesn't get worse
Fixing forkchoice tests, then spectests
* Fixing forkchoice tests, then spectests. Now asking for help...
* Fix TestForkChoice_GetProposerHead
* Fix broken build
* Resolve TODO(preston) items
* Changelog fragment
* Resolve TODO(preston) items again
* Resolve lint issues
* Use consistant field names for sinceSlotStart (no spaces)
* Manu's feedback
* Renamed StartTime -> UnsafeStartTime, marked as deprecated because it doesn't handle overflow scenarios.
Renamed SlotTime -> StartTime
Renamed SlotAt -> At
Handled the error in cases where StartTime was used.
@james-prysm feedback
* Revert beacon-chain/blockchain/receive_block_test.go from 1b7844de
* Fixing issues after rebase
* Accepted suggestions from @potuz
* Remove CanonicalHeadSlot from merge conflicts
---------
Co-authored-by: potuz <potuz@prysmaticlabs.com>
* poc changes for safe validator shutdown
* simplifying health routine and adding safe shutdown after max restarts reached
* fixing health tests
* fixing tests
* changelog
* gofmt
* fixing runner
* improve how runner times out
* improvements to ux on logs
* linting
* adding in max healthcheck flag
* changelog
* Update james-prysm_safe-validator-shutdown.md
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/service.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing some feedback from radek
* addressing some more feedback
* fixing name based on feedback
* fixing mistake on max health checks
* conflict accidently checked in
* go 1.23 no longer needs you to stop for the ticker
* Update flags.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* wip no unit test for recursive healthy host find
* rework healthcheck
* gaz
* fixing bugs and improving logs with new monitor
* removing health tracker, fixing runner tests, and adding placeholder for monitor tests
* fixing event stream check
* linting
* adding in health monitor tests
* gaz
* improving test
* removing some log.fatals
* forgot to remove comment
* missed fatal removal
* doppleganger should exit the node safely now
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek review
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek feedback
* read up on more suggestions and making fixes to channel
* suggested updates after more reading
* reverting some of this because it froze the validator after healthcheck failed
* fully reverting
* some improvements I found during testing
* Update cmd/validator/flags/flags.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* preston's feedback
* clarifications on changelog
* converted to using an event feed instead of my own channel publishing implementation, adding relevant logs
* preston log suggestion
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* wip
* fixing unit tests
* changing is aggregator function
* wip
* fully removing the use of committee from validator client, adding a wrapper type for duties
* fixing tests
* fixing linting
* fixing more tests
* changelog
* adding some more tests
* Update proto/prysm/v1alpha1/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's feedback
* removing accidently checked in
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Implement static analysis to prevent panics
* Add nopanic to nogo
* Fix violations and add exclusions
Fix violations and add exclusions for all
* Changelog fragment
* Use pass.Report instead of pass.Reportf
* Remove strings.ToLower for checking init method name
* Add exclusion for herumi init
* Move api/client/beacon template function to init and its own file
* Fix nopanic testcase
* adding in check for non prysm node and moving the prysm endpoint to the prysm beacon client
* fixing a bug connecting to a non prysm client and moving the prysm api call to the prysm beacon client
* changelog
* fixing linting
* Update validator/client/metrics.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* change field IDs in `AggregateAttestationAndProofElectra`
* fix typo in `validator.proto`
* correct slashing indices length and shashings length
* check length in indexed attestation's `ToConsensus` method
* use `fieldparams.BLSSignatureLength`
* Add length checks for execution request
* fix typo in `beacon_state.proto`
* fix typo in `ssz_proto_library.bzl`
* fix error messages about incorrect types in block factory
* add Electra case to `BeaconBlockContainerToSignedBeaconBlock`
* move PeerDAS config items to PeerDAS section
* remove redundant params
* rename `PendingDepositLimit` to `PendingDepositsLimit`
* improve requests unmarshaling errors
* rename `get_validator_max_effective_balance` to `get_max_effective_balance`
* fix typo in `consolidations.go`
* rename `index` to `validator_index` in `PendingPartialWithdrawal`
* rename `randomByte` to `randomBytes` in `validators.go`
* fix for version in a comment in `validator.go`
* changelog <3
* Revert "rename `index` to `validator_index` in `PendingPartialWithdrawal`"
This reverts commit 87e4da0ea2.
* Prepare for future fork boilerplate.
* Implement the Fulu fork boilerplate.
* `Upgraded state to <fork> log`: Move from debug to info.
Rationale:
This log is the only one notifying the user a new fork happened.
A new fork is always a little bit stressful for a node operator.
Having at least one log indicating the client switched fork is something useful.
* Update testing/util/helpers.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/state/state-native/state_trie.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Fix Radek's comment.
* Fix Radek's comment.
* Fix Radek's comment.
* Remove Electra struct type aliasing.
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* wip passing e2e
* reverting temp comment
* remove unneeded comments
* fixing merge errors
* fixing more bugs from merge
* fixing test
* WIP moving code around and fixing tests
* unused linting
* gaz
* temp removing these tests as we need placeholder/wrapper APIs for them with the removal of the gateway
* attempting to remove dependencies to gRPC gateway , 1 mroe left in deps.bzl
* renaming flags and other gateway services to http
* goimport
* fixing deepsource
* git mv
* Update validator/package/validator.yaml
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/package/validator.yaml
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update cmd/beacon-chain/flags/base.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing feedback
* missed lint
* renaming import
* reversal based on feedback
* fixing web ui registration
* don't require mux handler
* gaz
* removing gRPC service from validator completely, merged with http service, renames are a work in progress
* updating go.sum
* linting
* trailing white space
* realized there was more cleanup i could do with code reuse
* adding wrapper for routes
* reverting version
* fixing dependencies from merging develop
* gaz
* fixing unit test
* fixing dependencies
* reverting unit test
* fixing conflict
* updating change log
* Update log.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* gaz
* Update api/server/httprest/server.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* addressing some feedback
* forgot to remove deprecated flag in usage
* gofmt
* fixing test
* fixing deepsource issue
* moving deprecated flag and adding timeout handler
* missed removal of a flag
* fixing test:
* Update CHANGELOG.md
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing feedback
* updating comments based on feedback
* removing unused field for now, we can add it back in if we need to use the option
* removing unused struct
* changing api-timeout flag based on feedback
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Adds a wrapper package
* Gazelle
* Add in Empty Span
* Revert It Back
* Add back reference
* Set It As Empty
* fix missing import
* remove redundant alias
* remove unused
---------
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Saolyn <sammy@rosso.red>
* Cleanup part 1
* Cleanup part 2
* Cleanup part 3
* remove lock field init
* doc for SignerConfig
* remove vars
* use full Keymanager word in function
* revert interface rename
* linter
* fix build issues
* review
