<!-- Thanks for sending a PR! Before submitting:
1. If this is your first PR, check out our contribution guide here
https://docs.prylabs.network/docs/contribute/contribution-guidelines
You will then need to sign our Contributor License Agreement (CLA),
which will show up as a comment from a bot in this pull request after
you open it. We cannot review code without a signed CLA.
2. Please file an associated tracking issue if this pull request is
non-trivial and requires context for our team to understand. All
features and most bug fixes should have
an associated issue with a design discussed and decided upon. Small bug
fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
Bug fix
**What does this PR do? Why is it needed?**
validator fallbacks shouldn't work on nodes that are syncing as many of
the tasks validators perform require the node to be fully synced.
- 206 or any other code is interpreted as "not ready"
- 200 interpreted as "ready"
**Which issues(s) does this PR fix?**
continuation of https://github.com/OffchainLabs/prysm/pull/15401
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
* Vendor go-bip39 dependency locally to third_party/
The github.com/tyler-smith/go-bip39 repository has been deleted from GitHub
but is still needed for BIP-39 mnemonic functionality in the validator wallet
system. This change vendors v1.1.0 of the library into third_party/go-bip39/
to ensure continued availability.
Changes:
- Copy go-bip39 v1.1.0 source from Go module cache to third_party/go-bip39/
- Create BUILD.bazel files for main package and wordlists subpackage
- Update 5 BUILD.bazel files to reference local vendored version instead of external dependency
- Remove go-bip39 from go.mod and deps.bzl
- All builds and tests pass successfully
The vendored package includes all 9 language wordlists (English, Chinese Simplified/Traditional,
Czech, French, Italian, Japanese, Korean, Spanish) and maintains the original import paths for
compatibility.
* Changelog fraagment
* use go mod replace for vendored lib
* Run gazelle
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Remove Beacon API endpoints that were deprecated in Electra
* changelog <3
* build fix
* remove more stuff
* fix post-submit e2e and remove structs
* list endpoints in the changelog
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* fix: replace fmt.Printf with proper test error handling in web3signer test
* Update keymanager_test.go
* Create DeVikingMark_fix-web3signer-test-error-handling.md
* Fix misleading log msg on shutdown
gRPCServer.GracefulStop blocks until it has been shutdown. Logging
"Initiated graceful stop" after it has been completed is misleading.
Names are added to the message to discern services. Also, a minimum test
is added mainly to verify the change made with this commit.
* Add changelog fragment file
* Capitalize log messages
* Update endtoend test for fixed log messages
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* fix: use assigned committee index in attestation data after Electra
* update change log
* update add unit test
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* adding web 3 signer changes for fulu
* missed adding fulu values
* add accounting for fulu version
* updated web3signer version and fixing linting
* Update validator/keymanager/remote-web3signer/types/requests_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/keymanager/remote-web3signer/types/mock/mocks.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek suggestions
* removing redundant check and removing old function, changed changelog to reflect
* gaz
* radek's suggestion
* fixing test as v1 proof is no longer used
* fixing more tests
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* removing ssz-only flag
* gaz
* reverting other uses of sszonly
* gaz
* adding kasey and radek's suggestions
* update changelog
* adding test
* radek advice with new headers and tests
* adding logs and fixing comments
* adding logs and fixing comments
* gaz
* Update validator/client/beacon-api/rest_handler_client.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update api/apiutil/header.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek's comments
* adding another failing case based on radek's suggestion
* another unit test
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Convert genesis times from seconds to time.Time
* Fixing failed forkchoice tests in a new commit so it doesn't get worse
Fixing failed spectest tests in a new commit so it doesn't get worse
Fixing forkchoice tests, then spectests
* Fixing forkchoice tests, then spectests. Now asking for help...
* Fix TestForkChoice_GetProposerHead
* Fix broken build
* Resolve TODO(preston) items
* Changelog fragment
* Resolve TODO(preston) items again
* Resolve lint issues
* Use consistant field names for sinceSlotStart (no spaces)
* Manu's feedback
* Renamed StartTime -> UnsafeStartTime, marked as deprecated because it doesn't handle overflow scenarios.
Renamed SlotTime -> StartTime
Renamed SlotAt -> At
Handled the error in cases where StartTime was used.
@james-prysm feedback
* Revert beacon-chain/blockchain/receive_block_test.go from 1b7844de
* Fixing issues after rebase
* Accepted suggestions from @potuz
* Remove CanonicalHeadSlot from merge conflicts
---------
Co-authored-by: potuz <potuz@prysmaticlabs.com>
* Reword DV selection proofs
* Add changelog
* Expect 1 call to DomainData in DV update duties test
---------
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* moving the ticker from chain start to right before the main loop and also after the wait for activation edge case
* fixing unit test
* adding in a unit test
* adding in comment based on potuz's feedback
* Add log capitalization analyzer and apply fixes across codebase
Implements a new nogo analyzer to enforce proper log message capitalization and applies the fixes to all affected log statements throughout the beacon chain, validator, and supporting components.
Co-Authored-By: Claude <noreply@anthropic.com>
* Radek's feedback
---------
Co-authored-by: Claude <noreply@anthropic.com>
* poc changes for safe validator shutdown
* simplifying health routine and adding safe shutdown after max restarts reached
* fixing health tests
* fixing tests
* changelog
* gofmt
* fixing runner
* improve how runner times out
* improvements to ux on logs
* linting
* adding in max healthcheck flag
* changelog
* Update james-prysm_safe-validator-shutdown.md
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/service.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/runner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing some feedback from radek
* addressing some more feedback
* fixing name based on feedback
* fixing mistake on max health checks
* conflict accidently checked in
* go 1.23 no longer needs you to stop for the ticker
* Update flags.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* wip no unit test for recursive healthy host find
* rework healthcheck
* gaz
* fixing bugs and improving logs with new monitor
* removing health tracker, fixing runner tests, and adding placeholder for monitor tests
* fixing event stream check
* linting
* adding in health monitor tests
* gaz
* improving test
* removing some log.fatals
* forgot to remove comment
* missed fatal removal
* doppleganger should exit the node safely now
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek review
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/health_monitor.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/client/validator.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek feedback
* read up on more suggestions and making fixes to channel
* suggested updates after more reading
* reverting some of this because it froze the validator after healthcheck failed
* fully reverting
* some improvements I found during testing
* Update cmd/validator/flags/flags.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* preston's feedback
* clarifications on changelog
* converted to using an event feed instead of my own channel publishing implementation, adding relevant logs
* preston log suggestion
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* adding in proto for getdutiesv2
* adding in GetDutiesV2
* updating changelog and mock
* fixing tests
* breaking up function into smaller functions for gocognit
* reverting some changes so we can break it into a separate PR for validator client only
* reverted too much adding mock change back in
* removing reserved based on preston's feedback
* adding duties tests back in
* Update beacon-chain/rpc/prysm/v1alpha1/validator/duties.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* updating based on preston's feedback
* updating build for new files
* maybe there's some flake with how the state is used, trying with it moved
* reverting config override
* reverting all changes to get duties v1 based on preston's feedback
* Update proto/prysm/v1alpha1/validator.proto
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing partial feedback
* adding small comment
* reorganizing function based on feedback
* removing unused field
* adding some more unit tests
* adding attribute for pubkeys to span
* preston's feedback
* fixing current to next
* probably safer to register current period now
* Update beacon-chain/core/helpers/beacon_committee.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* adding in preston's comment
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Force duties update on received blocks.
- Change the context on UpdateDuties to be passed by the calling
function.
- Change the context passed to UpdateDuties to not be dependent on a
slot context.
- Change the deadlines to be forced to be an entire epoch.
- Force duties to be initialized when receiving a HeadEvent if they
aren't already.
- Adds a read lock on the event handling
* review
* Add deadlines at start and healthyagain
* cancel once
