#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
* fix: replace fmt.Printf with proper test error handling in web3signer test
* Update keymanager_test.go
* Create DeVikingMark_fix-web3signer-test-error-handling.md
* adding web 3 signer changes for fulu
* missed adding fulu values
* add accounting for fulu version
* updated web3signer version and fixing linting
* Update validator/keymanager/remote-web3signer/types/requests_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update validator/keymanager/remote-web3signer/types/mock/mocks.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* radek suggestions
* removing redundant check and removing old function, changed changelog to reflect
* gaz
* radek's suggestion
* fixing test as v1 proof is no longer used
* fixing more tests
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Add log capitalization analyzer and apply fixes across codebase
Implements a new nogo analyzer to enforce proper log message capitalization and applies the fixes to all affected log statements throughout the beacon chain, validator, and supporting components.
Co-Authored-By: Claude <noreply@anthropic.com>
* Radek's feedback
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* Fixed otelhttp client setups.
Note: This may not be the best solution as the http client is defined in many places. There should be a canoncial http client with the proper setup.
* Changelog fragment
* go mod tidy and gazelle
* updating blockv2 to handle electra blocks
* adding aggregate attesation and proof electra
* gaz
* changelogs
* updating web3signer dependency
* test mock was flipped
* fixing hex value
* accidently checked in dependency changes
* preston feedback
* readding old metrics to not break linting
* review feedback and changelog
* gaz
* Adds a wrapper package
* Gazelle
* Add in Empty Span
* Revert It Back
* Add back reference
* Set It As Empty
* fix missing import
* remove redundant alias
* remove unused
---------
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Saolyn <sammy@rosso.red>
* First take at updating everything to v5
* Patch gRPC gateway to use prysm v5
Fix patch
* Update go ssz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* WIP
* fixing tests
* fixing bazel
* fixing api client
* fixing tests
* fixing more tests and bazel
* fixing trace and more bazel issues
* fixing router path function definitions
* fixing more tests and deep source issues
* adding delete test
* if a route is provided, reregister before the catch all on the middleware.
* fixing linting
* fixing deepsource complaint
* gaz
* more deepsource issues
* fixing missed err check
* changing how routes are registered
* radek reviews
* Update validator/rpc/handlers_keymanager.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Update validator/rpc/handlers_keymanager.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* fixing unit test after sammy's review
* adding radek's comments
---------
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Fix exported names that start with a package name
* A few more renames
* Fix exported names that start with a package name
* A few more renames
* Radek's feedback
* Fix conflict
* fix keymanager test
* Fix comments
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* wip
* adding deneb block
* adding in support for blobs and fixing unit tests for deneb
* fixing linting
* gaz
* adding support for new web3signer version
* fixing tag name
* addressing feedback
* fixing tests
* adding unit test for review
* updating test name
* updating unit tests and length logic
* adding in lengthfor root
* adjusting max blob length
* fixing mock
* fixing another mock
* gaz
* adding network configs
* removing duplicate
* changing based on nishant's feedback
* Update validator/keymanager/remote-web3signer/v1/requests.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Update validator/keymanager/remote-web3signer/metrics.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* sammy's suggestions
* removing temp file
---------
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* removing all prysm remote signer code
* fixing unit tests
* resolving more build issues
* resolving deepsource complaint
* fixing lint
* trying to fix bazel library
* trying testonly true
* removing assert and require from non test settings
* fixing bazel and tests
* removing more unused files related to remote signer
* fixing linting
* reverting some changes
* reverting a change that broke some code
* removing typo
* fixing unit test
* fixing mnemonic information
* updating proto name
* updating generated files
* updating names on block v2 and v3 signing types
* fixing import format
* fixing unit tests
* fixing unit test
* fixing more conflicts
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* panic in SizeSSZ
* moving slowly
* adapt old code to new interfaces
* return interfaces from factory functions
* replace the rest of WrappedSignedBeaconBlock
* WrappedBeaconBlock
* WrappedBeaconBlockBody
* miscellaneous
* Test_BeaconBlockIsNil
* replace usages of BeaconBlockIsNil
* replace usages of mutator
* fix all build errors
* fix some more issues
* mutator changes
* relax assertions when initializing
* revert changes in object_mapping.go
* allow calling Proto on nil
* Revert "allow calling Proto on nil"
This reverts commit ecc84e4553.
* modify Copy and Proto methods
* remove unused var
* fix block batch tests
* correct BUILD file
* Error when initializing nil objects
* one more error fix
* add missing comma
* rename alias to blocktest
* add logging
* error when SignedBeaconBlock is nil
* fix last test
* import fix
* broken
* working
* test fixes
* reduce complexity of processPendingBlocks
* simplified
* initial commit
* adjusting logic to match web3signer specifications
* making Epoch mandatory
* fixing unit test
* rolling back some changes on slots
* fixing unit tests
* testing enable validator registration
* adding validator registration to configs
* fixing prefix
* fixing shasum
* rolling back e2e validator registration until fully supported
* updating web3signer version
* changing types based on feedback
* initial commit
* fixing bazel
* removing extra space
* adding blindedbeaconblock
* adding unit tests
* fixing bazel build
* switching to switch statement
* fixing function parameters
* fixing ineffectual err error
* deleting unused files
* updating web3signer version to support bellatrix
* adding metrics
* testing longer run
* changing log level to help find errors
* changing logging back to all to better understand the issue
* testing better way to get public keys for web3signer on validator
* fixing bazel
* rolling back changes
* missed rolling back 1 thing
* adding flag back in
* adding comments back in
* testing lower participation
* adding logs to see web3signer keys for comparison
* fix bazel
* adding more logs temporariliy
* switching hex utility function
* web3signer doesn't support deposits, so changing this
* rolling back unintended unit test
* rolling back some changes for debugging
* Update validator/keymanager/remote-web3signer/v1/web3signer_types.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/components/web3remotesigner.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/endtoend_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/endtoend_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* fixing merge conflict
* Update validator/keymanager/remote-web3signer/v1/requests.go
* Update testing/endtoend/endtoend_test.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/keymanager/remote-web3signer/v1/requests.go
Co-authored-by: terencechain <terence@prysmaticlabs.com>
* prysm doesn't currently support deposits for web3signer
* reverting back to old implementation
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
* replace eth2 types
* replace protos
* regen proto
* replace
* gaz
* deps
* amend
* regen proto
* mod
* gaz
* gaz
* ensure build
* ssz
* add dep
* no more eth2 types
* no more eth2
* remg
* all builds
* buidl
* tidy
* clean
* fmt
* val serv
* gaz
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
