#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
* Vendor go-bip39 dependency locally to third_party/
The github.com/tyler-smith/go-bip39 repository has been deleted from GitHub
but is still needed for BIP-39 mnemonic functionality in the validator wallet
system. This change vendors v1.1.0 of the library into third_party/go-bip39/
to ensure continued availability.
Changes:
- Copy go-bip39 v1.1.0 source from Go module cache to third_party/go-bip39/
- Create BUILD.bazel files for main package and wordlists subpackage
- Update 5 BUILD.bazel files to reference local vendored version instead of external dependency
- Remove go-bip39 from go.mod and deps.bzl
- All builds and tests pass successfully
The vendored package includes all 9 language wordlists (English, Chinese Simplified/Traditional,
Czech, French, Italian, Japanese, Korean, Spanish) and maintains the original import paths for
compatibility.
* Changelog fraagment
* use go mod replace for vendored lib
* Run gazelle
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Migrate Prysm repo to Offchain Labs organization ahead of Pectra upgrade v6
* Replace prysmaticlabs with OffchainLabs on general markdowns
* Update mock
* Gazelle and add mock.go to excluded generated mock file
* First take at updating everything to v5
* Patch gRPC gateway to use prysm v5
Fix patch
* Update go ssz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* WIP
* fixing tests
* fixing bazel
* fixing api client
* fixing tests
* fixing more tests and bazel
* fixing trace and more bazel issues
* fixing router path function definitions
* fixing more tests and deep source issues
* adding delete test
* if a route is provided, reregister before the catch all on the middleware.
* fixing linting
* fixing deepsource complaint
* gaz
* more deepsource issues
* fixing missed err check
* changing how routes are registered
* radek reviews
* Update validator/rpc/handlers_keymanager.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Update validator/rpc/handlers_keymanager.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* fixing unit test after sammy's review
* adding radek's comments
---------
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Return error when using unsupported mnemonic language
* Fix failing tests
* More test fixes
* Add MnemonicLanguage to CreateWalletRequest
* Add MnemonicLanguage to remaining tests
* Add language to error
* Add missing (.)
* Set default language + fix renumbered fields
* Replace hardcoded language with default var
* default set
* gaz
* fix proto field setting
* fix up
* Remove WithMnemonicLanguage from tests
* Remove WithMnemonicLanguage from tests
* Fix conflicting information failure
* Gazelle + remove hardcoded english values
* fix
* Fix tests
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Migrating Keymanager account list functionality into each keymanager type
* Addressing review comments
* Adding newline at end of BUILD.bazel
* bazel run //:gazelle -- fix
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* imported -> local
* reverting to the state of the PR at eb1e3c3d1
* use changes from develop
* del
* rem patch
* patch
* rename to local
* gazelle
* add back build
* imported rename
* gaz
* local
* merge fix + remove proto changes
* comment revert
* build
* gofmt and one new reference
* gofmt pt2
* Update validator/accounts/wallet_edit.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/rpc/accounts.go
* rename
* gaz
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* Value assigned to a variable is never read before being overwritten
* The result of append is not used anywhere
* Suspicious assignment of range-loop vars detected
* Unused method receiver detected
* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"
This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.
* Method modifies receiver
* Fix test
* Duplicate imports detected
* Incorrectly formatted error string
* Types of function parameters can be combined
* One more "Unused method receiver detected"
* Unused parameter detected in function
* begin
* rem deleted code
* delete keystores all tests
* surface errors to user
* add in changes
* del
* tests
* slice
* begin import process
* add import keystores logic
* unit tests for import
* tests for all import keystores keymanager issues
* change proto
* pbs
* renaming works
* use proper request
* pb
* comment
* gaz
* fix up cli cmd
* test
* add gw
* precond
* tests
* radek comments
* begin
* implement delete and filter export history
* rem deleted code
* delete keystores all tests
* gaz
* test
* double import fix
* test
* surface errors to user
* add in changes
* edit proto
* edit
* del
* tests
* gaz
* slice
* duplicate key found in request
* Add a tool to split a series of mnemonic keys into distinct wallets
* split func
* keysplit tool
* gaz/viz
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* refactor / move waiting for activation updates
* Commentary
* Update test to follow the full code path
* gofmt and goimports
* manual imports fixes
* Apply suggestions from code review
typo fixes
* Remove redundant handleAccountsChanged and chan. Thanks @nisdas
* var sub = to sub :=
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Make 1 occurence of 'foo', 'bar' and 'fizz!'
* Make 1 occurence of 'merkleizing list that is too large, over limit' string
* Limit password occurrences
* Make only 1 occurence of 'strongPass'
* Limit testMnemonic occurrences
* Limit expected epoch error messages
* Rename errors and use constant
* Update bazel dependencies
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* initial implementation
* remove listening for changes from wallet creation
* goimports
* test fix
* more goimports
* listen for changes when initializing wallet through gRPC
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* begin hd wallet refactor
* further simplify the new derived keymanager
* make it almost a full wrapper around an imported keymanager
* fix up the EIP test
* deprecated derived
* fixing keymanager tests
* fix up derived tests
* refactor initialize keymanager
* simplify hd
* pass some tests
* pass accounts list test
* gaz
* regenerate protos without create account privilege
* enforce account recovery on wallet create
* allow accounts delete to work
* remove mentions of accounts create
* resolve comments and go mod
* fix up tests
* build fixes
* remove insecure warning
* revert
* fix proto file
* remove create account message
* gaz
* remove account create
* update web api protos
* fix up imports
* change func sig
* tidy
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* add --enable --disable flags for validator accounts
* refactor DeleteAccountConfig into AccountConfig to be used for enable and disable feature
* add `disable` flag for validator accounts
* [wip] add method to disable account
* refactor account delete
* add disable & enable with proper filters
* fix keymanager unit tests
* update DisabledPublicKeys to be a string instead of [][]byte
* fix FetchValidatingPrivateKeys to only fetch active keys with new string format
* fix FetchValidationPrivateKeys with new DisabledPublicKeys format (as a string)
* rename file + update AccountsConfig to include Disable, Enable and Delete distinct attributes
* rename accounts_activation -> accounts_enable_disable
* revert changes from using string to [][]byte for DisabledPublicKeys
* add FetchAllValidatingPublicKeys to preserve the functionality for accounts list, backup and delete
* fix unit tests
* convert publickeys from [][]byte to str before passing it to pb message
* add unit tests for disable keys
* add unit tests for EnableAccounts
* revert WORKSPACE LLM for now
* ran gazelle
* move function to convert KeymanagerOpts to Config inside rpc and run gazelle
* add unit tests for FetchAllValidatingPublicKeys
* fix keymanageropts for InteropKey
* Fix mistake for enable accounts
* add docstring to DisableAccountsCli and EnableAccountsCli
* remove previous testnet and add toledo & pyrmont
* fix build from source
* clean up
* update again
* change everything
* workaround for now
* fix versioning
* all passing now
* fix build issues
* clean up
* revert use of MulVerify
* gaz
* stub
* Apply suggestions from code review
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* fix all
* fix test
* todo
* fix stub
* revert back
* make deep source happy
* Update shared/bls/herumi/public_key.go
* Update shared/bls/blst/signature.go
* Update shared/bls/blst/signature_test.go
* imports
* move iface to common, export errors
* rm iface build
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
