mirror of
https://github.com/OffchainLabs/prysm.git
synced 2026-01-08 07:03:58 -05:00
6b5ba5ad01
#### This PR sets the foundation for the new logging features. --- The goal of this big PR is the following: 1. Adding a log.go file to every package: [_commit_](54f6396d4c) - Writing a bash script that adds the log.go file to every package that imports logrus, except the excluded packages, configured at the top of the bash script. - the log.go file creates a log variable and sets a field called `package` to the full path of that package. - I have tried to fix every error/problem that came from mass generation of this file. (duplicate declarations, different prefix names, etc...) - some packages had the log.go file from before, and had some helper functions in there as well. I've moved all of them to a `log_helpers.go` file within each package. 2. Create a CI rule which verifies that: [_commit_](b799c3a0ef) - every package which imports logrus, also has a log.go file, except the excluded packages. - the `package` field of each log.go variable, has the correct path. (to detect when we move a package or change it's name) - I pushed a commit with a manually changed log.go file to trigger the ci check failure and it worked. 3. Alter the logging system to read the prefix from this `package` field for every log while outputing: [_commit_](b0c7f1146c) - some packages have/want/need a different log prefix than their package name (like `kv`). This can be solved by keeping a map of package paths to prefix names somewhere. --- **Some notes:** - Please review everything carefully. - I created the `prefixReplacement` map and populated the data that I deemed necessary. Please check it and complain if something doesn't make sense or is missing. I attached at the bottom, the list of all the packages that used to use a different name than their package name as their prefix. - I have chosen to mark some packages to be excluded from this whole process. They will either not log anything, or log without a prefix, or log using their previously defined prefix. See the list of exclusions in the bottom. - I fixed all the tests that failed because of this change. These were failing because they were expecting the old prefix to be in the generated logs. I have changed those to expect the new `package` field instead. This might not be a great solution. Ideally we might want to remove this from the tests so they only test for relevant fields in the logs. but this is a problem for another day. - Please run the node with this config, and mention if you see something weird in the logs. (use different verbosities) - The CI workflow uses a script that basically runs the `hack/gen-logs.sh` and checks that the git diff is zero. that script is `hack/check-logs.sh`. This means that if one runs this script locally, it will not actually _check_ anything, rather than just regenerate the log.go files and fix any mistake. This might be confusing. Please suggest solutions if you think it's a problem. --- **A list of packages that used a different prefix than their package names for their logs:** - beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix "cache" - beacon-chain/core/transition/log.go — package transition, prefix "state" - beacon-chain/db/kv/log.go — package kv, prefix "db" - beacon-chain/db/slasherkv/log.go — package slasherkv, prefix "slasherdb" - beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner" - beacon-chain/light-client/log.go — package light_client, prefix "light-client" - beacon-chain/operations/attestations/log.go — package attestations, prefix "pool/attestations" - beacon-chain/operations/slashings/log.go — package slashings, prefix "pool/slashings" - beacon-chain/rpc/core/log.go — package core, prefix "rpc/core" - beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix "rpc/beaconv1" - beacon-chain/rpc/eth/validator/log.go — package validator, prefix "beacon-api" - beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix "rpc" - beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator, prefix "rpc/validator" - beacon-chain/state/stategen/log.go — package stategen, prefix "state-gen" - beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix "checkpoint-sync" - beacon-chain/sync/initial-sync/log.go — package initialsync, prefix "initial-sync" - cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p" - config/features/log.go -- package features, prefix "flags" - io/file/log.go — package file, prefix "fileutil" - proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf" - validator/client/beacon-api/log.go — package beacon_api, prefix "beacon-api" - validator/db/kv/log.go — package kv, prefix "db" - validator/db/filesystem/db.go — package filesystem, prefix "db" - validator/keymanager/derived/log.go — package derived, prefix "derived-keymanager" - validator/keymanager/local/log.go — package local, prefix "local-keymanager" - validator/keymanager/remote-web3signer/log.go — package remote_web3signer, prefix "remote-keymanager" - validator/keymanager/remote-web3signer/internal/log.go — package internal, prefix "remote-web3signer- internal" - beacon-chain/forkchoice/doubly... prefix is "forkchoice-doublylinkedtree" **List of excluded directories (their subdirectories are also excluded):** ``` EXCLUDED_PATH_PREFIXES=( "testing" "validator/client/testutil" "beacon-chain/p2p/testing" "beacon-chain/rpc/eth/config" "beacon-chain/rpc/prysm/v1alpha1/debug" "tools" "runtime" "monitoring" "io" "cmd" ".well-known" "changelog" "hack" "specrefs" "third_party" "bazel-out" "bazel-bin" "bazel-prysm" "bazel-testlogs" "build" ".github" ".jj" ".idea" ".vscode" ) ```
Web3Signer
Web3Signer is a popular remote signer tool by Consensys to allow users to store validation keys outside the validation
client and signed without the vc knowing the private keys. Web3Signer Specs are found by
searching Consensys' Web3Signer API specification
issue: https://github.com/prysmaticlabs/prysm/issues/9994
API interface: https://github.com/ethereum/remote-signing-api
Features
CLI
detailed info found on https://docs.prylabs.network/docs/wallet/web3signer
Flags used on validator client
--validators-external-signer-url=http://localhost:9000
with hex keys
--validators-external-signer-public-keys=0xa99a...e44c,0xb89b...4a0b
with url
--validators-external-signer-public-keys=https://web3signer.com/api/v1/eth2/publicKeys
API
- Get Public keys: returns all public keys currently stored with web3signer excluding newly added keys if reload keys was not run.
- Sign: Signs a message with a given public key. There are several types of messages that can be signed ( web3signer
type to prysm type):
- BLOCK <- *validatorpb.SignRequest_Block
- ATTESTATION <- *validatorpb.SignRequest_AttestationData
- AGGREGATE_AND_PROOF <- *validatorpb.SignRequest_AggregateAttestationAndProof
- AGGREGATION_SLOT <- *validatorpb.SignRequest_Slot
- BLOCK_ALTAIR <- *validatorpb.SignRequest_BlockAltair
- BLOCK_BELLATRIX <- *validatorpb.SignRequest_BlockBellatrix
- BLINDED_BLOCK_BELLATRIX <- *validatorpb.SignRequest_BlindedBlockBellatrix
- DEPOSIT <- not supported
- RANDAO_REVEAL <- *validatorpb.SignRequest_Epoch
- VOLUNTARY_EXIT <- *validatorpb.SignRequest_Exit
- SYNC_COMMITTEE_MESSAGE <- *validatorpb.SignRequest_SyncMessageBlockRoot
- SYNC_COMMITTEE_SELECTION_PROOF <- *validatorpb.SignRequest_SyncAggregatorSelectionData
- SYNC_COMMITTEE_CONTRIBUTION_AND_PROOF <- *validatorpb.SignRequest_ContributionAndProof
- Reload Keys: reloads all public keys from the web3signer.
- Get Server Status: returns OK if the web3signer is ok.
Files Added and Files Changed
-
Files Added:
- validator/keymanager/remote-web3signer package
-
Files Modified:
- modified: cmd/validator/flags/flags.go
- modified: validator/accounts/accounts_backup.go
- modified: validator/accounts/accounts_list.go
- modified: validator/accounts/iface/wallet.go
- modified: validator/accounts/userprompt/prompt.go
- modified: validator/accounts/wallet/wallet.go
- modified: validator/accounts/wallet_create.go
- modified: validator/client/runner.go
- modified: validator/client/validator.go
- modified: validator/keymanager/remote-web3signer/keymanager.go
- modified: validator/keymanager/types.go