#### This PR sets the foundation for the new logging features.
---
The goal of this big PR is the following:
1. Adding a log.go file to every package:
[_commit_](54f6396d4c)
- Writing a bash script that adds the log.go file to every package that
imports logrus, except the excluded packages, configured at the top of
the bash script.
- the log.go file creates a log variable and sets a field called
`package` to the full path of that package.
- I have tried to fix every error/problem that came from mass generation
of this file. (duplicate declarations, different prefix names, etc...)
- some packages had the log.go file from before, and had some helper
functions in there as well. I've moved all of them to a `log_helpers.go`
file within each package.
2. Create a CI rule which verifies that:
[_commit_](b799c3a0ef)
- every package which imports logrus, also has a log.go file, except the
excluded packages.
- the `package` field of each log.go variable, has the correct path. (to
detect when we move a package or change it's name)
- I pushed a commit with a manually changed log.go file to trigger the
ci check failure and it worked.
3. Alter the logging system to read the prefix from this `package` field
for every log while outputing:
[_commit_](b0c7f1146c)
- some packages have/want/need a different log prefix than their package
name (like `kv`). This can be solved by keeping a map of package paths
to prefix names somewhere.
---
**Some notes:**
- Please review everything carefully.
- I created the `prefixReplacement` map and populated the data that I
deemed necessary. Please check it and complain if something doesn't make
sense or is missing. I attached at the bottom, the list of all the
packages that used to use a different name than their package name as
their prefix.
- I have chosen to mark some packages to be excluded from this whole
process. They will either not log anything, or log without a prefix, or
log using their previously defined prefix. See the list of exclusions in
the bottom.
- I fixed all the tests that failed because of this change. These were
failing because they were expecting the old prefix to be in the
generated logs. I have changed those to expect the new `package` field
instead. This might not be a great solution. Ideally we might want to
remove this from the tests so they only test for relevant fields in the
logs. but this is a problem for another day.
- Please run the node with this config, and mention if you see something
weird in the logs. (use different verbosities)
- The CI workflow uses a script that basically runs the
`hack/gen-logs.sh` and checks that the git diff is zero. that script is
`hack/check-logs.sh`. This means that if one runs this script locally,
it will not actually _check_ anything, rather than just regenerate the
log.go files and fix any mistake. This might be confusing. Please
suggest solutions if you think it's a problem.
---
**A list of packages that used a different prefix than their package
names for their logs:**
- beacon-chain/cache/depositsnapshot/ package depositsnapshot, prefix
"cache"
- beacon-chain/core/transition/log.go — package transition, prefix
"state"
- beacon-chain/db/kv/log.go — package kv, prefix "db"
- beacon-chain/db/slasherkv/log.go — package slasherkv, prefix
"slasherdb"
- beacon-chain/db/pruner/pruner.go — package pruner, prefix "db-pruner"
- beacon-chain/light-client/log.go — package light_client, prefix
"light-client"
- beacon-chain/operations/attestations/log.go — package attestations,
prefix "pool/attestations"
- beacon-chain/operations/slashings/log.go — package slashings, prefix
"pool/slashings"
- beacon-chain/rpc/core/log.go — package core, prefix "rpc/core"
- beacon-chain/rpc/eth/beacon/log.go — package beacon, prefix
"rpc/beaconv1"
- beacon-chain/rpc/eth/validator/log.go — package validator, prefix
"beacon-api"
- beacon-chain/rpc/prysm/v1alpha1/beacon/log.go — package beacon, prefix
"rpc"
- beacon-chain/rpc/prysm/v1alpha1/validator/log.go — package validator,
prefix "rpc/validator"
- beacon-chain/state/stategen/log.go — package stategen, prefix
"state-gen"
- beacon-chain/sync/checkpoint/log.go — package checkpoint, prefix
"checkpoint-sync"
- beacon-chain/sync/initial-sync/log.go — package initialsync, prefix
"initial-sync"
- cmd/prysmctl/p2p/log.go — package p2p, prefix "prysmctl-p2p"
- config/features/log.go -- package features, prefix "flags"
- io/file/log.go — package file, prefix "fileutil"
- proto/prysm/v1alpha1/log.go — package eth, prefix "protobuf"
- validator/client/beacon-api/log.go — package beacon_api, prefix
"beacon-api"
- validator/db/kv/log.go — package kv, prefix "db"
- validator/db/filesystem/db.go — package filesystem, prefix "db"
- validator/keymanager/derived/log.go — package derived, prefix
"derived-keymanager"
- validator/keymanager/local/log.go — package local, prefix
"local-keymanager"
- validator/keymanager/remote-web3signer/log.go — package
remote_web3signer, prefix "remote-keymanager"
- validator/keymanager/remote-web3signer/internal/log.go — package
internal, prefix "remote-web3signer-
internal"
- beacon-chain/forkchoice/doubly... prefix is
"forkchoice-doublylinkedtree"
**List of excluded directories (their subdirectories are also
excluded):**
```
EXCLUDED_PATH_PREFIXES=(
"testing"
"validator/client/testutil"
"beacon-chain/p2p/testing"
"beacon-chain/rpc/eth/config"
"beacon-chain/rpc/prysm/v1alpha1/debug"
"tools"
"runtime"
"monitoring"
"io"
"cmd"
".well-known"
"changelog"
"hack"
"specrefs"
"third_party"
"bazel-out"
"bazel-bin"
"bazel-prysm"
"bazel-testlogs"
"build"
".github"
".jj"
".idea"
".vscode"
)
```
**What type of PR is this?**
Other
**What does this PR do? Why is it needed?**
This pull request modifies the `PULL_REQUEST_TEMPLATE.md` to ensure the
developer checked that their PR works as expected.
Some contributors push some changes, without even running the modified
client once to see if their changes work as expected.
Avoidable back-and-forth trips between the contributor and the reviewers
could be prevented thanks to running the modified client.
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description to this PR with sufficient context for
reviewers to understand this PR.
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Flatten spectests directory: Move all spectests to a single directory per network.
Commands ran:
```
cd testing/spectest/
```
Then for each network, I ran the following command twice.
```
find . -type f -name '*_test.go' -exec bash -c '
for file; do
dir=$(dirname "$file")
base=$(basename "$file" _test.go)
new_name="./${dir#./}__${base}_test.go"
git mv "$file" "$new_name"
done
' bash {} +
```
Then updated the packages with a command like
```
sed -i 's/package [a-zA-Z0-9_]\+/package mainnet/g' *.go
```
Updated commit from 5edadd7b to address @Kasey's feedback.
* Fix panic when checking types. String is not compatible with DeepEqual.
* Docs: add commentary on the filename convention
* Add a section about nightly tests to the spectest readme. Ref https://github.com/OffchainLabs/prysm/pull/15312
* Set shard_count to optimal value... one!
* Changelog fragment
* use latest unclog release
* Update spectest build instructions after #9122
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Update rules_go to v0.53.0
* Update staticcheck to v0.6.0
* Update to go 1.24.0
* Update github.com/trailofbits/go-mutexasserts to latest
* Use rules_go @ cf3c3af34bd869b864f5f2b98e2f41c2b220d6c9
* Provide the go binary to SszGen.
https://github.com/bazel-contrib/rules_go/pull/4173
* Unskip SA9003
* Update github ci checks to go1.24.0
* CI: Update gosec to v2.22.1 and golangci to v1..64.5
* Temporarily disable usetesting lint check for go1.24
* gosec: Disable G115 - integer overflow conversion
* gosec: Ignore G407 for "hardcoded" IV. It's not hardcoded.
* Fix uses of rand.Seed. This is a no-op in go1.24 and deprecated since go1.20.
* Changelog fragment
* PR to update workflow check dependencies
* Updated build checkout version to v4
* Updated to go 1.23.0
* Updated lint version to v1.60.3
* Revert to 1.22.3
* Updated go to 1.23
* revert
* Updated setup-go to v5
* Update lint to 1.60.2
* Revert changes
* Update Lint version to v1.60.3
* Update lint to go 1.23.0
* Update golanci.yml to 1.23.0
* Revert and keep to golang 1.22.4
* Disable mnd
* Downgrade to current version
* Add update to go 1.26
* Update to go 1.22.6
* Update .golangci.yml to 1.22.6
* Bump go version up
* Update to go 1.22 compatible version
* Fix NoSec declarations
* Skip Gosec in GolangCi
* Avoid Bug In Analyzer
* Add in Gohashtree and Update to 1.22.4
* Fix Go Sum
* Enable mirror linter and fix findings
* Use latest version of golangci-lint
* Use v1.55.2 instead of latest
---------
Co-authored-by: terence <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update staticcheck to latest
* Add static checks while ignoring for third party / external stuff
* Added a hack to keep go mod happy.
* disable SA2002
* Pin go mod tidy checker image to golang:1.20-alpine
* fixing squashing changes, migrates beacon , account, and auth endpoints on validator client
* adding accounts endpoints
* fixing tests and query endpoints
* adding auth endpoint and fixing unit tests
* removing unused files and updating node file to skip gRPC
* ineffectual assignment fix
* rolling back a change to fix e2e
* fixing issues with ui
* updating with webui version 2.0.5
* updating package name flag in readme
* removing restore assets functions
* adding nomemcopy flag to see if vulenerability scan passes
* making data non compressed to avoid copy vulnerability
* Update beacon-chain/rpc/eth/shared/structs_validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updating site_data, and skipping static analysis on file
* adding back deprecation comment notice
* updating workflows to ignore generated
* addressing radek comments
* missed a conversion
---------
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* add in github workflow for fuzzing that runs with cron
* every day
* go version
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* update github actions
* use quotes or it is go 1.2
lol
* Update gosec
* Update gosec
* Update go lint
* fix gosec violations
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
The comments of the template were not inside the braces and appeared when creating a new Issue.
Thanks
Co-authored-by: terencechain <terence@prysmaticlabs.com>
* Update go to 1.19.3
* Update other items to 1.19
* Update golangci-lint to latest release
* Run gofmt -s with go1.19
* Huge gofmt changes
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Enable gocognit linter
Currently the gocognit complexity threshold is set to 95 to make
sure no existing files will fail the linter. In future we will
reduce this threshold to a much lower one.
The recommended threshold is usually 30. Our code base has maximum
of 97 right now...But it's better late than never to pay attention
to our code compexity.
* Test to see github complains
* Resume to 97
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* github workflows: pin go version to 1.17
* Update go.yml
* Revert "Update go.yml"
This reverts commit 4a2d36d05d.
* pin golangci-lint
* try go 1.17
* Revert "Revert "Update go.yml""
This reverts commit 8a89663874.
* move and increase version of checkout
* attempt to ignore export path
* try with entrypoint only
* try some rearranging of stuff
* Split up jobs
* Use hack mentioned in https://github.com/securego/gosec/issues/469\#issuecomment-643823092
* Delete dappnode release trigger
* rm id
* try pin golangci-lint version
* try pin golangci-lint version
* Do not provide a specific go version and lets see what happens
* comment checkout, wtf is wrong with github actions
* it works locally...
* trying with some cache key for lint...
* Revert "trying with some cache key for lint..."
This reverts commit c4f5ae4495.
* try tellign it to skip go installation
* revert commented line, do something to satisify deepsource
* do something to satisify deepsource
* add changes
* fix CI
* wait here
* Fix gazelle with the correct directive for libp2p
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
