55fe85c887
**What type of PR is this?** Feature **What does this PR do? Why is it needed?** This PR allows devs to test against a specific run of the nightly reference test generator. **Acknowledgements** - [x] I have read [CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md). - [x] I have included a uniquely named [changelog fragment file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd). - [x] I have added a description with sufficient context for reviewers to understand this PR. - [x] I have tested that my changes work as expected and I added a testing plan to the PR description (if applicable).
Spec Tests
Spec testing vectors: https://github.com/ethereum/consensus-spec-tests
To run all spectests:
bazel test //... --test_tag_filters=spectest
Adding new tests
New tests must adhere to the following filename convention:
{mainnet/minimal/general}/$fork__$package__$test_test.go
An example test is the phase0 epoch processing test for effective balance updates. This test has a spectest path of {mainnet, minimal}/phase0/epoch_processing/effective_balance_updates/pyspec_tests.
There are tests for mainnet and minimal config, so for each config we will add a file by the name of phase0__epoch_processing__effective_balance_updates_test.go since the fork is phase0, the package is epoch_processing, and the test is effective_balance_updates.
Running nightly spectests
Since PR 15312, Prysm has support to download "nightly" spectests from github via a starlark rule configuration by environment variable.
Set --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly or --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly-<run_id> when running spectest to download the "nightly" spectests.
Note: A GITHUB_TOKEN environment variable is required to be set. The github token does not need to be associated with your main account; it can be from a "burner account". And the token does not need to be a fine-grained token; it can be a classic token.
bazel test //... --test_tag_filters=spectest --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly
bazel test //... --test_tag_filters=spectest --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly-21422848633