9b626864f0
* Flatten spectests directory: Move all spectests to a single directory per network.
Commands ran:
```
cd testing/spectest/
```
Then for each network, I ran the following command twice.
```
find . -type f -name '*_test.go' -exec bash -c '
for file; do
dir=$(dirname "$file")
base=$(basename "$file" _test.go)
new_name="./${dir#./}__${base}_test.go"
git mv "$file" "$new_name"
done
' bash {} +
```
Then updated the packages with a command like
```
sed -i 's/package [a-zA-Z0-9_]\+/package mainnet/g' *.go
```
Updated commit from 5edadd7b to address @Kasey's feedback.
* Fix panic when checking types. String is not compatible with DeepEqual.
* Docs: add commentary on the filename convention
* Add a section about nightly tests to the spectest readme. Ref https://github.com/OffchainLabs/prysm/pull/15312
* Set shard_count to optimal value... one!
* Changelog fragment
* use latest unclog release
* Update spectest build instructions after #9122
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
1.5 KiB
Spec Tests
Spec testing vectors: https://github.com/ethereum/consensus-spec-tests
To run all spectests:
bazel test //... --test_tag_filters=spectest
Adding new tests
New tests must adhere to the following filename convention:
{mainnet/minimal/general}/$fork__$package__$test_test.go
An example test is the phase0 epoch processing test for effective balance updates. This test has a spectest path of {mainnet, minimal}/phase0/epoch_processing/effective_balance_updates/pyspec_tests.
There are tests for mainnet and minimal config, so for each config we will add a file by the name of phase0__epoch_processing__effective_balance_updates_test.go since the fork is phase0, the package is epoch_processing, and the test is effective_balance_updates.
Running nightly spectests
Since PR 15312, Prysm has support to download "nightly" spectests from github via a starlark rule configuration by environment variable.
Set --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly when running spectest to download the "nightly" spectests.
Note: A GITHUB_TOKEN environment variable is required to be set. The github token must be a fine grained token.
bazel test //... --test_tag_filters=spectest --repo_env=CONSENSUS_SPEC_TESTS_VERSION=nightly