Commit Graph

8681 Commits

Author SHA1 Message Date
Joel Rousseau
f2f10e7381 add tests for beaconcha baseURL (#13308)
* add tests for different environments

* sort import

* remove goerli for prater

* clean

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-13 13:44:51 +00:00
Nishant Das
9aa55b47f3 Clean Up Deposit Processing Logs (#13611)
* add it

* increase it
2024-02-13 10:41:07 +00:00
Preston Van Loon
c4c6b47d9b validator/exit: Print other testnet beaconcha.in URLs (#13610) 2024-02-12 19:36:58 +00:00
Manu NALEPA
06a5548424 Slasher: Fixes double votes false positive and attester slashings duplication. (#13596)
* `Test_processAttestations`: Remove duplicated tests.

* Sort indexed attestations by data root.

* `processAttestations`: Don't return duplicate slashings anymore.

Fix https://github.com/prysmaticlabs/prysm/issues/13592.

* `AttesterDoubleVote`: Rename fields.

* Detect double votes in different batches.

In order to do that:
1. Each attestation of the batch is tested against the other attestations of the batch.
2. Each attestation of the batch is tested against the content of the database.
2. Attestations are saved into the database.

Fixes https://github.com/prysmaticlabs/prysm/issues/13590.
2024-02-12 17:35:22 +00:00
terence
5e74c798d4 Use Gwei for value in get header log (#13608) 2024-02-12 15:14:58 +00:00
Nishant Das
7b955c94ec Reduce Lookahead Steps Parameter (#13599)
* reduce lookahead steps

* test
2024-02-12 14:39:42 +00:00
terence
256a05bfd5 Remove deprecated flags: optional engine and registration (#13606) 2024-02-10 04:59:00 +00:00
Preston Van Loon
03068ba781 db: clear blobs when using --clear-db or --force-clear-db (#13605)
* Call Close() as part of ClearDB

* Add method to clear blob storage

* Clear blob storage when clearing DB
2024-02-10 03:52:30 +00:00
Nishant Das
5df8b83a05 Validate Range Availibility (#13587)
* fix it

* check range avail

* add test cases

* add checks

* kasey's review

* gaz
2024-02-09 23:41:08 +00:00
terence
db653b8863 Remove deprecated build block parallel flag (#13539) 2024-02-09 22:13:51 +00:00
Manu NALEPA
af203efa0c Slasher: Refactor and add tests (#13589)
* `helpers.go`: Improve naming consistency.

* `detect_attestations.go`: Improve readability.

* `receive.go`: Add `attsQueueSize` in log message.

* `checkSlashableAttestations`: Improve logging.

`avgBatchProcessingTime` is not displayed any more if not batch is
processed.

* `loadChunks`: Use explicit `chunkKind` and `chunkIndices`.

* `getChunk`: Use specific `chunkIndex` and `chunkKind`.

* `validatorIndicesInChunk` -> `validatorIndexesInChunk`.

* `epochUpdateForValidator`: Use explicit arguments.

* `getChunk`: Change order of arguments.

* `latestEpochWrittenForValidator`: Use `ok` parameter.

So the default value is not any more considered as the absence of
value.

* `applyAttestationForValidator`: Use explicit arguments.

* `updateSpans`: Use explicit arguments.

* `saveUpdatedChunks`: Use explicit arguments.

* `checkSurrounds`: Use explicit arguments.

We see here that, previously, in `checkSlashableAttestations`,
`checkSurrounds` was called with the default value of `slashertypes`: `MinSpan`.

Now, we set it expliciterly at `MinSpan`, which may explicit a bug.

* `epochUpdateForValidator`: Set modified by the function argument first.

* `applyAttestationForValidator`: Set mutated argument `chunksByChunkIdx`first.

* `applyAttestationForValidator`: Rename variables.

* `Test_processQueuedAttestations`: Fix test.

Two tests were actually exactly the same.

* `updateSpans`: Keep happy path in the outer scope.

Even if in this case the "happy" path means slashing.

* `checkSurrounds`: Rename variable.

* `getChunk`: Avoid side effects.

It adds a few lines for callers, but it does not modify any more
arguments and it does what it says: getting a chunk.

* `CheckSlashable`: Flatten.

* `detect_attestations_test.go`: Simplify.

* `CheckSlashable`: Add error log in case of missing attestation.

* `processQueuedAttestations`: Extract a sub function.

So testing will be easier.

* `processAttesterSlashings` and `processProposerSlashings`: Improve.

* `processAttesterSlashings`: Return processed slashings.

* `createAttestationWrapper`: Rename variables.

* `signingRoot` ==> `headerRoot` or `dataRoot`.

Before this commit, there is two typse of `signing root`s floating around.
- The first one is a real signing root, aka a hash tree root computed from an object root and
a domain. This real signing root is the object ready to be signed.
- The second one is a "false" signing root, which is actually just the hash tree root of an object. This object is either the `Data` field of an attestation, or the `Header` field of a block.

Having 2 differents objects with the same name `signing root` is quite confusing.
This commit renames wrongly named `signing root` objects.

* `createAttestationWrapper` => `createAttestationWrapperEmptySig`.

So it's clear for the user that the created attestation wrapper has an empty signature.

* Implement `createAttestationWrapper`.

* `processAttestations`: Return processed attester slashings.

* Test `processAttestations` instead of `processQueuedAttestations`.

By testing `processAttestations` instead of `processQueuedAttestations`, we get rid of a lot of tests fixtures, including the 200 ms sleep.

The whole testing duration is shorter.

* `Test_processAttestations`: Allow multiple steps.

* `Test_processAttestations`: Add double steps tests.

Some new failing tests are commented with a corresponding github issue.

* `NextChunkStartEpoch`: Fix function comment.

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* `chunks.go`: Avoid templating log messages.

* `checkSlashableAttestations`: Simplify duration computation.

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-09 21:02:18 +00:00
terence
5582c558c6 Remove deprecated aggregate parallel flag (#13538) 2024-02-09 20:18:36 +00:00
terence
573d9739ea Remove disable vectorized htr (#13537) 2024-02-09 18:28:13 +00:00
terence
bb18fa3f71 Remove deprecated late reorg flag (#13536) 2024-02-09 17:15:08 +00:00
Aditya Asgaonkar
6a605e6b6d Fork choice filter changes (#13464)
* implement confirmation rule prerequisite - f.c. filter changes

* update tests

* update WORKSPACE for spec v1.4.0-beta.6

* run bazel gazelle

* Fix consensus_spec sha256

* drift also forkchoice time when drifting the service on tests

* update minimal kzg_commitment_inclusion_proof_depth.size

* fix mock engine client

* remove unnecessary helper & revert test changes

* revert change of proof size in minimal preset

* fix tests

* fix loader test

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2024-02-09 17:01:44 +00:00
Nishant Das
a0787e2379 Handle Syncing Executon Client (#13597)
* fix it

* manu's review

* fix failing tests
2024-02-09 09:07:48 +00:00
Nishant Das
621bda068d Suppress Unwanted P2P Errors (#13598)
* supress unwanted errors

* gaz

* mod
2024-02-09 05:13:57 +00:00
Radosław Kapka
91504eb95a Improve vc logs (#13573)
* duties

* atts

* revert some changes

* revert timeTillDuty

* Manu's review

* Revert "Auxiliary commit to revert individual files from 6806ca9fbe18101f58ccb40fe191c61c183735a8"

This reverts commit 0820c870d2627950179b0edf7ce62ee4fa4a03a3.

* remove trash

* more reivew

* making Manu happy

* test fixes
2024-02-08 18:24:03 +00:00
Sammy Rosso
5afb1255fe Add /eth/v1/beacon/deposit_snapshot endpoint (#13514)
* Add endpoint

* Uncomment in InitializeRoutes

* Add test

* Add 404

* Add more checks

* Test improvements

* Ssz

* Add ssz tags

* Add DepositSnapshot to bazel

* Fix tests

* Fix max size

* Resolve conflicts

* Revert untouched code

* Fix test + review

* Lint

* Oops

* Preston + Radek' review

* Only return 3 finalized roots

* Change to deposit contract depth

* Radek' review

* Gaz

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-02-07 22:53:08 +00:00
Manu NALEPA
9d6160e112 Slasher: Remove unused RPC. (#13594) 2024-02-07 21:11:58 +00:00
james-prysm
1383546999 Beacon API: get blob fix retention cases (#13585)
* fixing the handling for certain cases

* fixing tests

* Update beacon-chain/rpc/eth/blob/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update comment based on review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-06 22:31:17 +00:00
Justin Traglia
01116f7f82 Fix a few minor nits in protobuf definitions (#13512) 2024-02-06 21:17:32 +00:00
Thabokani
692ebd313f Fix typos in doc (#13583)
Signed-off-by: Thabokani <149070269+Thabokani@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
v4.2.2-rc.0
2024-02-06 10:18:21 +00:00
Nishant Das
6fa656c1ee Add Sync Checker (#13580)
* fix it

* add it in

* typo

* fix tests

* fix tests

* export and add test

* preston's review
2024-02-06 02:34:30 +00:00
Dhruv Bodani
55a29a4670 Implement beacon committee selections (#13503)
* implement beacon committee selections

* fix build

* fix lint

* fix lint

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* move beacon committee selection structs to validator module

* fix bazel build files

* add support for POST and GET endpoints for get state validators query

* add a handler to return error from beacon node

* move beacon committee selection to validator top-level module

* fix bazel

* re-arrange fields to fix lint

* fix TestServer_InitializeRoutes

* fix build and lint

* fix build and lint

* fix TestSubmitAggregateAndProof_Distributed

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-05 15:43:51 +00:00
Potuz
e2e7e84a96 Get the right head state when proposing a failed reorg (#13579)
* Get the right head state when proposing a failed reorg

* add unit test

* split logic
2024-02-05 13:40:35 +00:00
terence
91b0a93df7 Enhance EL block height log (#13582) 2024-02-05 01:52:01 +00:00
Preston Van Loon
8839015312 docker: Add coreutils to docker images (#13564)
* Add coreutils to docker images

* add coreutils dependencies

* Add a prysmaticlabs.com/uploads backup of the deb files

* Run gazelle and fix issues

* Remove broken tar, change http_archive deps to debian_archive, remove http mirrors in favor of snapshot

* Add comments about which deps are required by other deps
2024-02-03 19:21:21 +00:00
terence
61ab4bf7ca Rename block by range request log (#13561) 2024-02-03 19:20:04 +00:00
Radosław Kapka
e3ce1bde45 Move API structs to api module (#13577) 2024-02-03 11:57:01 +00:00
Nishant Das
9d1189b222 Do Not Cache For Non Active Public Keys (#13581)
* fix it

* clean up
2024-02-03 05:19:54 +00:00
KeienWang
74f5452a64 Fix typo in [beacon-chain/cache/depositsnapshot/deposit_cache_test.go]: Corrected a spelling error. (#13532)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-02-03 05:14:32 +00:00
Nishant Das
ea1204d3c7 Fix Slashing Gossip Checks (#13574)
* fix it

* add for proposals too
2024-02-02 23:13:22 +00:00
Radosław Kapka
d9ac69752b Return consensus block value in Wei (#13575)
* Return consensus block value in Wei

* Return consensus block value in Wei

* review
2024-02-02 18:17:40 +00:00
terence
52af63f25a Revise blob sidecar not found log (#13571)
* Update blob sidecar not found log

* Use fields
2024-02-01 20:48:59 +00:00
james-prysm
2dad245bc8 handle slice out of range (#13568)
* handle slice out of range

* adding some tests
2024-02-01 16:59:40 +00:00
Potuz
9a9990605c Update Gohashtree to v0.0.4-beta (#13569)
* Update Gohashtree to v0.0.4-beta

* go mod tidy

* go mod tidy

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-01 15:42:56 +00:00
james-prysm
2cddb5ca86 fixing jwt auth checks (#13565) 2024-02-01 15:13:52 +00:00
Nishant Das
73ce28c356 make it the default (#13556) 2024-01-31 10:27:26 +00:00
Manu NALEPA
7a294e861e Beacon node slasher improvement (#13549)
* Slasher: Ensure all gorouting are stopped before running `Stop` actions.

Fixes #13550.
In tests, `exitChan` are now useless since waitgroup are used to wait
for all goroutines to be stopped.

* `slasher.go`: Add comments and rename some variables. - NFC

* `detect_blocks.go`: Improve. - NFC

- Rename some variables.
- Add comments.
- Use second element of `range` when possible.

* `chunks.go`: Remove `_`receivers. - NFC

* `validateAttestationIntegrity`: Improve documentation. - NFC

* `filterAttestations`: Avoid `else`and rename variable. - NFC

* `slasher.go`: Fix and add comments.

* `SaveAttestationRecordsForValidators`: Remove unused code.

* `LastEpochWrittenForValidators`: Name variables consistently. - NFC

Avoid mixes between `indice(s)`and `index(es)`.

* `SaveLastEpochsWrittenForValidators`: Name variables consistently. - NFC

* `CheckAttesterDoubleVotes`: Rename variables and add comments. - NFC

* `schema.go`: Add comments. - NFC

* `processQueuedAttestations`: Add comments. - NFC

* `checkDoubleVotes`: Rename variable. - NFC

* `Test_processQueuedAttestations`: Ensure there is no error log.

* `shouldNotBeSlashable` => `shouldBeSlashable`

* `Test_processQueuedAttestations`: Add 2 test cases:
- Same target with different signing roots
- Same target with same signing roots

* `checkDoubleVotesOnDisk` ==> `checkDoubleVotes`.

Before this commit, `checkDoubleVotes` did two tasks:
- Checking if there are any slashable double votes in the input
  list of attestations with respect to each other.
- Checking if there are any slashable double votes in the input
  list of attestations with respect to our database.

However, `checkDoubleVotes` is called only in
`checkSlashableAttestations`.

And `checkSlashableAttestations` is called only in:
- `processQueuedAttestations`, and in
- `IsSlashableAttestation`

Study of case `processQueuedAttestations`:
---------------------------------------------
In `processQueuedAttestations`, `checkSlashableAttestations`
is ALWAYS called after
`Database.SaveAttestationRecordsForValidators`.

It means that, when calling `checkSlashableAttestations`,
`validAtts` are ALREADY stored in the DB.

Each attestation of `validAtts` will be checked twice:
- Against the other attestations of `validAtts` (the portion of
  deleted code)
- Against the content of the database.

One of those two checks is redundent.
==> We can remove the check against other attestations in `validAtts`.

Study of case `Database.SaveAttestationRecordsForValidators`:
----------------------------------------------------------------
In `Database.SaveAttestationRecordsForValidators`,
`checkSlashableAttestations` is ALWAYS called with a list of
attestations containing only ONE attestation.

This only attestaion will be checked twice:
- Against itself, and an attestation cannot conflict with itself.
- Against the content of the database.

==> We can remove the check against other attestations in `validAtts`.

=========================

In both cases, we showed that we can remove the check of attestation
against the content of `validAtts`, and the corresponding test
`Test_checkDoubleVotes_SlashableInputAttestations`.

* `Test_processQueuedBlocks_DetectsDoubleProposals`: Wrap proposals.

So we can add new proposals later.

* Fix slasher multiple proposals false negative.

If a first batch of blocks is sent with:
- validator 1 - slot 4 - signing root 1
- validator 1 - slot 5 - signing root 1

Then, if a second batch of blocks is sent with:
- validator 1 - slot 4 - signing root 2

Because we have two blocks proposed by the same validator (1) and for
the same slot (4), but with two different signing roots (1 and 2), the
validator 1 should be slashed.

This is not the case before this commit.
A new test case has been added as well to check this.

Fixes #13551

* `params.go`: Change comments. - NFC

* `CheckSlashable`: Keep the happy path without indentation.

* `detectAllAttesterSlashings` => `checkSurrounds`.

* Update beacon-chain/db/slasherkv/slasher.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update beacon-chain/db/slasherkv/slasher.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* `CheckAttesterDoubleVotes`: Keep happy path without indentation.

Well, even if, in our case, "happy path" mean slashing.

* 'SaveAttestationRecordsForValidators': Save the first attestation.

In case of multiple votes, arbitrarily save the first attestation.
Saving the first one in particular has no functional impact,
since in any case all attestations will be tested against
the content of the database. So all but the first one will be
detected as slashable.

However, saving the first one and not an other one let us not
to modify the end to end tests, since they expect the first one
to be saved in the database.

* Rename `min` => `minimum`.

Not to conflict with the new `min` built-in function.

* `couldNotSaveSlashableAtt` ==> `couldNotCheckSlashableAtt`

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-01-31 09:49:14 +00:00
james-prysm
258123341e add a log and update size for promptui (#13542) 2024-01-30 17:19:31 +00:00
Preston Van Loon
224b136737 Revert "set limit to multiple of burst for goerli" (#13552)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-30 06:10:12 +00:00
Nishant Das
3ed4866eec Makes Our New Deposit Trie The Default (#13555)
* make 4881 the default

* fix failed build
2024-01-30 05:15:52 +00:00
kasey
373c853d17 set limit to multiple of burst for goerli (#13544)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-01-27 22:12:08 +00:00
terence
23b0718b5f Add metric for data availability wait time (#13534)
* Add metric for data availability wait time

* Kasey's feedback

* Kasey's feedback
2024-01-26 18:17:25 +00:00
terence
3a9854145c Correct metrics from ns to ms (#13540) 2024-01-26 17:43:30 +00:00
Radosław Kapka
1b70d2b566 Fetch unaggregated atts in GetAggregateAttestation (#13533) 2024-01-26 17:08:58 +00:00
Nishant Das
59b310a221 make it the same (#13531) v4.2.1 v4.2.1-rc.3 2024-01-26 05:35:27 +00:00
Nishant Das
22b6d1751d Enable Backfill in E2E (#13524)
* enable backfill for devmode

* enable backfill

* gaz

* move to its own package

* fix panic

* fix bug

* gaz

* kasey's review
2024-01-26 04:37:41 +00:00
Potuz
9c13d47f4c fix off by one (#13529) v4.2.1-rc.2 2024-01-26 00:05:56 +00:00