Commit Graph

9992 Commits

Author SHA1 Message Date
Potuz
96df81d5c5 Hook to slot stream instead of block stream on the VC (#13327)
* Hook to slot stream instead of block stream on the VC

* Implement StreamSlots in the BN

* mock update

* fix tests

* don't return from stream

* Terence's review

* deepsource second complain

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2023-12-13 23:13:56 +00:00
terence
c47c52152b Enhance Pruning Logs (#13331)
* Log prunning info

* Added start log

* Log prunning info

* Added start log
2023-12-13 19:24:47 +00:00
james-prysm
4cbe144a6c CLI: fixing account import ux bugs (#13328)
* fixing account import checking wallet twice, and adding sub folder search with a depth of 2

* removing uneeded check

* fixing unit test

* adding reset cache to fix potential flake

* improving test based on feedback
2023-12-13 17:11:32 +00:00
Justin Traglia
52b9b65adb Add sanity checks for bundle from builder (#13319)
* Add sanity checks for bundle from builder

* Add more checks to BlobsBundle.ToProto()

* Fix minor typo

* Fix tests & add new ones

* Add tests for ToProto

* Add "not" to error message

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-13 15:54:00 +00:00
Radosław Kapka
ea59b1ec71 Increase buffer of events channel (#13329) 2023-12-13 15:37:45 +00:00
Radosław Kapka
175c484c44 Uncomment e2e flakiness (#13326) 2023-12-13 12:50:13 +00:00
Nishant Das
8aaab86987 fix it (#13325) 2023-12-13 11:01:01 +00:00
Preston Van Loon
381116a3e8 Fix missing testnet versions. Issue #13288 (#13323) 2023-12-12 21:44:14 +00:00
Sammy Rosso
3d61fd0436 Blob filesystem add pruning during blob write (#13275)
* Add prune during write

* Fix merge errors

* Add test

* Add test timeout

* Gaz

* Check prune at midpoint

* Fix slot number

* More checks
2023-12-12 21:27:15 +00:00
james-prysm
b19d24c581 Remove signed block requirement from no-verify functions (#13314)
* removing fake wrappers

* fixing conficts and missed tests

* fixing more conflicts

* addressing missed unit test

* fixing nogo error

* fixing more unit tests

* fixing more tests
2023-12-12 20:18:40 +00:00
Radosław Kapka
8387088a52 Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-12 19:46:19 +00:00
Potuz
ce7452c97a update spectests to 1.4.0-beta.5 (#13318)
* update spectests to 1.4.0-beta.5

* add spec config
2023-12-12 18:27:48 +00:00
james-prysm
5e56b5fdd7 Beacon APIs: re enabling blob events (#13315)
* re enabling blob events

* terence's comments

* Update beacon-chain/rpc/eth/events/events_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-12 15:58:11 +00:00
Nishant Das
bfaba378f6 activate deneb (#13311) 2023-12-12 08:04:55 +00:00
Sammy Rosso
3bd116db16 Blob filesystem add pruning at startup (#13253)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Add pruning at startup

* Add nil check

* Fix merge errors

* Fix test

* Fix test

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-11 17:08:52 +00:00
terence
7d2ddaee43 Test improvement TestValidateVoluntaryExit_ValidExit (#13313) 2023-12-11 08:01:56 +00:00
terence
122a7782ff Initialize blob storage for initial sync service (#13312) 2023-12-11 07:52:07 +00:00
terence
9b1b6f9be6 Use verified blob for gossip checks (#13294)
* Use blob verifier for gossip rules

* Fixing tests

* Fix lint

* Mocks

* Trying Kasey's rec

* mock verifier init workaround

* Add more tests

* Reset deneb epoch for exit test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-11 00:37:45 +00:00
Delweng
0eb08a4f96 beacon-chain/rpc: use BalanceAtIndex instead of Balances to reduce memory copy (#13279)
* beacon-chain/rpc: use BalanceAtIndex instead of Balances

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: stream use BalanceAtIndex is sufficient

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: fix commit review

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: http2 -> httputil

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-10 00:04:45 +00:00
Radosław Kapka
bdfa06ed65 Simplify post-evaluation in Beacon API evaluator (#13309)
* Simplify post-evaluation in Beacon API evaluator

* revert slottime

* remove unused error
2023-12-09 07:54:31 +00:00
Manu NALEPA
a94f2b93e3 filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
* `filterAndCacheActiveKeys`: Add test cases

- Validator is in unknown status (to be fitered out)
- Validator is in pending status,
   with activation period > current period (to be filtered out)
- Validator is in pending status,
   with activation period == current period (to be kept)

* `filterAndCacheActiveKeys`: Keep exiting keys

Initially:
-------
If a validator is in exiting (so, with status==EXITING != ACTIVE) state,
it will be filtered out by the `filterAndCacheActiveKeys` function.
The validator won't be registered to the beacon node.

If this exiting validator has to propose a block:
- the block will be proposed using local block building only.
- the fee recipient will be the one set in the beacon node.

(Additionally, if the beacon node Prysm without any
fee recipient defined at the beacon node level, the fee recipient
will default on the `0x00000...` burn address.)

This commit modifies the `filterAndCacheActiveKeys` function
by stopping filtering out exiting validators.
2023-12-09 07:53:08 +00:00
Radosław Kapka
4c47756aed HTTP endpoints cleanup (#13251)
* remove validation package

* structs cleanup

* merge with apimiddleware removal

* more validation and Bls capitalization

* builder test fix

* use strconv for uint->str conversions

* use DecodeHexWithLength

* use exact param names

* rename http package to httputil

* change conversions to fmt.Sprintf

* handle query paramsd and route variables

* spans and receiver name

* split structs, move bytes helper

* missing ok check

* fix reference to indexed failure

* errors fixup

* add godoc to helper

* fix BLS casing and chainhead ref

* review

* fix import in tests

* gzl
2023-12-08 20:37:20 +00:00
Nishant Das
440841d565 only run it in the middle of an epoch (#13303) 2023-12-08 15:14:01 +00:00
Preston Van Loon
ff99616833 Fix staticcheck violations (#13301)
* Fix violations of sa2002

* Fix violations of sa4005

* Fix violations of sa4010

* Fix violations for sa4023

* Comment on commented static checks
2023-12-08 13:07:52 +00:00
Preston Van Loon
f537a98fcd Add staticchecks to bazel builds (#13298)
* Update staticcheck to latest

* Add static checks while ignoring for third party / external stuff

* Added a hack to keep go mod happy.

* disable SA2002

* Pin go mod tidy checker image to golang:1.20-alpine
2023-12-08 05:42:55 +00:00
Radosław Kapka
cee38660c7 Gracefully handle unknown validator index in the REST VC (#13296)
* Gracefully handle unknown validaor index in the REST VC

* add apostrophes
2023-12-08 04:30:50 +00:00
james-prysm
481d77bfde APIs: reusing grpc cors middleware for rest (#13284)
* reusing grpc cors middleware for rest

* addressing radek's comments

* Update api/server/middleware.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing to recommended name

* fixing naming

* fixing rename on test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-08 04:24:18 +00:00
Nishant Das
590317553c Support New Subnet Backbone (#13179)
* add in changes

* fix it up

* fix test

* gaz

* lint

* add back

* fix tests

* fix it

* fix tests

* add lib

* fix it
2023-12-08 04:07:48 +00:00
terence
68b7d1009e Update README.md (#13302) 2023-12-08 04:07:10 +00:00
james-prysm
b5b8825cc8 Beacon API: fix get blob returns 500 instead of empty (#13297)
* fix blob api, should return empty if no indicies were found

* fixing small bug with slice
2023-12-07 22:33:26 +00:00
Justin Traglia
382b8b23c2 Ensure partial blob is deleted if there's an error (#13292)
* Ensure partial blob is deleted if there's an error

* Add debug log if file is removed
2023-12-07 20:52:16 +00:00
kasey
40a3ebab91 initialize sig cache for verification.Initializer (#13295)
* initialize sig cache for verification.Initializer

* gaz

* lint

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 20:14:01 +00:00
james-prysm
83af9a5694 Beacon API: update Deneb endpoints after removing blob signing (#13235)
* making needed changes to beacon API based on removal of blobsidecar from block contents

* fixing tests and reverting some changes to be addressed later

* fixing generated code from protos

* gaz

* fixing get blob handler and adding blob storage to the blob service

* updating unit tests

* WIP

* wip tests

* got tests passing but needs cleanup

* removing gomod and gosum changes

* fixing more tests

* fixing more tests

* fixing more tests

* gaz

* moving some proto types around

* removing unneeded unit test

* fixing proposer paths

* adding more tests

* fixing more tests

* improving more unit tests

* updating one blob only unit test

* changing arguments of buildBlobSidecar

* reverting a change based on feedback

* terence's review items

* fixing test based on new develop changes

* radek's comments

* addressed more comments from radek

* adding in blobs to test data

* fixing casing in test

* removing extra line

* fixing issue from bad merge

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* moving core getblob business logic to blocker based on radek's comment

* fixing mock blocker

* gaz

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-07 17:37:11 +00:00
Nishant Das
6a45323ab7 only run metrics for canonical blocks (#13289) 2023-12-07 11:03:23 +00:00
kasey
4008ea736f Verify roblobs (#13245)
* scaffolding for verification package

* WIP blob verification methods

* lock wrapper for safer forkchoice sharing

* more solid cache and verification designs; adding tests

* more test coverage, adding missing cache files

* clearer func name

* remove forkchoice borrower (it's in another PR)

* revert temporary interface experiment

* lint

* nishant feedback

* add comments with spec text to all verifications

* some comments on public methods

* invert confusing verification name

* deep source

* remove cache from ProposerCache + gaz

* more consistently early return on error paths

* messed up the test with the wrong config value

* terence naming feedback

* tests on BeginsAt

* lint

* deep source...

* name errors after failure, not expectation

* deep sooource

* check len()==0 instead of nil so empty lists work

* update test for EIP-7044

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 02:36:25 +00:00
Justin Traglia
4e4fb9ad52 Split blob pruning into two funcs (#13285) 2023-12-06 23:39:02 +00:00
kasey
737e0e0d3a Use functional options for --blob-retention-epochs (#13283)
* blob retention period functional opts

* missed unstaged change

* missed other init after cleardb

* fix ineffassign

* fix dup import

* config failsafe for tests

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-06 20:20:34 +00:00
Potuz
604c82626f Allow requests for old target roots (#13281) 2023-12-06 17:11:34 +00:00
Nishant Das
e1a3852f08 push up the defaults (#13278) 2023-12-06 16:46:46 +00:00
Preston Van Loon
a40cc40edf CI: Add merge queue events trigger for github workflows (#13282) 2023-12-06 16:13:13 +00:00
james-prysm
0e3c1d42f6 Beacon API: routes unit test (#13276)
* refactoring to add a routes unit test for rest handlers

* gaz

* updating names for functions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 15:17:13 +00:00
Radosław Kapka
f41e603e5a Simplify Beacon API evaluator (#13265)
* Simplify Beacon API evaluator

* remove comment

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 14:54:33 +00:00
Radosław Kapka
28c3330375 Don't fetch duties for unknown keys (#13269)
* Don't fetch duties for unknown keys

* test
2023-12-06 14:18:45 +00:00
Nishant Das
cb465086e8 Fix Optimistic Sync Evaluator (#13262)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 13:41:47 +00:00
Nishant Das
cdd7958739 Fix Domain Data Caching (#13263)
* fix domain data caching

* fix locking of domain data

* preston's review
2023-12-06 12:04:19 +00:00
Nishant Das
97a522827b Bump Up Gossip Queue Size (#13277) 2023-12-06 09:49:26 +00:00
Preston Van Loon
b84b795f23 Relax file permissions check on existing directories (#13274) 2023-12-05 19:39:45 -06:00
Sammy Rosso
f40b8583f7 Blob filesystem: prune blobs (#13147)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Fix linter errors

* Fix retentionSlot calculation

* Move + use MaxEpochsToPersistBlobs

* Remove unused ctx

* Prestons suggestion

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Rename

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-05 21:07:34 +00:00
Preston Van Loon
a6b6a938de blobstorage: Improve mkdirall error (#13271) 2023-12-05 19:57:08 +00:00
Brandon Liu
c78d698d89 Add --jwt-id flag (#13218)
* add jwt-id flag

* optimize unit test for jwt-id

* Add jwt-id to help text

* gofmt

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-05 19:02:25 +00:00