Compare commits

...

1007 Commits

Author SHA1 Message Date
kasey
3fa6d3bd9d update to latest version of our fastssz fork (#14519)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-10-08 19:24:50 +00:00
Rupam Dey
56f0eb1437 feat: add Electra support to light client functions (#14506)
* add Electra to switch case in light client functions

* replace `!=` with `<` in `blockToLightClientHeaderXXX`

* add Electra tests

* update `CHANGELOG.md`

* add constant for Electra

* add constant to `minimal.go`

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-10-08 18:13:13 +00:00
Bastin
7fc5c714a1 Light Client consensus types (#14518)
* protos/SSZ

* interfaces

* types

* cleanup/dedup

* changelog

* use createBranch in headers

* error handling

---------

Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
2024-10-08 17:07:56 +00:00
james-prysm
cfbfccb203 Fix: validator status cache does not clear upon key removal (#14504)
* adding fix and unit test

* Update validator.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* fixing linting

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-10-07 14:35:10 +00:00
Potuz
884b663455 Move back ConvertKzgCommitmentToVersionedHash to primitives package (#14508)
* Move back ConvertKzgCommitmentToVersionedHash to primitives package

* Changelog
2024-10-07 14:33:23 +00:00
Radosław Kapka
0f1d16c599 Flip committee aware packing flag (#14507)
* Flip committee aware packing flag

* changelog

* fix deprecated flag name

* test fixes

* properly use feature in tests

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_attestations_test.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-10-07 13:36:10 +00:00
kasey
c11e3392d4 SSE implementation that sheds stuck clients (#14413)
* sse implementation that sheds stuck clients

* Radek and James feedback

* Refactor event streamer code for readability

* less-flaky test signaling

* test case where queue fills; fixes

* add changelog entry

* james and preston feedback

* swap our Subscription interface with an alias

* event.Data can be nil for the payload attr event

* deepsource

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-10-04 21:18:17 +00:00
Radosław Kapka
f498463843 Register deposit snapshot endpoint (#14503) 2024-10-04 16:42:30 +00:00
Radosław Kapka
cf4ffc97e2 Update block Beacon APIs to Electra (#14488)
* Update block Beacon APIs to Electra

* CHANGELOG

* Revert "Auxiliary commit to revert individual files from 9bf238279a696dbcd65440606b0e3173f3be5e05"

This reverts commit a7ef57a2532f9ee02831d180926f7b84f5104a2b.

* review

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-10-04 08:21:08 +00:00
Bastin
3824e8a463 Make block_to_lightclient_header_XXX functions private (#14502)
* make block to lightclient header functions private

* changelog

* fix `helpers.go`

---------

Co-authored-by: Rupam Dey <117000803+rupam-04@users.noreply.github.com>
Co-authored-by: rupam-04 <rpmdey2004@gmail.com>
2024-10-03 21:59:07 +00:00
Rupam Dey
21ca4e008f Update lc functions to use the dev branch of CL specs (#14471)
* create finalized header based on finalized block version

* changelog entry

* pass attested block from handlers

* fix core tests

* add test for attested header exectution fields

* changelog entry

* remove unused functions

* update `createLightClientBootstrapXXX` functions

* fix for getBootstrapAltair

* fix for getBootstrapAltair

* fix tests for Capella and Deneb

* update `CHANGELOG.md`

* remove Electra from `createLightClientBootstrap` switch case

* update dependencies

* minor fixes

* remove unnecessary comments

* replace `!reflect.DeepEqual` with `!=`

* replace `%s` with `%#x` for `[32]byte`

---------

Co-authored-by: Inspector-Butters <mohamadbastin@gmail.com>
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
2024-10-03 20:52:21 +00:00
Bastin
6af44a1466 Fix lc execution header bug (#14468)
* create finalized header based on finalized block version

* changelog entry

* pass attested block from handlers

* fix core tests

* add test for attested header exectution fields

* changelog entry

* remove unused functions

* Update beacon-chain/core/light-client/lightclient.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update beacon-chain/core/light-client/lightclient.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update beacon-chain/core/light-client/lightclient.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* remove finalized header from default update

* remove unused functions

* bazel deps

---------

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
2024-10-03 17:29:22 +00:00
Owen
2e29164582 allow users to publish blobs (#14442)
* allow users to publish blobs

Allowing users to publish blobs before publishing blocks, gives the blobs a head start. They can begin to propagate around the network while the block is being validated.

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

* Update beacon-chain/rpc/prysm/beacon/handlers.go

---------

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
2024-10-01 20:13:41 +00:00
Ferran Borreguero
6d499bc9fc Enable electra interop genesis (#14465)
* Add Electra hard fork to interop genesis

* Update Changelog

* Fix develop merge

* gazelle

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-10-01 14:35:36 +00:00
Bastin
7786cb5684 create lc finalized header based on finalized block version (#14457)
* create finalized header based on finalized block version

* changelog entry

* refactor tests
2024-09-30 16:38:51 +00:00
terence
003b70c34b Update sepolia bootnodes (#14485)
* Add sepolia bootnodes

* Change log
2024-09-26 23:28:36 +00:00
Preston Van Loon
71edf96c7d Add sepolia config.yaml check with eth-clients/sepolia (#14484)
* Add sepolia config.yaml check with eth-clients/sepolia

* Update CHANGELOG.md
2024-09-26 13:26:12 +00:00
Potuz
ddafedc268 Implement consensus-specs/3875 (#14458)
* WIP

- beacon-chain builds

* pass blockchain tests

* pass beacon-chain/execution tests

* Passing RPC tests

* fix building

* add changelog

* fix linters

* Spectests

* copy requests on Copy()

* Fix tests

* Fix config test

* fix verification tests

* add aliases for Electra types

* double import and unskip spectests

* Remove unnecessary comment
2024-09-25 17:06:52 +00:00
Nishant Das
7e5738bfcd Update Pubsub Library To Use Gossipsub 1.2 (#14428)
* Update Libp2p Packages

* Update CHANGELOG

* Finally Fix All Tests

* Fix Build

* Fix Build

* Fix TestP2P Connection Initialization

* Fix TestP2P Host Options

* Fix Test By Removing WaitGroup
2024-09-24 16:15:19 +00:00
Preston Van Loon
315c05b351 async/event: Use geth's implementation of event/feed.go (#14362)
* Use geth's event

* run gazelle
2024-09-23 19:57:04 +00:00
Jun Song
3662cf6009 Add Electra case for GetBeaconStateV2 (#14466)
* Add Electra case for GetBeaconStateV2

* Add CHANGELOG.md entry

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-09-20 21:20:13 +00:00
james-prysm
98d8b50b0e deflake pushsettings test - custom pattern matcher (#14453)
* adding custom matcher

* changelog

* removing unneeded map

* fixing merge conflict
2024-09-20 21:15:47 +00:00
Jun Song
1a1cc25bd1 Deprecate pb.gw.go files (#14464)
* Clean pb.gw.go files

* Deprecate pb.gw.go file support
2024-09-20 17:18:17 +00:00
james-prysm
bc9c7193a9 FIX: removing extra container for ssz blob sidecar response (#14451)
* removing extra container for ssz blob sidecar response

* gaz and changelog

* gaz

* fixing dependencies

* fixing test

* updating values based on feedback"
2024-09-16 14:28:47 +00:00
Radosław Kapka
7ac3c01b5b Refactor light client functions (#14434)
* Use correct types in light client functions

* conversions

* more refactoring

* test fixes

* changelog

* error fix

* revert test changes

* revert test skip

* Update api/server/structs/conversions_lightclient.go

Co-authored-by: Rupam Dey <117000803+rupam-04@users.noreply.github.com>

* use BlockToLightClientHeader

* reviewer suggestion

* Revert "use BlockToLightClientHeader"

This reverts commit f3df56ded5.

---------

Co-authored-by: Rupam Dey <117000803+rupam-04@users.noreply.github.com>
2024-09-13 22:08:28 +00:00
Bastin
ed6f69e868 Upgrade LightClient DB (#14432)
* add kv tests for capella and deneb

* test execution fields

* Update proto/eth/v2/custom.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update proto/eth/v2/custom.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* Update proto/eth/v2/custom.go

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* refactor tests using deepEqual

---------

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-09-13 03:53:02 +00:00
Preston Van Loon
222b360c66 Electra: Remove signing domain for consolidations (#14437)
* Electra: Remove signing domain for consolidations. See https://github.com/ethereum/consensus-specs/pull/3915

* Update changelog
2024-09-13 02:00:41 +00:00
Sammy Rosso
170a864239 Otel migration (#14424)
* remove opencensus

* gaz

* update dependencies

* add missing dependencies

* fix test?

* Fix note relevance

* add otel http transport middleware

* gaz

* tidy up

* gaz

* changelog

* feedback

* gaz

* fix merge issues
2024-09-12 23:00:20 +00:00
Rupam Dey
b5cfd0d35d feat: implement block_to_light_client_header (#14433)
* implement `block_to_light_client_header` upto Deneb

* update `CHANGELOG.md`

* refactor: remove unnecessary variables

* move functions to `core/light-client/lightclient.go`

* feat: add tests

* refactors

* add blinded beacon block support to tests

* revert "add blinded beacon block support to tests"

* add tests for blinded beacon block

* lint

* refactor: move common code outside of if-else

* fix dependencies
2024-09-12 17:53:51 +00:00
hopinheimer
28181710b0 version bump on k8s io client-go and apimachinery (#14444)
* version bump on k8s io client-go and apimachinery

* bazel file served

* fixing build issues

* some changes in noops functions

* Update CHANGELOG.md

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2024-09-12 14:59:43 +00:00
terence
2f756b7ec4 Refactor logging proposed block (#14443) 2024-09-12 14:08:10 +00:00
terence
df4ca54a76 Remove unused blobs bundle cache and usages (#14438) 2024-09-12 01:28:22 +00:00
james-prysm
875e3e5e7d E2E: fixing builder route registration after gorilla mux removal (#14445)
* fixing builder route registration afte gorilla mux removal

* changelog
2024-09-11 23:15:08 +00:00
Md Amaan
a5317f8117 Replaced mux with http.Servemux (#14416)
* Replaced mux with http.Servmux

* updated change log

* james suggestions

* lint

* lint fix 2

* passed middlewares from validatorclient

* gazelle fix

* fixed issue

* added middlewares field to rpc config

* suggestions applied

* updated godoc

* fixed TestCors

* refactor

* godoc added

* cli code removed and lint fixed

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-09-11 19:39:05 +00:00
Potuz
38b92c0171 Avoid allocating in loops (#14439)
* Avoid allocating in loops

* change process slashings

* add changelog

* Kasey's review

* Only marked trie dirty if changed
2024-09-11 17:58:07 +00:00
Potuz
a03b34af77 Use read only validators on ApplyToEveryValidator (#14426)
* Use read only validators on ApplyToEveryValidator

* Use ReadFromEveryValidator on slashing

* change changelog

* Revert "Use ReadFromEveryValidator on slashing"

This reverts commit 74c055bddb56e0573075c71df8a40f1c6a9bfdfd.
2024-09-10 11:34:42 +00:00
james-prysm
4c14bd8be2 looking at ways to reduce validator registration calls (#14371)
* looking at ways to reduce validator registration calls

* small mistake, should be epoch start

* adding more optimizations for reducing registration calls while covering more edgecases

* linting

* adding change log and force full push override

* fixing bug and adding tests

* changing if statement just to be safe

* potuz feedback for easier readability

* more review feedback for simplicity

* more review suggestions from potuz

* fix unit test

* reduce redundancy

* Update CHANGELOG.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* small nitpick

* fixing typo

* updating logs

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-09-09 16:30:41 +00:00
Radosław Kapka
62b8e63a0a Compare with nil before invoking IsNil() on an interface (#14431)
* Compare with nil before invoking `IsNil()` on an interface

* changelog

* review
2024-09-06 19:39:33 +00:00
Rupam Dey
eec3b0b7fe feat: introduce Capella and Deneb full-node.md lc changes (#14376)
* feat: introduce Capella and Deneb `full-node.md` lc changes

* add switch-case and replace `[][]byte` with `[][]string`

* return version name in http header

* populate header and use `interfaces.ReadOnlyBeaconBlock`

* fix lint

* merge cases in switch case and replace `interfaces.ExecutionData` with `*ExecutionPayloadHeader`

* minor fixes

* refactor `createLightClientBootstrapCapella` and `createLightClientBootstrapDeneb`

* use lightclientheader instead of different versions

* fix failing `TestLightClientHandler_GetLightClientBootstrap` tests

* fix lint

* refactor handlers

* refactor handlers more

* refactor handlers even more

* create conversions_lightclient

* fix lint errors

* add deneb and capella proto headers

* update lightclientbootstrap proto struct to capella&deneb

* update usecases

* update usecases

* resolve panic in header.GetBeacon

* fix spacings

* refactor core/lightclient.go

* fix isBetterUpdate

* use errors.wrap instead of fmt.errorf

* changelog entry

* fix lint errors

* fix api structs to use json rawMessage

* inline unmarshal

* remove redundant nil check

* revert remove redundant nil check

* return error in newLightClientUpdateToJSON

* inline getExecutionData

* better error handling

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Inspector-Butters <mohamadbastin@gmail.com>
Co-authored-by: Bastin <43618253+Inspector-Butters@users.noreply.github.com>
2024-09-06 17:06:31 +00:00
james-prysm
2bffb83a00 gateway flag changes breaking release e2e (#14418)
* e2e release test breaks due to changes in flag naming

* changing approach to fix

* adding some small alias test

* fixing test, changelog, and flag name

* Update config_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-09-05 21:19:56 +00:00
james-prysm
45fd3eb1bf gRPC Gateway Removal (#14089)
* wip passing e2e

* reverting temp comment

* remove unneeded comments

* fixing merge errors

* fixing more bugs from merge

* fixing test

* WIP moving code around and fixing tests

* unused linting

* gaz

* temp removing these tests as we need placeholder/wrapper APIs for them with the removal of the gateway

* attempting to remove dependencies to gRPC gateway , 1 mroe left in deps.bzl

* renaming flags and other gateway services to http

* goimport

* fixing deepsource

* git mv

* Update validator/package/validator.yaml

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/package/validator.yaml

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

* missed lint

* renaming import

* reversal based on feedback

* fixing web ui registration

* don't require mux handler

* gaz

* removing gRPC service from validator completely, merged with http service, renames are a work in progress

* updating go.sum

* linting

* trailing white space

* realized there was more cleanup i could do with code reuse

* adding wrapper for routes

* reverting version

* fixing dependencies from merging develop

* gaz

* fixing unit test

* fixing dependencies

* reverting unit test

* fixing conflict

* updating change log

* Update log.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* gaz

* Update api/server/httprest/server.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* addressing some feedback

* forgot to remove deprecated flag in usage

* gofmt

* fixing test

* fixing deepsource issue

* moving deprecated flag and adding timeout handler

* missed removal of a flag

* fixing test:

* Update CHANGELOG.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

* updating comments based on feedback

* removing unused field for now, we can add it back in if we need to use the option

* removing unused struct

* changing api-timeout flag based on feedback

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-09-04 15:40:31 +00:00
terence
963a1b4cb7 Fix electra balance update (#14410) 2024-09-03 14:41:40 +00:00
Rupam Dey
77c845043d fix: make some places use sync committee period instead of epoch (#14406)
* fix: make some places use sync committee period instead of epoch

* add fix details to `CHANGELOG.md`

* update `CHANGELOG.md` comment
2024-09-02 16:44:08 +00:00
Brandon Liu
342bb0fcef Add value for MaxBuilderEpochMissedSlots (#14334)
* add value for MaxBuilderEpochMissedSlots

* make usage for MaxBuilderEpochMisedSlots more friendly

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-08-30 15:08:50 +00:00
Preston Van Loon
93e6bd7929 Flip --enable-experimental-state to opt-out. (#14398) 2024-08-29 20:42:32 +00:00
Bastin
3015eea4e3 Fix lightclient header (#14389)
* change LCUpdate to use LCHeader

* fix api struct usages

* fix api struct finalized_header

* add lightclientheader to proto structs

* fix proto usages

* fix proto usages in events

* fix uppercase field in protobuf defenition

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-29 15:57:21 +00:00
Nishant Das
2f42f7e313 Add a Tracing Wrapper Package (#14207)
* Adds a wrapper package

* Gazelle

* Add in Empty Span

* Revert It Back

* Add back reference

* Set It As Empty

* fix missing import

* remove redundant alias

* remove unused

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Saolyn <sammy@rosso.red>
2024-08-27 20:00:53 +00:00
Preston Van Loon
a7c86a6d1b Update CONTRIBUTING.md to highlight no trivial changes (#14384) 2024-08-26 18:05:24 +00:00
Md Amaan
2399451869 Hardcoded GenesisValidatorsRoot (#14365)
* hardcoded GenesisValidatorsRoot

* added in mainnet.config

* updated desc

* added it in all testnets

* minor change

* added roots instead of empty and fn to compute byte32 from hex

* added in e2e testnet_config

* fixed test

* minor fix

* removed fn and added bytes output directly

* Add test for genesis validator root mainnet value

* removed root from minimal and testnet

* removed root

* Update CHANGELOG.md

* Fix bazel package visiblity

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
2024-08-26 16:50:29 +00:00
Taranpreet26311
19d9a1915d PR to update workflow check dependencies (#14379)
* PR to update workflow check dependencies

* Updated build checkout version to v4

* Updated to go 1.23.0

* Updated lint version to v1.60.3

* Revert to 1.22.3

* Updated go to 1.23

* revert

* Updated setup-go to v5

* Update lint to 1.60.2

* Revert changes

* Update Lint version to v1.60.3

* Update lint to go 1.23.0

* Update golanci.yml to 1.23.0

* Revert and keep to golang 1.22.4

* Disable mnd

* Downgrade to current version

* Add update to go 1.26

* Update to go 1.22.6

* Update .golangci.yml to 1.22.6
2024-08-26 16:25:38 +00:00
james-prysm
261921ae4c reduce validator registration logs (#14370)
* reduce validator registration logs

* reverting a log change that's probably better as warn

* Update CHANGELOG.md

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
2024-08-23 16:47:47 +00:00
Sammy Rosso
6ad8a104dd Re-use duplicate code in aggregator (#14342)
* move shared duplicate code

* rename function
2024-08-23 16:38:49 +00:00
Jay
50e53265a1 bugfix : Removed the default value of the bootnode flag to prevent it from being overridden during testnet usage (#14357)
* Removed the default value of the bootnode flag to prevent it from being overridden during testnet usage

* bugfix for checking stringslice flag to use isSet
2024-08-23 16:28:26 +00:00
Preston Van Loon
3392fdb21d Add CHANGELOG.md and update CONTRIBUTING.md (#13673)
* Add github action requiring a diff to changelog.md.

* Update CONTRIBUTING.md with CHANGELOG.md requirements and restrictions on trivial PRs

* Update tj-actions/changed-files

* Backfill CHANGELOG.md

* Remove hashtag references to PRs

* Append PR template

* Add unreleased changes

* Update CONTRIBUTING.md

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-23 15:11:06 +00:00
Rupam Dey
dd3c9652c3 fix: replace BeaconBlockHeader in createLightClientBootstrap with LightClientHeader (#14374)
* fix: replace `BeaconBlockHeader` in `createLightClientBootstrap` with `LightClientHeader`

* minor fix in `handlers_test.go`

* check if `beacon` is `nil` instead of `header`

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-22 17:13:10 +00:00
Bastin
022a53f8f2 create light-client-updates bucket (#14266)
* create light-client-updates bucket

* Electra committe validation for aggregate and proof (#14317)

* Electra committe validation for aggregate and proof

* review

* update comments

* Refactor get local payload (#14327)

* Refactor get local payload

* Fix go lint: new line

* add lightclient db kv functions

* lightclient db tests

* move blockchain/lightclient.go to core/light-client package

* add comparison check for start and end period

* create testing/utils/lightcilent.go

* lightclient db tests

* fix imports and usages

* fix imports and usages in process_block_helpers

* fix bazel dependencies

* remove unnecessary nil check

* add more tests for lightclient kv functions

* refactor tests

* refactor kv.LightClientUpdates

* fix db to return every update that is available in the requested range

* run gazzele fix

* return empty map in case of empty db

* fix goimports errors

* goimports

* Revert "Auxiliary commit to revert individual files from aa7ce6f37cb6767cf11642b022b2ce59d42ae621"

This reverts commit 33c707f5bd164386449dc14ff27d95ad5f195161.

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence <terence@prysmaticlabs.com>
Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
2024-08-22 16:00:18 +00:00
Preston Van Loon
2fa3547644 Update spectests to v1.5.0-alpha.5. Copied from #14352 (#14368) 2024-08-22 14:03:48 +00:00
Rupam Dey
7c213ce161 feat: implement PayloadProof function (#14356)
* feat: implement function `PayloadProof` to calculate proof of execution payload

* remove comments

* feat: implement function to compute field roots of

* feat: implement function to compute `BeaconBlock` field roots and add tests

* fix dependencies

* check if interface implements the assserted type

* fix: lint

* replace `ok != true` with `!ok`

* remove unused parameter from `PayloadProof`

* remove test and move `PayloadProof` to `blocks/proofs.go`

* remove `PayloadProof` from `fieldtrie`

* replace `fieldtrie.ProofFromMerkleLayers` with `trie.ProofFromMerkleLayers`

* Update container/trie/sparse_merkle.go

* update dependencies

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
2024-08-21 16:04:35 +00:00
Radosław Kapka
ed3d7d49ec Update block publishing Beacon APIs to Electra (#14361)
* Update block publishing Beacon APIs to Electra

* linter
2024-08-20 16:57:08 +00:00
Potuz
068139a78a remove unused function (#14360) 2024-08-19 16:20:57 +00:00
Nishant Das
8dd7361b6a Fix Gas Limit in Genesis (#14359)
* Fix Gas Limit in Genesis

* Comment

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-08-19 14:29:49 +00:00
Radosław Kapka
41ea1d230a Electra API struct conversions (#14339)
* Electra API conversions

* reduce function complexity

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-08-19 11:46:56 +00:00
John
9e25026519 Fix error handling to use os.IsNotExist (#14315)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-16 20:18:16 +00:00
Jun Song
9e9559df60 docs: Add consolidation_requests_root on comment (#14335)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-16 19:46:00 +00:00
kira
7a5a6c7e54 fix a deep source error (#14345)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-16 19:06:00 +00:00
Nishant Das
be317c439d Allow Block With Blobs To Be Proposed in Electra (#14353)
* Allow blobs in Electra

* Add Test

* Add Error for unknown request type

* Fix Test

* fix tests

* clean up logic

---------

Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
2024-08-16 15:39:15 +00:00
terence
f43383a3fb Proposer checks gas limit before accepting builder's bid (#14311)
* Proposer checks gas limit before accepting builder's bid

* James feedback

* Use cache
2024-08-15 21:03:11 +00:00
terence
22f6f787e1 Update spec tests to v1.5.0-alpha.4 (#14340)
* Update spec tests to v1.5.0-alpha.4

* Add Fix for Off By 1 In Consolidations

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2024-08-15 05:59:41 +00:00
james-prysm
44b3986025 removing skip from test (#14343) 2024-08-14 16:33:01 +00:00
james-prysm
6cb845660a fixing electra attestation inconsistencies (#14331)
* fixing electra attestation inconsistencies

* adding dependencies

* removing helper circular dependency

* adding error

* simplifying function

* improving get committee index function

* fixing more instances of GetData().committeeIndex and fixing tests

* Update proto/prysm/v1alpha1/attestation.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback and fixing linting

* removing unused functions and associated tests

* fixing test error checks

* removing helpers.VerifyAttestationBitfieldLengths to reduce beaconCommitteeFromState calls

* small optimizations

* fixing linting

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-14 15:24:53 +00:00
Radosław Kapka
de2c866707 Keep read lock in BeaconState.getValidatorIndex (#14341) 2024-08-14 13:09:29 +00:00
Rupam Dey
74ddb84e0a feat: implement `ComputeFieldRootsForBlockBody` function (#14278)
* feat: (WIP)implement ``ComputeFieldRootsForBlockBody`` function to compute roots of fields in BlockBody

* calculate field roots upto ``deposits``

* add some required constants fix some errors

* implement ``ComputeFieldRootsForBlockBody`` function for all fields in ``BeaconBlockBody``

* populate `fieldRoots` based on block body version

* fix constants

* `bazel run //:gazelle -- fix`

* remove nested `if`s

* formatting 1

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* formatting 2

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* remove unrequired check

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* update naming 1

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* update function name

* add test for Phase0 block body

* update Phase0 test

* update phase0 test without setting any fields

* fix some errors in Phase0 test

* don't set fields

* add altair test

* add tests upto Electra

* fix the function for deneb and newer forks

* update dependencies

* add checks to ensure interface implements the asserted type

---------

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-14 12:46:07 +00:00
Sammy Rosso
0c6a068fd5 Remove identical beacon state (#14338)
* rename + delete

* remove ref
2024-08-14 10:16:19 +00:00
Jun Song
fad92472d8 fix(tests): Fix v1alpha1/node/server_test.go (#14321)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-08-13 08:25:43 +00:00
james-prysm
2a44e8e6ec test util functions for electra field generation (#14320)
* adding some util functions for electra field generation

* linting

* adding missed linting
2024-08-12 15:36:03 +00:00
terence
e3d27f29c7 Refactor get local payload (#14327)
* Refactor get local payload

* Fix go lint: new line
2024-08-09 17:04:43 +00:00
Radosław Kapka
e011f05403 Electra committe validation for aggregate and proof (#14317)
* Electra committe validation for aggregate and proof

* review

* update comments
2024-08-08 16:32:19 +00:00
Potuz
b8cd77945d Move slasher handling down the pipeline (#14322) 2024-08-08 13:51:40 +00:00
Radosław Kapka
9a7f521f8a Fix state upgrade log (#14316) 2024-08-07 15:32:54 +00:00
Preston Van Loon
102f94f914 Clean up: Deduplicate fork spec tests pasta (#14289)
* Deduplicate code in spectest operations

Deduplicate code in blockchain core packages.

* Deduplicate code in blockchain core packages.
2024-08-07 15:16:37 +00:00
james-prysm
0c0a497651 engine_getPayloadBodiesByRangeV1 - fix, adding hexutil encoding on request parameters (#14314)
* adding hexutil encoding on request parameters

* fix for test

* fixing more tests

* deepsource fix
2024-08-07 15:05:24 +00:00
Sammy Rosso
e0785a8939 HTTP endpoint for GetValidatorActiveSetChanges (#14264)
* add GetValidatorActiveSetChanges

* fix linter

* fix errors

* James' review

* use stater

* fix merge conflict errors

* remove validator from func names

* rename util funcs
2024-08-06 11:56:43 +00:00
Potuz
af098e737e Check locally for min-bid and min-bid-difference (#14205)
* Check locally for min-bid and min-bid-difference

* fix tests

* Terence's fix

Co-authored-by: terence <terence@prysmaticlabs.com>

* Terence's review

* fix tests

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
2024-08-05 14:20:26 +00:00
Potuz
1e4ede5585 Reduce cognitive complexity of ReceiveBlock (#14296)
* Reduce cognitive complexity of ReceiveBlock

* Remove stray line

Co-authored-by: terence <terence@prysmaticlabs.com>

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
2024-08-05 12:44:49 +00:00
Sammy Rosso
fb2620364a HTTP endpoint for GetChainHead (#14262)
* add getChainHead endpoint

* James' review

* Radek' review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-05 08:49:37 +00:00
Ryan
68b38b6666 remove broken link (#14291)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-02 21:36:09 +00:00
Jun Song
ff3e0856a1 fix(tests): Correct misleading variable names and expressions in test files (#14292)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-02 21:32:53 +00:00
Potuz
85f334b663 Return parent root from forkchoice (#14293) 2024-08-02 14:36:37 +00:00
Potuz
10f520accb process Electra operations in the right order (#14294)
* process Electra operations in the right order

* godoc
2024-08-02 14:30:52 +00:00
Sammy Rosso
836608537e HTTP endpoint for GetValidatorParticipation (#14261)
* add endpoint

* remove canonicalFetcher

* Add replayerBuilder to coreService

* fix endpoint template

* fix string query params

* gaz

* fix linter

* test fix

* Radek' review

* remove unused request struct + gaz

* linter

* gaz

---------

Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-08-02 13:55:17 +00:00
Radosław Kapka
13e09c58f6 Committee-aware attestation packing (#14245)
* initial algorithm

* ready for review

* feature flag

* typo

* review

* comment fix

* fix TestProposer_sort_DifferentCommittees

* flag usage
2024-08-02 11:53:43 +00:00
Potuz
600ca08aa8 Use the beacon block root to verify the prestate (#14290) 2024-08-01 20:22:37 +00:00
james-prysm
0ed74b3c4a removing explicit deadlines on push proposer settings (#14285)
* removing deadlines

* fixing mock interface implementation
2024-07-31 18:28:50 +00:00
Potuz
7c69a9aa1c Refactor CommitteeAssignments to get all committees at once (#14284) 2024-07-31 16:01:19 +00:00
Potuz
c50cfb044a Get all beacon committees at once (#14282)
* Get all beacon committees at once

* Update beacon-chain/core/helpers/beacon_committee.go

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-31 12:52:23 +00:00
ethfanWilliam
38d4e179ba chore: fix unfound image (#14167)
* Remove Feature Flag From Prater (#12082)

* Use Epoch boundary cache to retrieve balances (#12083)

* Use Epoch boundary cache to retrieve balances

* save boundary states before inserting to forkchoice

* move up last block save

* remove boundary checks on balances

* fix ordering

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* chore: fix unfound image

* fix link

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-31 12:10:18 +00:00
shangchengbabaiban
be80728320 chore: fix some comments (#14270)
Signed-off-by: shangchengbabaiban <shuang.cui@live.cn>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-31 11:06:15 +00:00
james-prysm
09028033c0 adding corrected returns under handle error (#14280) 2024-07-30 22:02:25 +00:00
james-prysm
52c036c3ab missed adding electra process epoch on state replay - devnet 1 (#14272)
* missed adding electra process epoch on state replay

* abstracting common functionality as to not forget in processing

* removing unnessesary error check

* breaking out core process slots logic out

* fixing small bug in epoch processing

* adding small log to fix linting and make use of logrus in the package

* adding some unit tests for process epoch

* potuz's review feedback

* reversing processing order based on feedback
2024-07-30 20:52:55 +00:00
james-prysm
2fc7cdeba7 Push proposer settings every slot (#14155)
* test push settings every slot

* updating comments

* changing design a little bit based on feedback

* adding corrected deadline
2024-07-30 19:30:56 +00:00
james-prysm
6f7976766d moving cloner functions to beacon_block.go (#14265)
* moving cloner functions for beacon block and adding fuzzing tests

* fixing test
2024-07-29 15:13:21 +00:00
Samuel VIDEAU
5c369361b0 Fix Event stream with carriage return support (#14250)
* fix: event stream with carriage return

* lint: bazel

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-07-24 21:13:17 +00:00
terence
7d48b45152 Proposer: filter attestations for invalid signature (#14225)
* Proposer: filter attestations for invalid signature

* Nishant's feedback

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Add more checks based on potuz's feedback

* Fix filter by fork choice conditions

* Potuz's feedback

* Pass in epoch

* Potuz feedback 2

* Fix test

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-07-23 23:19:14 +00:00
John
345aabe996 Fix: Adjust fieldRefCount for capella, deneb, electra (#14252)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-23 22:02:48 +00:00
terence
5d04b36680 Remove no deposit requests found debug log (#14257) 2024-07-23 20:40:19 +00:00
Rupam Dey
cd8907f76c feat: implement `is_better_update` for light client (#14186)
* feat: implement ``is_better_update`` for light client

* replace ``!=`` with ``!reflect.DeepEqual``

* chore: import constant instead of redeclaring and some minor fixes

* feat: add test for ``newHasSupermajority != oldHasSupermajority``

* chore: capitalise first letter of function names

* refactor: rewrite test setups in a more readable manner

* two test cases

* feat: add tests for ``IsBetterUpdate``

* refactor: remove double import

* wording fix 1

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* wording fix 2

Co-authored-by: Radosław Kapka <radoslaw.kapka@gmail.com>

* minor fixes

* fix: change function names to lowercase and fix dependencies

---------

Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-23 19:16:53 +00:00
james-prysm
b108d5bf54 moving cloners to beacon_block.go and adding fuzzing (#14255)
* moving beacon block fields from cloners to individual files and adding fuzzing tests

* adding missed tests

* fixing tests:

* removing deep not equals for now

* adding clarifying comment
2024-07-23 16:42:35 +00:00
james-prysm
4d823acf45 moving cloners to attestation.go and adding fuzzing (#14254)
* moving cloners to attestation.go and adding fuzzing

* fixing bazel

* fixing build
2024-07-22 22:31:12 +00:00
james-prysm
aa868e5e8c remove eip6110flag for validator index cache (#14173)
* using fork instead of eip6110flag for validator index cache

* updating validator indicies on initialize from poroto unsafe

* fixing bad version check

* attempting to fix unit test

* reverting test changes

* changing incorrect initialization

* updating comment

* radek's comment'
2024-07-22 14:39:48 +00:00
james-prysm
b1be6cd20b creating execution payload copy and fuzz tests (#14246)
* creating execution payload copy and fuzz tests

* fixing build

* removing unused code
2024-07-22 13:34:42 +00:00
stellrust
fd9321f6ba chore: fix comment for struct field (#14218)
Signed-off-by: stellrust <gohunter@foxmail.com>
Co-authored-by: terence <terence@prysmaticlabs.com>
2024-07-22 12:41:27 +00:00
james-prysm
8364226b68 payload electra cloning (#14239)
* poc for payload electra cloning

* partial fixes

* fixing build

* addressing kasey's comment

* forgot to unexport interface

* making test more generic

* making fuzzing slightly more robust

* renaming based on kasey's comment

* using fuzz test in same package to avoid exporting interface

* fixing build
2024-07-19 17:07:30 +00:00
Radosław Kapka
49055acf81 EIP-7549: Attestation packing (#14238)
* EIP-7549: Attestation packing

* new files

* change var name

* test fixes

* enhance comment

* unit test for Deneb state
2024-07-19 14:08:39 +00:00
Sammy Rosso
57ffc12f17 HTTP endpoint GetIndividualVotes (#14198)
* add http endpoint

* add tests

* Gaz

* Add pointers

* add endpoint to test

* Electra: EIP-7251 Update `process_voluntary_exit` (#14176)

* Electra: EIP-7251 Update `process_voluntary_exit`

* Add unit test for VerifyExitAndSignature EIP-7251

* @potuz peer feedback

* Avoid Cloning When Creating a New Gossip Message (#14201)

* Add Current Changes

* add back check

* Avoid a Panic

* fix: Multiple network flags should prevent the BN to start (#14169)

* Implement Initial Logic

* Include check in main.go

* Add tests for multiple flags

* remove usage of append

* remove config/features dependency

* Move ValidateNetworkFlags to config/features

* Nit

* removed NetworkFlags from cmd

* remove usage of empty string literal

* add comment

* add flag validation to prysctl validator-exit

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* fix tests

* Radek' review + tests

* fix tests

* Radek' review

* forgot one

* almost forgot the tests

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: kira <shyampkira@gmail.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-19 12:23:36 +00:00
Sammy Rosso
d066480a51 GetValidatorPerformance empty body bug (#14240)
* fix possible empty body panic bug

* gaz

* fix test
2024-07-19 10:56:13 +00:00
terence
0e8f98b2a4 Remove electra todo for process deposit requests (#14243) 2024-07-19 04:29:53 +00:00
james-prysm
3a734f51e0 removing unused naming convention (#14241) 2024-07-18 19:40:06 +00:00
james-prysm
c7e2d709cf electra payload fix for devnet1 (#14235)
* fixing marshalling and adding more to unit test

* updating missed consolidation requests

* renaming variables

* adding test gen

* reverting config change
2024-07-18 14:25:54 +00:00
Preston Van Loon
8b4b3a269b Fix typo (#14233) 2024-07-17 20:52:36 +00:00
james-prysm
2f76ba542f adding consolidation requests to json serialization (#14229)
* adding consolidation requests to json serialization

* fixing test

* missed file checkin

* fixing unit tests
2024-07-17 13:24:15 +00:00
Radosław Kapka
637cbc88e8 Add missing pieces of Electra (#14227)
* Add missing pieces of Electra

* update mock
2024-07-16 19:33:40 +00:00
james-prysm
fadff022a0 Payload attributes misleading value fix (#14209)
* draft solution to corrected suggested fee recipient

* removing electra from this PR

* gaz

* adding test for endpoint

* gaz
2024-07-16 15:38:15 +00:00
Preston Van Loon
05784a6c28 Electra: Add minimal spectests for sync_committee_updates (#14224) 2024-07-15 20:47:31 +00:00
Preston Van Loon
5a48e002dd Unskip electra spectests (#14220) 2024-07-15 19:14:32 +00:00
Preston Van Loon
d6f86269a4 Electra: process_registry_updates handle exiting validators (#14221)
* Handle case where validator is already exited

* unit test for a slashed validator
2024-07-15 17:53:34 +00:00
Preston Van Loon
422438f515 Electra: ProcessConsolidationRequests (#14219) 2024-07-15 16:22:20 +00:00
Potuz
e5b25071f9 Fix effective balance updates in Electra (#14215) 2024-07-15 15:28:56 +00:00
terence
498ee635e1 Remove warning eip_7251.proto is unused (#14213) 2024-07-15 15:23:09 +00:00
Preston Van Loon
c238c00630 Electra: EIP-7251 implement process_withdrawal updates (#14181)
* Electra: EIP-7251 implement process_withdrawal updates

* Unit tests for new process_withdrawal logic in electra
2024-07-12 16:59:04 +00:00
Preston Van Loon
3eacc37831 Electra: Forkchoice spectest fix (#14180) 2024-07-12 16:50:29 +00:00
Potuz
5267b4b4d4 Fix Merkle proof generator for Electra (#14211) 2024-07-12 16:38:17 +00:00
Radosław Kapka
ec84a1b49c EIP-7549: Allow multiple committee bits (#14203) 2024-07-12 15:37:29 +00:00
james-prysm
afb7383225 adding electra version to payload attribute event (#14212) 2024-07-12 15:21:06 +00:00
Preston Van Loon
a00b40fa81 State: HasPendingBalanceToWithdraw (#14200)
* Implement HasPendingBalanceToWithdraw to improve the best / average case lookup

* Add tests for HasPendingBalanceToWithdraw
2024-07-11 20:31:08 +00:00
Radosław Kapka
365c6252ba EIP-7549: validator client (#14158)
* EIP-7549: validator client

* server code

* tests

* build fix

* review

* remove context

* Revert "Auxiliary commit to revert individual files from 16fed79a1ae0bbe4a08cb9819c2785d6e34958dd"

This reverts commit f59e1459f3f7561e0483bc8542110794951585c5.
2024-07-11 18:41:09 +00:00
kira
7c81c7da90 fix: Multiple network flags should prevent the BN to start (#14169)
* Implement Initial Logic

* Include check in main.go

* Add tests for multiple flags

* remove usage of append

* remove config/features dependency

* Move ValidateNetworkFlags to config/features

* Nit

* removed NetworkFlags from cmd

* remove usage of empty string literal

* add comment

* add flag validation to prysctl validator-exit

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-07-10 11:08:37 +00:00
Nishant Das
f8950c8c40 Avoid Cloning When Creating a New Gossip Message (#14201)
* Add Current Changes

* add back check

* Avoid a Panic
2024-07-10 05:51:41 +00:00
Preston Van Loon
18be899aef Electra: EIP-7251 Update process_voluntary_exit (#14176)
* Electra: EIP-7251 Update `process_voluntary_exit`

* Add unit test for VerifyExitAndSignature EIP-7251

* @potuz peer feedback
2024-07-09 20:13:48 +00:00
Nishant Das
9dc3b645c4 Optimize ProcessRegistryUpdates Further (#14197)
* Add it in

* Clean it Up
2024-07-09 12:41:07 +00:00
Potuz
318561999d copy just the pointer in the votes loop (#14196) 2024-07-08 23:47:29 +00:00
Preston Van Loon
ae5b0b4391 Electra: EIP-7251 update process_pending_balance_deposits (#14177)
* EIP-7251: Implement changes from https://github.com/ethereum/consensus-specs/pull/3776

* Unskip epoch processing spectest

* EIP-7251: unit tests for logic in https://github.com/ethereum/consensus-specs/pull/3776

* Radek feedback
2024-07-08 16:37:57 +00:00
Nishant Das
74b5f6ecf2 Add Aggregated Public Key Method (#14178)
* Add aggregated key method

* Gazelle

* Manu's Review
2024-07-08 09:28:41 +00:00
Nishant Das
aea2a469cc Update Libp2p (#14192) 2024-07-08 06:29:10 +00:00
Khanh Hoa
7a394062e1 refactor: enable errorlint and refactor code (#14110)
* refactor: enable errorlint and refactor code

* revert

* revert

* add bazel

* gofmt

* gofmt

* gofmt

* gofmt

* gci

* lint

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-04 22:40:13 +00:00
Manu NALEPA
8070fc8ece Fix slasher disk usage leak. (#14151)
* `PruneProposalsAtEpoch`: Test return value in case of nothing to prune.

* `TestStore_PruneAttestations_OK`: Create unique validator indexes.

Before this commit, `attester2` for `j = n` was the same than
`attester1` for `j = n + 1`, resulting in erasure of a lot of attesters.

I guess it was not the initial intent.

* Slasher pruning: Check if the number of pruned items corresponds to the expectation.

Before this commit, if the pruning function did remove a superset
of the expected pruned items (including all the items), then the test would pass.

* Prune items that should be pruned and stop pruning items that should not be pruned.

The first 8 bytes of the key of `attestation-data-roots` and
`proposal-records` bytes correspond respectively to an encoded epoch and
and encoded slot.

The important word in this sentence is "encoded".
Before this commit, these slot/epoch are SSZ encoded, which means that
they are little-endian encoded.

However:
- `uint64PrefixGreaterThan` uses `bytes.Compare` which expects
   big-endian encoded values.
- `for k, _ := c.First(); k != nil; k, _ = c.Next()` iters over the
  keys in big-endian order.

The consequence is:
- Some items that should be pruned are not pruned, provoking a disk
  usage leak.
- Some items that should not be pruned are pruned, provoking errors like
  in https://github.com/prysmaticlabs/prysm/issues/13658.

This commit encodes the slot/epoch as big-endian before storing them in
the database keys.

Why this bug has not been detected in unit test before?

The values used in unit tests before this commit in
`TestStore_PruneProposalsAtEpoch` and `TestStore_PruneAttestations_OK`
are `10` and `20`.
Unfortunately, checking if `littleEndian(20) > littlenEndien(10)`
with the `>` operator considering operands as big-endian encoded returns
the expected result...

Just replacing `20` by `30` trigs the bug.

* Make deepsource happy.

* Slasher: Migrate database from little-endian to big-endian.

* Update beacon-chain/slasher/service.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/db/slasherkv/migrate.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* `TestMigrate`: Fix documentation.

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-07-03 13:33:07 +00:00
Preston Van Loon
d6aeaf77b3 Electra: Unskip state-native tests for spectest v1.5.0-alpha.3 (#14174) 2024-07-02 19:33:27 +00:00
Preston Van Loon
40434ac209 Electra: Unskip passing spectests at v1.5.0-alpha.3 (#14175) 2024-07-02 19:25:19 +00:00
Preston Van Loon
0aab919d7c Electra: new process_consolidation_request (#14163)
* Electra: process_consolidation_request

* Enable consolidation_request spectests
2024-07-02 14:43:40 +00:00
Nishant Das
a8ecf5d118 Update to Go v1.22 (#13965)
* Bump go version up

* Update to go 1.22 compatible version

* Fix NoSec declarations

* Skip Gosec in GolangCi

* Avoid Bug In Analyzer

* Add in Gohashtree and Update to 1.22.4

* Fix Go Sum
2024-07-02 04:03:24 +00:00
kevaundray
8c0d6b27d0 use underscore to signify that it is not being used (#14162)
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-07-01 20:42:55 +00:00
Radosław Kapka
2e6f1de29a EIP-7549: Aggregation gRPC metods (#14115)
* impl

* tests

* generate mock

* review feedback

* gofmt
2024-07-01 18:40:33 +00:00
Gianguido Sorà
657750b803 validator/client: process Sync Committee roles separately (#13995)
* validator/client: process Sync Committee roles separately

In a DV context, to be compatible with the proposed selection endpoint, the VC must push all partial selections to it instead of just one.

Process sync committee role separately within the RolesAt method, so that partial selections can be pushed to the DV client appropriately, if configured.

* testing/util: re-add erroneously deleted state_test.go

* validator/client: fix tests

* validator/client: always process sync committee validator

Process sync committee duty at slot boundary as well.

* don't fail if validator is marked as sync committee but it is not in the list

ignore the duty and continue

* address code review comments

* fix build

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-07-01 17:18:45 +00:00
Preston Van Loon
af5eb82217 Electra: get_next_sync_committee_indices. (#14164) 2024-07-01 17:11:09 +00:00
terence
84e7f33fd9 Update to v1.5.0-alpha.3 spec tests (#14112)
* Update to v1.5.0-alpha.3 spec tests

* Fix deposit request

* Remove todo
2024-06-29 23:12:43 +00:00
Brindrajsinh Chauhan
ca83d29eef go version upgrade (#14161) 2024-06-29 23:08:16 +00:00
Afanti
0d49f6c142 chore: fix comments (#14149)
fix the typos in the code comments.
2024-06-29 21:03:15 +00:00
Preston Van Loon
041e81aad2 eip-7251: process_registry_updates (#14005)
* eip-7251: registry_updates

* Fix spectest violation

* More unit tests for process_registry_updates

* Update tests for ProcessRegistryUpdates

* Minor refactorings and benchmark

* Update beacon-chain/core/electra/registry_updates.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Wrap errors from process_registry_updates for easier debugging

* Rewrite process_registry_updates not to use st.Validators()

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-06-29 15:56:28 +00:00
james-prysm
028504ae9a EIP-6110: Supply validator deposits on chain (#13944)
* wip changes

* wip changes refactoring deposit functions

* wip on deposit functions

* wip changes to fix deposit processing

* more wip function logic

* gaz

* wip refactoring deposit changes

* removing circlular dependency and other small fix

* fixing circular dependencies

* fixing validators file

* fixing more tests

* fixing unit tests

* wip deposit packing

* refactoring packing

* changing packing code some more

* fixing packing change

* updating more tests

* removing comment

* fixing transition code for eip6110

* including inserts for validator index cache

* adding tests and placeholder test

* moving deposit related unit tests over

* adding in test for electra proposer packing

* spec test wip

* moving cache saving to the correct spot

* eip-6110: deposit spectests

* adding deposit receipt spec tests

* reverting altair operations in this pr and will be handled separately

* fixing renames but need to refactor process deposits

* gaz

* fixing names

* fixing linting

* fixing unit test

* fixing a test and updating test util

* bal never used

* fixing more tests

* fixing one more test

* addressing feedback

* refactoring process deposits to be part of their own fork instead of in blocks package

* adding new tests to cover functions and removing redundancies

* removing comment based on feedback

* resolving easy deepsource issue

* refactoring for appropriate aliasing and readability

* reverting some changes to simplify diff

* small edit to comment

* fixing linting

* fixing merge changes and review comments

* Update beacon-chain/rpc/prysm/v1alpha1/validator/proposer_deposits.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* partial review comments

* addressing slack feedback

* moving function from deposits to validator.go

* adding in batch verification and improving logs

---------

Co-authored-by: Preston Van Loon <preston@pvl.dev>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-06-29 01:53:20 +00:00
james-prysm
91c55c6880 Electra: Properly Calculate Proposer Probabilities for MaxEB (#14010)
* adding small change and tests for max eb spec change

* gaz

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-06-28 22:24:32 +00:00
Radosław Kapka
78cf75a0ed IndexedAtt wrapper for the slasher feed (#14150)
* `IndexedAtt` wrapper for the slasher feed

* test fixes

* fix simulator

* fix e2e

* Revert "Auxiliary commit to revert individual files from 191bbf77accfc2523fa9f909837a2e9dca132afa"

This reverts commit 2b0441a23a0e5f66e50cf36c3bbfbb39d587b17b.

* extract interface from channel
2024-06-28 15:44:19 +00:00
terence
fa370724f1 Increase attestation seen cache exp time to two epochs (#14156) 2024-06-28 00:20:08 +00:00
Barnabas Busa
5edc64d88c fix: update holesky repository layout (#14057)
* fix: update holesky repository layout

* Update holesky-testnet dependency

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
2024-06-27 19:36:24 +00:00
Preston Van Loon
7898e65d4e Electra v1.5.0-alpha.3 changes: Move consolidations from beacon block body to the execution payload header. (#14152) 2024-06-27 19:28:30 +00:00
vickkkkkyy
6d63dbe1af removed some internal expired jump targets (#14128) 2024-06-27 19:26:54 +00:00
Preston Van Loon
eab9daf5f5 process_registry_updates without a full copy of the validator set (#14130)
* Avoid copying validator set in ProcessRegistryUpdates

* Fix bug with sortIndices. Thanks to @terencechain for the expert debugging
2024-06-27 17:49:24 +00:00
Nishant Das
4becd7b375 Remove Backoff when Searching for a lot of non-viable peers. Affects peer performance (#14148) 2024-06-27 03:27:00 +00:00
james-prysm
f230a6af58 Finalized validator index cache saving (#14146)
* adding logic for saving validator indicies in new cache

* updating index cache for eip6110

* Update setters_misc.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* linting

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-06-26 20:58:28 +00:00
james-prysm
539b981ac3 Web3signer: persistent public keys (#13682)
* WIP

* broken and still wip

* more wip improving saving

* wip

* removing cyclic dependency

* gaz

* fixes

* fixing more tests and how files load

* fixing wallet tests

* fixing test

* updating keymanager tests

* improving how the web3signer keymanager works

* WIP

* updated keymanager to read from file

* gaz

* reuse readkeyfile function and add in duplicate keys check

* adding in locks to increase safety

* refactored how saving keys work, more tests needed:

* fix test

* fix tests

* adding unit tests and cleaning up locks

* fixing tests

* tests were not fixed properly

* removing unneeded files

* Update cmd/validator/accounts/wallet_utils.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* review feedback

* updating flags and e2e

* deepsource fix

* resolving feedback

* removing fatal test for now

* addressing manu's feedback

* gofmt

* fixing tests

* fixing unit tests

* more idomatic feedback

* updating log files

* updating based on preston's suggestion

* improving logs and event triggers

* addressing comments from manu

* truncating was not triggering key file reload

* fixing unit test

* removing wrong dependency

* fix another broken unit test

* fixing bad pathing on file

* handle errors in test

* fixing testdata dependency

* resolving deepsource and comment around logs

* removing unneeded buffer

* reworking ux of web3signer file, unit tests to come

* adding unit tests for file change retries

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* updating based on review feedback

* missed err check

* adding some aliases to make running easier

* Update validator/keymanager/remote-web3signer/log.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's review

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing more review feedback and linting

* fixing tests

* adding log

* adding 1 more test

* improving logs

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-06-26 15:36:32 +00:00
Khanh Hoa
aad29ff9fc refactor: use go ticker instead of timer (#14134)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-06-25 21:28:45 +00:00
james-prysm
4722446caf updating process deposit function to be more readable and fork specific, breaking out components of pr 13944 (#14139) 2024-06-25 18:19:42 +00:00
Radosław Kapka
b8aad84285 EIP-7549: attestation pool (#14121)
* implementation

* test fixes

* Electra tests

* remove aggregator tests

* id comments and tests

* make Id equal to [33]byte
2024-06-25 13:18:07 +00:00
Sammy Rosso
5f0d6074d6 Prysmctl ui bug (#14140)
* Fix ui bug

* Better logging
2024-06-25 13:13:42 +00:00
james-prysm
9d6a2f5390 Remove electra duplicate helpers (#14138)
* removing duplicate helper functions to reduce 6110 size

* linting
2024-06-24 21:22:46 +00:00
Preston Van Loon
490ddbf782 Enable golang.org/x/tools/go/analysis/errorsas static analysis check (#14135) 2024-06-24 17:20:45 +00:00
Radosław Kapka
adc875b20d EIP-7549: p2p and sync (#14085)
* EIP-7549: p2p and sync

* small cleanup

* fuzz fix

* deepsource

* review

* fix ineffectual assignment

* fix pubsub

* update ComputeSubnetForAttestation

* review

* review
2024-06-24 13:57:11 +00:00
kasey
8cd249c1c8 update codegen dep and cleanup organization (#14127)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-06-20 23:33:54 +00:00
Preston Van Loon
305d5850e7 ssz: Move stateutil.SliceRoot to ssz package (#14123) 2024-06-20 20:55:15 +00:00
Radosław Kapka
df3a9f218d More tracing in the validator client (#14125)
* More tracing in the validator client

* change context expectation in tests
2024-06-20 16:13:53 +00:00
Preston Van Loon
ae451a3a02 Update github.com/prysmaticlabs/go-bitfield (#14120) 2024-06-18 14:33:06 +00:00
Radosław Kapka
17561a6576 Do not fail production when consensus block value is unavailable (#14111)
* Do not fail production when consensus block value is unavailable

* add log

* use empty string instead of 0

* build fix
2024-06-14 18:30:40 +00:00
james-prysm
b842b7ea01 proposer settings log ux (#14106)
* adding some logs to improve debugging

* fixing log functions

* Update config/proposer/loader/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-06-14 14:16:43 +00:00
sam (jgscripts)
9bbe12e28c Correcting spelling errors (#14107)
* fix small spelling error

* fix small grammar error

* fix small spelling errors

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-06-14 13:41:33 +00:00
Delweng
0674cf64cc chore: make deepsource happy (#14081)
* chore(pruner): return error directly

Signed-off-by: jsvisa <delweng@gmail.com>

* chore(rpc): unused method receiver

Signed-off-by: jsvisa <delweng@gmail.com>

* fix(rpc): use net.JoinHostPort instead of fmt.Sprintf

Signed-off-by: jsvisa <delweng@gmail.com>

* chore(amiddleware):use http.NoBody instead of nil

Signed-off-by: jsvisa <delweng@gmail.com>

* chore(rpc): rm notused params

Signed-off-by: jsvisa <delweng@gmail.com>

* chore(p2p): comment

Signed-off-by: jsvisa <delweng@gmail.com>

* feat(db/prune): reduce complexity

Signed-off-by: jsvisa <delweng@gmail.com>

* chore(db/pruner): name

Signed-off-by: jsvisa <delweng@gmail.com>

* Revert "chore(pruner): return error directly"

This reverts commit d76e745f60.

Signed-off-by: jsvisa <delweng@gmail.com>

* revert back pruner.go

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2024-06-13 16:12:04 +00:00
james-prysm
3413d05b34 Electra: field renames (#14091)
* renaming functions and fields based on consensus changes

* execution api rename

* fixing test

* reverting spectests changes, it should be changed with new version

* reverting temporarily

* revert exclusions
2024-06-12 15:16:31 +00:00
Patrice Vignola
070a765d24 Add stub for VerifySignature when build tag blst_disabled is set (#12246)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-06-12 12:46:23 +00:00
Preston Van Loon
8ac1647436 Delete unused config (#14016) 2024-06-11 15:51:17 +00:00
james-prysm
dfe31c9242 adding in softer check for content type (#14097) 2024-06-10 17:15:23 +00:00
Radosław Kapka
b7866be3a9 EIP 7549 spectests (#14027)
* EIP 7549 spectests

* merge fix
2024-06-10 16:33:43 +00:00
Radosław Kapka
8413660d5f Keep only the latest value in the health channel (#14087)
* Increase health tracker channel buffer size

* keep only the latest value

* Make health test blocking as a regression test for PR #14807

* Fix new race conditions in the MockHealthClient

---------

Co-authored-by: Preston Van Loon <preston@pvl.dev>
2024-06-06 18:45:35 +00:00
Radosław Kapka
e037491756 Deprectate EnableDebugRPCEndpoints flag (#14015)
* Deprectate `EnableDebugRPCEndpoints` flag

* test fix

* add flag to deprecated list

* disable by default

* test fixes
2024-06-05 12:04:21 +00:00
kasey
ea2624b5ab always close cache warm chan to prevent blocking (#14080)
* always close cache warm chan to prevent blocking

* test that waitForCache does not block

* combine defers to reduce cognitive overhead

* lint

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-06-04 22:08:06 +00:00
james-prysm
1b40f941cf middleware for content type and accept headers (#14075)
* middleware for content type

* adding accept middleware too and tests

* Update beacon-chain/rpc/endpoints.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/endpoints.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/endpoints.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/endpoints.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* including radek's review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-06-04 20:38:21 +00:00
terence
57830435d7 Process epoch error to use correct state version (#14069)
* Process epoch error to use correct state version

* Fix return instead
2024-06-03 19:54:28 +00:00
Nishant Das
44d850de51 Change It To Debug (#14072) 2024-06-03 10:50:00 +00:00
Radosław Kapka
b08e691127 Organize validator flags (#14028)
* Organize validator flags

* whitespace

* fix comment in test

* remove unneeded flags
2024-05-31 18:36:18 +00:00
Radosław Kapka
968e82b02d EIP-7549 gRPC (part 1) (#14055)
* interfaces move

* build fix

* remove annoying warning

* more build fixes

* review

* core code

* tests part 1

* tests part 2

* TranslateParticipation doesn't need Electra

* remove unused function

* pending atts don't need Electra

* tests part 3

* build fixes

* review

* EIP-7549 gRPC part 1
2024-05-31 17:24:06 +00:00
james-prysm
de04ce8329 EIP-7002:Execution layer triggerable withdrawals (#14031)
* wip fixing 7002 branch

* fixing tests and functions

* fixing linting

* temp fix for transition

* adding unit tests for method

* fixing linting

* partial review from terence

* Update withdrawals.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update withdrawals.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update withdrawals.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update beacon-chain/core/electra/withdrawals.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-31 16:55:49 +00:00
terence
5efecff631 Fix mockgen sh (#14068)
* Fix mockgen sh

* Radek's suggestion

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Generate prysm chain client

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-31 16:24:17 +00:00
Radosław Kapka
3ab759e163 One more validator client cleanup (#14048)
* interface names

* interface method names

* inspection

* regenerate pb and mock

* Revert beacon node changes

* build fix

* review

* more functions

* combine parameters
2024-05-31 15:53:58 +00:00
james-prysm
836d369c6c api fix for panic on unsynced unfound block (#14063)
* api fix for panic

* adding test

* fixing how we handle the error
2024-05-31 14:46:38 +00:00
Nishant Das
568273453b Update Libp2p Dependencies (#14060)
* Update to v0.35.0 and v0.11.0

* Update Protobuf

* Update bazel deps
2024-05-30 08:56:55 +00:00
Nishant Das
7a4ecb6060 Restrict Dials From Discovery (#14052)
* Fix Excessive Subnet Dials

* Handle backoff in Iterator

* Slow Down Lookups

* Add Flag To Configure Dials

* Preston's Review

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Reduce polling period

* Manu's Review

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-05-30 06:57:26 +00:00
terence
82f0ea5b11 Fix dependent root retrival for first epoch (#14059) 2024-05-30 04:15:00 +00:00
Sammy Rosso
6fddd13cb2 Multiple BN HTTP resolver (#13433)
* http resolver

* Redo

* Revert "Redo"

This reverts commit 5437c44ac2.

* Revert "http resolver"

This reverts commit 206207b530.

* Add host change to ValidatorClient + Validator

* Update mockgen

* Tidy

* Add mock validator

* Update gomock

* Gaz

* Solve interface issues

* Fix host

* Fix test

* Add tests

* Add endpoint change log

* Fix log

* Gen mock

* Fix test

* Fix deepsource

* Lint + deepsource

* Move to healthCheckRoutine

* Fix build errors

* Switch host to string

* Forgot a couple

* Radek' review

* Add PushProposerSettings to goroutine

* Radek' review

* James' review + test fix

* Radek' suggestion

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Check if new node is healthy

* Fix linter errors

* Add host switch logic to ChangeHost

* Lint + comment

* Fix messy merge

* rename ChangeHost to SetHost

* improve log

* remove log

* switch one node

* rename param

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
2024-05-29 01:43:06 +00:00
terence
43c7659d18 Fix dependent root retrival genesis case (#14053)
* Fix dependent root retrival genesis case

* Remove print
2024-05-28 16:50:41 +00:00
Radosław Kapka
2d15e53dab Eip 7549 core (#14037)
* interfaces move

* build fix

* remove annoying warning

* more build fixes

* review

* core code

* tests part 1

* tests part 2

* TranslateParticipation doesn't need Electra

* remove unused function

* pending atts don't need Electra

* tests part 3

* build fixes

* review

* remove newline

* review

* fix test
2024-05-28 13:56:36 +00:00
Radosław Kapka
2f2152e039 Only log error when aggregator check fails (#14046)
* Only log error when aggregator check fails

* review
2024-05-27 18:05:46 +00:00
Preston Van Loon
2542189efc eip-7251: process_effective_balance_updates (#14003)
* eip-7251: process_effective_balance_updates

Spectests for process_effective_balance_updates

process_effective_balance_updates unit tests

* PR feedback from the amazing @rkapka

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-05-25 00:12:38 +00:00
terence
8e6d39a44b Support electra blob type for by range request (#14047)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-05-24 18:56:40 +00:00
terence
c35889d4c6 Fix CommitteeAssignments to not return every validator (#14039)
* Rewrite CommitteeAssignments to not return every validator

* Potuz's feedback
2024-05-24 16:56:42 +00:00
Sammy Rosso
10dedd5ced Fix race conditions + cleanup (#14041) 2024-05-23 22:07:16 +00:00
james-prysm
d2966a4c5b Electra core transition operations (#14001)
* adding electra operations

* enabling spec tests

* adding electra process epoch

* skipping spec tests for now

* Update testing/spectest/minimal/electra/fork_transition/BUILD.bazel

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* fixing naming

* gaz

* fixing more bazel build stuff

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-05-23 18:01:38 +00:00
kasey
62b5c43d87 paranoid underflow protection without error handling (#14044)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-23 17:57:53 +00:00
kasey
b04baa93cd convert ZeroWei to a func to avoid shared pointer (#14043)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-23 15:15:50 +00:00
james-prysm
2e84208169 WebFix develop (#14040)
* fixing issues introduced by PR 13593

* missed setting db

* linting
2024-05-23 14:07:30 +00:00
kasey
2265af58ae Unwrap payload bid (#14035)
* engine GetPayload returns entire response

* deprecate PayloadValueTo(Gwei|Wei)

* return entire bid from builder getter

* get bid value from api bid (not ExecutionData)

* plumb bid and bundle through BuildBlockParallel

* rm ValueInGwei

* removing wei/gwei fields from the payload wrappers

* commentary around the little-endian situation

* finish the job in BuildBlockParallel

* light self-review cleanup

* fix spectest mock

* restore engine timeout

* lint fixes

* de-duplicate imports

* remove errant comment

* James feedback

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-22 22:16:32 +00:00
Sammy Rosso
4d190c41cc Fix TestNodeHealth_Concurrently race condition (#14033) 2024-05-22 18:25:42 +00:00
Radosław Kapka
0fbb27d8e3 EIP-7549: Add aggregate attestation interfaces (#14029)
* interfaces move

* build fix

* remove annoying warning

* more build fixes

* review
2024-05-22 16:14:08 +00:00
Bhargava Shastry
3df3e84270 Fuzz ssz round trip (#14006)
* Initial commit

* Fuzz all generated ssz en/decoders.

* Make proto message comparison aware of nested proto messages and differing unknown fields therein.

* Don't name receiver variable if not necessary.

* Run gazelle
2024-05-21 20:09:49 +00:00
Radosław Kapka
30cc23c5de Substantial VC cleanup (#13593)
* Cleanup part 1

* Cleanup part 2

* Cleanup part 3

* remove lock field init

* doc for SignerConfig

* remove vars

* use full Keymanager word in function

* revert interface rename

* linter

* fix build issues

* review
2024-05-21 16:39:00 +00:00
kasey
9befb6bd06 g/wei math->primitives, some new helpers (#14026)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-20 15:40:46 +00:00
Radosław Kapka
8a12b78684 Remove Beacon API Postman collection (#14014) 2024-05-17 20:40:13 +00:00
kasey
46168607e8 Electra payload body engine methods (#14000)
* Combined v1/v2 payload body handling

* prevent overflows when dealing with electra fixture

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-17 20:31:07 +00:00
Preston Van Loon
1272b9e186 eip-7251: process_pending_balance_deposits (#14002)
* eip-7251: process_pending_balance_deposits

* Update beacon-chain/core/electra/balance_deposits_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Remove defensive check. A unit test shows nothing bad happens

* Safe sub to protect from underflow

* Use @kasey's idea for safer subtraction

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-15 17:29:38 +00:00
Preston Van Loon
fcbe19445a eip-7251: process consolidations (#13983)
* eip-7251: process_pending_consolidations and process_consolidations

* Consolidate unit tests + spectests

Fix failing spectest //testing/spectest/minimal/electra/operations:go_default_test

* Unskip consolidation processing for minimal spectests

* PR feedback

* Update beacon-chain/core/electra/consolidations_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/electra/consolidations_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Move consolidation limit check outside of the loop

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-15 13:50:33 +00:00
terence
2b4dffa87d Support Electra for WriteBlockChunk (#13999) 2024-05-15 02:52:42 +00:00
Preston Van Loon
49a6d02e12 Enable experimental_remote_downloader in CI. (#13996) 2024-05-14 18:04:39 +00:00
Preston Van Loon
2b06dfd4a3 Debugging attestation bitlist issues in minimal spec (#13997) 2024-05-14 13:56:00 +00:00
terence
6e81b4e84b Correctly return electra attestations for block getter (#13993) 2024-05-14 03:33:01 +00:00
terence
0de1282e1c Support version Electra for ForkchoiceUpdated (#13994)
* Support version Electra for ForkchoiceUpdated

* Update PbV3 version check
2024-05-13 22:21:18 +00:00
terence
e3db52ca1f Fix GetPayloadMethodV4 endpoint string (#13992) 2024-05-13 22:21:07 +00:00
Preston Van Loon
c5a36d4c70 eip-7251: queue_entire_balance_and_reset_validator, queue_excess_active_balance, and switch_to_compounding_validator with tests (#13982) 2024-05-13 18:12:38 +00:00
Preston Van Loon
e28b6695ba eip-7251: compute_consolidation_epoch_and_update_churn with tests (#13981)
tests for compute_consolidation_epoch_and_update_churn

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-05-13 14:25:07 +00:00
kasey
de177f74fb electra engine api support (#13978)
* electra engine api support

* add marshaling support for ExecutionPayloadElectra

* add receipts to json tests

* deep source

* simplify slice handling

* deep source lint about type/method order

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-12 20:19:08 +00:00
terence
e4310aef73 Update interop genesis for Electra (#13991) 2024-05-12 16:34:02 +00:00
terence
d71079e1d8 Add proposer suppot for Electra (#13987) 2024-05-12 13:55:01 +00:00
terence
c08d2f36b0 Add p2p support for Electra (#13985)
* Add p2p support for Electra

* Fix TestGossipTopicMappings_CorrectBlockType
2024-05-11 18:27:16 +00:00
terence
839a80e339 Add proposer gRPC suppot for Electra (#13984)
* Add proposer RPC suppot for Electra

* Kasey's feedback
2024-05-11 12:59:24 +00:00
Radosław Kapka
a35535043e Update state readme (#13890)
* README.md for the state package

* Update beacon-chain/state/state-native/README.md

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Revert "Update beacon-chain/state/state-native/README.md"

This reverts commit 6a4be3bae5.

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-05-10 16:49:02 +00:00
Preston Van Loon
323dd7b22d Electra: (partial) spectests (#13977)
* Electra epoch transition, re-exports only

* Buildable electra spectests

Fix minimal tests that are passing on mainnet

* Skip failing tests
2024-05-10 14:09:09 +00:00
terence
102128ca2e Add electra DB (#13975)
* Add electra DB

* Fix typo

* Revert deep ssz change

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-05-09 21:49:17 +00:00
Preston Van Loon
f3dd75a2c4 eip-7251: updated slash_validator with tests (#13976)
Tests for updated slash_validator
2024-05-09 20:40:54 +00:00
Preston Van Loon
0869814a0e eip-7251: updated initiate_validator_exit with tests (#13974)
eip-7251: tests for initiate_validator_exit
2024-05-09 15:21:22 +00:00
Brandon Liu
41edee9fe9 use time.NewTimer() to avoid possible memory leaks (#13800)
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-05-08 17:41:47 +00:00
james-prysm
2fa3694746 Electra: upgrade (#13933)
* wip fork logic upgrade

* fixing replay and fork.go

* improving process function and adding tests for transition

* updating unit tests and temporarily removing some fields on state_trie.go

* updating state

* wip adding upgrade to electra code

* adding some comments

* adding spec tests

* fixing values used in state transition logic

* updating upgrade test

* gofmt

* avoid dup word linting

* fixing spec tests for fork

* gaz

* fixing tests

* improving unit test with new getters

* fixing bazel for minimal fork test

* adding bazel file

* Update beacon-chain/core/electra/upgrade.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* addressing some comments and adding more tests

* addressing more feedback

* one more feedback

* changing value to interface after talking to preston

* adding missed review feedback

* fixing linting

* noticed I was using the wrong function in the state upgrade

* fixing and ignoring some deepsource issues

* moving core electra validator functions to helper to remove circular dependencies in other PRs

* missed deepsource complaint

* Update upgrade.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/util/electra_state.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update testing/util/electra_state.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

* removing deepsoure ignore comments

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-08 15:16:31 +00:00
Nishant Das
e9606b3635 Handle Each Blob In Its Own Goroutine (#13959) 2024-05-07 22:21:27 +00:00
dependabot[bot]
ed7c4bb6a7 Bump golang.org/x/net from 0.21.0 to 0.23.0 (#13895)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-07 21:57:45 +00:00
Preston Van Loon
c93fea4ec4 Update spectests to v1.5.0-alpha.2 (#13961) 2024-05-07 20:56:08 +00:00
Nishant Das
aa847991e0 Update Libp2p Dependencies (#13960)
* Update Libp2p

* Update Go Sum
2024-05-07 15:10:18 +00:00
Radosław Kapka
5f1b903bdf EIP-7549 beacon spec (#13946)
* EIP-7549 beacon spec

* reviews

* change signature of AttestingIndices
2024-05-07 13:48:23 +00:00
terence
49f3531aed Remove unused validator map copy method (#13954) 2024-05-07 00:47:05 +00:00
Preston Van Loon
9b2934f1f6 Electra: BeaconState implementation (#13919)
* Electra: Beacon State

* Electra: Beacon state fixes from PR 13919

* Add missing tests - part 1

* Split eip_7251_root.go into different files and reuse/share code with historical state summaries root. It's identical!

* Add missing tests - part 2

* deposit receipts start index getters and setters (#13947)

* adding in getters and setters for deposit receipts start index

* adding tests

* gaz

* Add missing tests - part 3 of 3

Update the electra withdrawal example with a ssz state containing pending partial withdrawals

* add tests for beacon-chain/state/state-native/getters_balance_deposits.go

* Add electra field to testing/util/block.go execution payload

* godoc commentary on public methods

* Fix failing test

* Add balances index out of bounds check and relevant tests.

* Revert switch case electra

* Instead of copying spectest data into testdata, use the spectest dependency

* Deepsource fixes

* Address @rkapka PR feedback

* s/MaxPendingPartialsPerWithdrawalSweep/MaxPendingPartialsPerWithdrawalsSweep/

* Use multivalue slice compatible accessors for validator and balance in ActiveBalanceAtIndex

* More @rkapka feedback. What a great reviewer!

* More tests for branching logic in ExitEpochAndUpdateChurn

* fix build

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-05-06 18:04:33 +00:00
Preston Van Loon
26355768a0 Spectest: v1.5.0-alpha.1 (#13934)
* Update spectests to v1.5.0-alpha.1

* Add PeerDAS config change
2024-05-03 18:53:46 +00:00
kasey
80bff0dc2d Fork-specific consensus-types interfaces (#13948)
* fork-specific interface for electra

* add electra to wrapped payload switch

* use electra body in block factory

* deepsource

* rm pb getters from electra payload

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-05-03 17:12:27 +00:00
kasey
c312a88aa3 Removes fork-specific concrete type getters (#13941)
* removing typed pb accessors

* refactor ssz api resp handlers to avoid typed pbs

* json get block handler refactor

* SubmitBlindedBlock to use generic json handling

* update SubmitBlindedBlock

* clear out more usages of PbForkname methods

* remove fork-specific getters from block interface

* remove usages of payload pb methods

* remove pb helpers from execution payload interface

* Update beacon-chain/rpc/eth/beacon/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/builder/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/builder/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/builder/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/builder/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update api/client/builder/client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Radek review

* fix error message

* deal with wonky builder responses

* ✂️

* gaz

* lint

* tweaks for deep source

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-05-02 20:52:27 +00:00
terence
625818d556 EIP6110: add validator index cache (#13943)
* EIP6110: add validator index cache

* Add tests

* Radek's feedback
2024-05-02 16:18:23 +00:00
Radosław Kapka
2c5a2e8ec7 Electra attestation interfaces (#13937)
* config values

* block protos

* get_committee_indices

* proto and ssz

* attestation interface

* Revert "Auxiliary commit to revert individual files from deadb2183723511721b3288c7168808a4fa97c64"

This reverts commit 32ad5009537bc5ec0e6caf9f52143d380d00be85.

* todos

* get_attesting_indices

* Revert "Auxiliary commit to revert individual files from dd2789723f90b15eb1f874b561d88d11dcc9c0bf"

This reverts commit f39644ed3cb6f3964fc6c86fdf4bd5de2a9668c8.

* beacon spec changes

* Fix pending attestation. Build ok

* Electra: add electra version

* Electra: consensus types

* gocognit exclusion

* @potuz's suggestion

* build fix

* interfaces for indexed att and slashing

* indexed att usage

* BuildSignedBeaconBlockFromExecutionPayload

* slashing usage

* grpc stubs

* remove unused methods

* Electra attestation interfaces

* cleanup

* tests

* make linter happy

* simple casting

* test fixes

* Fix spectest failures

* Regen pb and ssz files

* Handle "not ok" type assertion cases

* Setters that check version should always return an error. SetAttesterSlashings and SetAttestations

* gofmt

* Fix TestMinSpanChunksSlice_CheckSlashable

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@pvl.dev>
2024-04-30 21:29:38 +00:00
Sammy Rosso
ae16d5f52c Run correct test (#13935) 2024-04-30 10:15:07 +00:00
terence
d69be8a766 Fix spelling (#13929) 2024-04-28 02:40:26 +00:00
Preston Van Loon
8df62a537b Electra: beacon-chain/core/helpers (#13921)
* Electra helpers

* Electra helper tests and other fixes

* @terencechain feedback
2024-04-26 18:37:32 +00:00
Preston Van Loon
bf5e667351 Electra: HTR util for DepositReceipt and ExecutionLayerWithdrawalRequest (#13924)
* Electra: HTR utils for DepositReceipts and ExecutionLayerWithdrawalRequests

* Tests for HTR utils
2024-04-26 16:47:00 +00:00
Preston Van Loon
0e5c2bd18e Electra: Minor proto changes, cloner additions (#13923)
* Electra: more proto changes

* Roundtrip test for cloners
2024-04-26 15:57:51 +00:00
Preston Van Loon
3233e64ace beacon-chain/cache: Convert tests to cache_test blackbox testing (#13920)
* beacon-chain/cache: convert to blackbox tests (package cache_test)

* Move balanceCacheKey to its own file to satisify go fuzz build
2024-04-26 02:50:58 +00:00
Preston Van Loon
751117a308 Electra: full beacon config (#13918)
* Electra: full beacon config

Fix TestGetSpec

* Fix beacon config spec compliance test so that it properly loads the config from spec tests. Tests failing for now.

* fix tests and comply with spec presets
2024-04-26 00:22:54 +00:00
kasey
8d9024f01f use [32]byte keys in the filesystem cache (#13885)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-25 04:59:21 +00:00
terence
a9862f32f3 Protobufs for Electra devnet-0 (#13905)
* block protos

* proto and ssz

* stubs

* Enable Electra spec test

* Pull in EIP-7251 protobuf changes

From: https://github.com/prysmaticlabs/prysm/pull/13903

* All EIP7549 containers are passing

* All EIP7251 containers passing

* including changes from eip7002

* Everything passing except for beacon state hash tree root

* fixing eletra state to use electra payload

* Fix minimal test. Skip beacon state test

* Perston's feedback

---------

Co-authored-by: rkapka <radoslaw.kapka@gmail.com>
Co-authored-by: james-prysm <james@prysmaticlabs.com>
2024-04-24 21:38:03 +00:00
Sammy Rosso
c8d6f47749 Remove EnableEIP4881 flag (#13826)
* Remove EnableEIP4881 flag

* Gaz

* Fix missing error handler

* Remove old tree and fix tests

* Gaz

* Fix build import

* Replace depositcache

* Add pendingDeposit tests

* Nishant's fix

* Fix unsafe uint64 to int

* Fix other unsafe uint64 to int

* Remove: RemovePendingDeposit

* Deprecate and remove DisableEIP4881 flag

* Check: index not greater than deposit count

* Move index check
2024-04-24 13:24:51 +00:00
Manu NALEPA
a6f134e48e Revert "zig: Update zig to recent main branch commit (#13142)" (#13908)
This reverts commit b24b60dbd8.
2024-04-24 03:56:52 +00:00
Preston Van Loon
fdbb5136d9 spectests: fail hard on missing test folders (#13913) 2024-04-24 02:49:33 +00:00
Preston Van Loon
2c66918594 Refactor beacon-chain/core/helpers tests to be black box (#13906) 2024-04-23 15:24:47 +00:00
Manu NALEPA
a0dac292ff Do not remove blobs DB in slasher. (#13881) 2024-04-23 08:09:26 +00:00
terence
75857e7177 Simplify prune invalid by reusing existing fork choice store call (#13878) 2024-04-23 03:09:16 +00:00
Preston Van Loon
0369f70b0b Electra beacon config (#13907)
* Update spectests to v1.5.0

* Add electra config

* Fix tests in beacon-chain/rpc/eth/config

* gofmt
2024-04-22 22:14:57 +00:00
Alec Thomas
276b97530f Change example.org DNS record (#13904)
DNS Changed for this record causing tests to fail

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-04-22 16:36:30 +00:00
Radosław Kapka
d5daf49a9a Use correct port for health check in Beacon API e2e evaluator (#13892) 2024-04-22 16:18:07 +00:00
james-prysm
feb16ae4aa consistent auth token for validator apis (#13747)
* wip

* fixing tests

* adding more tests especially to handle legacy

* fixing linting

* fixing deepsource issues and flags

* fixing some deepsource issues,pathing issues, and logs

* some review items

* adding additional review feedback

* updating to follow updates from https://github.com/ethereum/keymanager-APIs/pull/74

* adjusting functions to match changes in keymanagers PR

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/auth_token.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* review feedback

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-04-18 16:26:49 +00:00
kasey
219301339c Don't return error that can be internally handled (#13887)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-17 18:28:01 +00:00
Radosław Kapka
aec349f75a Upgrade the Beacon API e2e evaluator (#13868)
* GET

* POST

* Revert "Auxiliary commit to revert individual files from 615feb104004d6a945ededf5862ae38325fc7ec2"

This reverts commit 55cf071c684019f3d6124179154c10b2277fda49.

* comment fix

* deepsource
2024-04-15 05:56:47 +00:00
terence
5f909caedf Remove unused IsViableForCheckpoint (#13879) 2024-04-14 16:38:25 +00:00
Radosław Kapka
ba6dff3adb Return syncing status when node is optimistic (#13875) 2024-04-12 10:24:30 +00:00
Radosław Kapka
8cd05f098b Use read only validators in Beacon API (#13873) 2024-04-12 07:19:40 +00:00
Radosław Kapka
425f5387fa Handle overflow in retention period calculation (#13874) 2024-04-12 07:17:12 +00:00
Nishant Das
f2ce115ade Revert Peer Log Changes (#13872) 2024-04-12 06:49:01 +00:00
kasey
090a3e1ded Fix bug from PR 13827 (#13871)
* fix AS cache bug, tighten ro constructors

* additional coverage on AS cache filter

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-11 23:07:44 +00:00
kasey
c0acb7d352 Backfill throttling (#13855)
* add a sleep between retries as a simple throttle

* unit test

* deepsource

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-11 15:22:29 +00:00
Radosław Kapka
0d6070e6fc Use retention period when fetching blobs (#13869) 2024-04-11 14:06:00 +00:00
Manu NALEPA
bd00f851f0 e2e: Expected log Running node with peerId= -> Running node with. (#13861)
Rationale:
The `FindFollowingTextInFile` seems to have troubles with `logrus` fields.
2024-04-09 07:51:56 +00:00
Nishant Das
1a0c07deec Extend Broadcast Window For Attestations (#13858)
* fix it

* make check better
2024-04-08 04:49:20 +00:00
kasey
04f231a400 Initsync skip local blobs (#13827)
* wip - init-sync skip available blob req

* satisfy deep source

* gaz

* don't need to sort blobs; simplify blobRequest stack

* wip debug log to watch blob skip behavior

* unit tests for new blob req generator

* refactor to reduce blob req func count

* log when WaitForSummarizer fails

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-05 19:09:43 +00:00
Manu NALEPA
be1bfcce63 P2P: Add QUIC support (#13786)
* (Unrelated) DoppelGanger: Improve message.

* `beacon-blocks-by-range`: Add `--network` option.

* `ensurePeerConnections`: Remove capital letter in error message.

* `MultiAddressBuilder{WithID}`: Refactor.

* `buildOptions`: Improve log.

* `NewService`: Bubbles up errors.

* `tcp` ==> `libp2ptcp`

* `multiAddressBuilderWithID`: Add the ability to build QUIC multiaddr

* `p2p Start`: Fix error message.

* `p2p`: Add QUIC support.

* Status: Implement `{Inbound,Outbound}Connected{TCP,QUIC}`.

* Logging: Display the number of TCP/QUIC connected peers.

* P2P: Implement `{Inbound,Outbound}ConnectedWithProtocol`.

* Hide QUIC protocol behind the `--enable-quic` feature flag.

* `e2e`: Add `--enable-quic` flag.

* Add `--enable-quic` in `devModeFlag`.

* `convertToMultiAddrs` ==> `retrieveMultiAddrsFromNode`.

* `convertToAddrInfo`: Ensure `len(infos) == 1`.
2024-04-04 12:21:35 +00:00
Manu NALEPA
8cf5d79852 Remove the Goerli/Prater support. (#13846) 2024-04-03 19:19:17 +00:00
redistay
f7912e7c20 chore: fix some comments (#13843)
Signed-off-by: redistay <wujunjing@outlook.com>
2024-04-02 22:19:15 +00:00
terence
caa8be5dd1 Beacon-api: broadcast blobs in the event of seen block (#13830)
* Beacon-api: broadcast blobs in the event of seen block

* Fix parameters

* Fix test

* Check forkchoice

* Ran go format

* Revert "Ran go format"

This reverts commit 091e77e81d6e2b9861fecc27c0bad1898033f9a3.

* James feedback

* Radek's feedback

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Fix bad tests

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-04-02 18:12:58 +00:00
cui
0c15a30a34 using slices.Index (#13836) 2024-04-02 16:30:05 +00:00
cui
7bce1c0714 using slices.IndexFunc (#13839) 2024-04-02 16:06:27 +00:00
Radosław Kapka
d1084cbe48 Send correct state root with finalization event (#13842) 2024-04-02 15:32:32 +00:00
cui
2cc3f69a3f using slices.Contains (#13835) 2024-04-01 21:26:52 +00:00
cui
a861489a83 using slices.ContainsFunc (#13838) 2024-04-01 21:15:38 +00:00
cui
0e1c585f7d using slices.Contains (#13837) 2024-04-01 21:10:46 +00:00
cui
9df20e616c using slices.IndexFunc (#13834) 2024-04-01 20:04:40 +00:00
kasey
53fdd2d062 allow other pkgs to check for blobs in pruning cache (#13788)
* allow other pkgs to check for blobs in pruning cache

* address deepsource complaints

* custom error to simplify test setup

* add AllAvailable method

* make storage summary slot field private

* unit test and off-by-one fix

* remove comment with copy of tested function

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-04-01 14:19:51 +00:00
Sammy Rosso
2b4bb5d890 Fixed spelling mistakes in comments (#13833) 2024-04-01 11:12:20 +00:00
Nishant Das
38f208d70d Reject Empty Bundles (#13798)
* reject it

* test

* add test case
2024-04-01 04:37:36 +00:00
Nishant Das
65b90abdda Maximize Peer Capacity When Syncing (#13820)
* maximize it

* fix it

* lint

* add test

* Update beacon-chain/sync/initial-sync/blocks_fetcher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* logs

* kasey's review

* kasey's review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-03-30 14:54:11 +00:00
kasey
f3b49d4eaf Repair idx 13486 (#13831)
* Revert "Modify the algorithm of `updateFinalizedBlockRoots` (#13486)"

This reverts commit 32fb183392.

* migration to fix index corruption from pr 13486

* bail as soon as we see 10 epochs without the bug

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-29 17:29:39 +00:00
Lorenzo
5b1da7353c feat(direct peers): configure static peers to be direct peers in pubsub options (#13773) 2024-03-29 04:00:40 +00:00
terence
9f17e65860 Fill in missing debug logs for blob p2p IGNORE/REJECT (#13825) 2024-03-28 16:11:55 +00:00
Manu NALEPA
9b2d53b0d1 Bump libp2p to v0.33.1 (#13784)
* Run bare `//:gazelle -- update-repos`.

--> Removes some blank lines.

* Update libp2p to `v0.33.1`.
2024-03-28 08:38:46 +00:00
terence
d6f9196707 Change goodbye message from rate limited peer to debug verbosity (#13819) 2024-03-28 04:14:37 +00:00
Potuz
1b0e09369e Add metrics to track pending attestations (#13815) 2024-03-27 18:20:53 +00:00
Potuz
12482eeb40 Remove check for duplicates in pending attestation queue (#13814)
* Remove check for duplicates in pending attestation queue

The current queue will only save 1 unaggregated attestation for a pending block because we wrap the object into a SignedAggregatedAttestationAndProof with a zeroed aggregator.

* fix tests
2024-03-27 16:39:45 +00:00
Joel Rousseau
acc307b959 Command-line interface for visualizing min/max span bucket (#13748)
* add max/min span visualisation tool cli

* go mod tidy

* lint imports

* remove typo

* fix epoch table value

* fix deepsource

* add dep to bazel

* fix dep import order

* change command name from span to slasher-span-display

* change command args style using - instead of _

* sed s/CONFIGURATION/SLASHER PARAMS//

* change double neg to double pos condition

* remove unused anonymous func

* better function naming

* add range condition

* [deepsource] Fix Empty slice literal used to declare a variable
    GO-W1027

* correct typo

* do not show incorrect epochs due to round robin

* fix import

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-03-27 16:15:39 +00:00
Afanti
c1d75c295a chore: enhance comment and more readable (#13792) 2024-03-27 14:43:55 +00:00
Potuz
fad118cb04 Simplify ValidateAttestationTime (#13813)
ValidateClock in ValidateAttestationTime is useless

The check is that the attSlot is not > than the currentslot + 128 slots.

Later there's a check that the attSlot start time is not > than current slot
start time + clockDisparity.

if attSlot > than currentSlot + 128 slots, then the second check would fail
anyway.

The lattest check already guarantees that the attSlot cannot be larger than the
currentSlot, therefore it may never happen that attEpoch > currentEpoch. We just
need to check for Deneb that attEpoch >= currentEpoch - 1.

Removes also some duplicated variables like the attestation epoch being computed
twice.
2024-03-27 14:17:16 +00:00
kasey
cdd1d819df Refactor batch verifier for sharing across packages (#13812)
* refactor batch verifier to share with pending queue

* unit test for batch verifier

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-27 12:36:17 +00:00
terence
97edffaff5 Add bid value metrics (#13804) 2024-03-26 14:58:41 +00:00
Radosław Kapka
6de7df6b9d Get genesis only once (#13796) 2024-03-26 03:26:34 +00:00
Sammy Rosso
14d7416c16 Spec test coverage report hack (#13718)
* Spec test report hack

* no export

* fix shell complaint

* shell fix?

* shell again?

* chmod +x ./hack/spectest-report.sh

* Review + improvements

* Remove unwanted change

* Add exclusion list

* Fix path + add eip6110 to exclusion

* Fix bazel path nonsense

* Add extra detail about specific test

* Cleanup exclusion list

* Add fail conditions

* Add mkdir

* Shorten filename + mkdir only if new

* Fix names

* Add to exclusion list

* Add report to .gitignore

* Back to stupid names

* Add Bazel flags option

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-25 16:10:32 +00:00
Radosław Kapka
6782df917a Utilize next slot cache in block rewards rpc (#13684)
* Utilize next slot cache in block rewards rpc

* msg fix

* tests
2024-03-25 08:56:20 +00:00
Bharath Vedartham
3d2230223f create the log file along with its parent directory if not present (#12675)
* Remove Feature Flag From Prater (#12082)

* Use Epoch boundary cache to retrieve balances (#12083)

* Use Epoch boundary cache to retrieve balances

* save boundary states before inserting to forkchoice

* move up last block save

* remove boundary checks on balances

* fix ordering

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* create the log file along with its parent directory if not present

* only give ReadWritePermissions to the log file

* use io/file package to create the parent directories

* fix ci related issues

* add regression tests

* run gazelle

* fix tests

* remove print statements

* gazelle

* Remove failing test for MkdirAll, this failure is not expected

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-22 15:32:08 +00:00
Preston Van Loon
b008a6422d Add tarball support for docker images (#13790) 2024-03-22 15:31:29 +00:00
Fredrik Svantes
d19365507f Set default LocalBlockValueBoost to 10 (#13772)
* Set default LocalBlockValueBoost to 10

* Update base.go

* Update mainnet_config.go
2024-03-22 13:18:20 +00:00
kasey
c05e39a668 fix handling of goodbye messages for limited peers (#13785)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-22 13:06:16 +00:00
Radosław Kapka
63c2b3563a Optimize GetDuties VC action (#13789)
* wait groups

* errgroup

* tests

* bzl

* review
2024-03-22 09:50:19 +00:00
Justin Traglia
a6e86c6731 Rename payloadattribute Timestamps to Timestamp (#13523)
Co-authored-by: terence <terence@prysmaticlabs.com>
2024-03-21 21:11:01 +00:00
Radosław Kapka
32fb183392 Modify the algorithm of updateFinalizedBlockRoots (#13486)
* rename error var

* new algo

* replay_test

* add comment

* review

* fill out parent root

* handle edge cases

* review
2024-03-21 21:09:56 +00:00
carrychair
cade09ba0b chore: fix some typos (#13726)
Signed-off-by: carrychair <linghuchong404@gmail.com>
2024-03-21 21:00:21 +00:00
Potuz
f85ddfe265 Log the slot and blockroot when we deadline waiting for blobs (#13774) 2024-03-21 20:29:23 +00:00
terence
3b97094ea4 Log da block root in hex (#13787) 2024-03-21 20:26:17 +00:00
Nishant Das
acdbf7c491 expand it (#13770) 2024-03-21 19:57:22 +00:00
Potuz
1cc1effd75 Revert "pass justified=finalized in Prater (#13695)" (#13709)
This reverts commit 102518e106.
2024-03-21 17:42:40 +00:00
james-prysm
f7f1d249f2 Fix get validator endpoint for empty query parameters (#13780)
* fix handlers for get validators

* removing log
2024-03-21 14:00:07 +00:00
kasey
02abb3e3c0 add log message if in da check at slot end (#13776)
* add log message if in da check at slot end

* don't bother logging late da check start

* break up defer with a var, too dense all together

* pass slot instead of block ref

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-20 19:31:09 +00:00
james-prysm
2255c8b287 setting missing beacon API (#13778) 2024-03-20 17:59:15 +00:00
terence
27ecf448a7 Add da waited time to sync block log (#13775) 2024-03-20 14:53:02 +00:00
james-prysm
e243f04e44 validator client on rest mode has an inappropriate context deadline for events (#13771)
* addressing errors on events endpoint

* reverting timeout on get health

* fixing linting

* fixing more linting

* Update validator/client/beacon-api/beacon_api_validator_client.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/rpc/eth/events/events.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* reverting change and removing line on context done which creates a superfluous response.WriteHeader error

* gofmt

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-20 13:19:05 +00:00
Manu NALEPA
fca1adbad7 Re-design TestStartDiscV5_DiscoverPeersWithSubnets test (#13766)
* `Test_AttSubnets`: Factorize.

* `filterPeerForAttSubnet`: `O(n)` ==> `O(1)`

* `FindPeersWithSubnet`: Optimize.

* `TestStartDiscV5_DiscoverPeersWithSubnets`: Complete re-design.

* `broadcastAttestation`: User `log.WithFields`.

* `filterPeer`: Refactor comments.

* Make deepsource happy.

* `TestStartDiscV5_FindPeersWithSubnet`: Add context cancellation.

Add some notes on `FindPeersWithSubnet` about
this limitation as well.
2024-03-20 03:36:00 +00:00
Radosław Kapka
b692722ddf Optimize SubmitAggregateSelectionProof VC action (#13711)
* Optimize `SubscribeCommitteeSubnets` VC action

* test fixes

* remove newline

* Optimize `SubmitAggregateSelectionProof`

* mock

* bzl gzl

* test fixes
2024-03-19 14:09:07 +00:00
Nishant Das
c4f6020677 add mplex timeout (#13745) 2024-03-19 13:37:23 +00:00
Chanh Le
d779e65d4e chore(kzg): Additional tests for KZG commitments (#13758)
* add a test explaining kzgRootIndex

* minor

* minor
2024-03-19 09:08:02 +00:00
terence
357211b7d9 Update spec test to official 1.4.0 (#13761) 2024-03-18 23:39:03 +00:00
Potuz
2dd48343a2 Set default fee recipient if tracked val fails (#13768) 2024-03-18 19:35:34 +00:00
james-prysm
7f931bf65b Keymanager APIs - get,post,delete graffiti (#13474)
* wip

* adding set and delete graffiti

* fixing mock

* fixing mock linting and putting in scaffolds for unit tests

* adding some tests

* gaz

* adding tests

* updating missing unit test

* fixing unit test

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/client/propose.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/propose.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's feedback

* fixing tests

* using wrapper for graffiti

* fixing linting

* wip

* fixing setting proposer settings

* more partial fixes to tests

* gaz

* fixing tests and setting logic

* changing keymanager

* fixing tests and making graffiti optional in the proposer file

* remove unneeded lines

* reverting unintended changes

* Update validator/client/propose.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* addressing feedback

* removing uneeded line

* fixing bad merge resolution

* gofmt

* gaz

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-03-18 15:03:08 +00:00
Nishant Das
fda4589251 Rewrite Pruning Implementation To Handle EIP 7045 (#13762)
* make it very big

* use new pruning implementation

* handle pre deneb

* revert cache change

* less verbose

* gaz

* Update beacon-chain/operations/attestations/prune_expired.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* gofmt

* be safer

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2024-03-18 12:57:21 +00:00
kasey
34593d34d4 allow blob by root within da period (#13757)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-18 03:15:17 +00:00
Potuz
4d18e590ed Rename mispelled variable (#13759) 2024-03-17 20:47:19 +00:00
Potuz
ec8b67cb12 Use headstate for recent checkpoints (#13746)
* Use headstate for recent checkpoints

* add the computed state to the checkpoint cache

* acquire a multilock
2024-03-17 18:50:49 +00:00
terence
a817aa0a8d New gossip cache size (#13756)
* New gossip cache size

Increase seen aggregate cache size to 4096

* Update cache size to 8192

* 16384
2024-03-17 03:02:00 +00:00
kasey
d76f55e97a adds a metric to track blob sig cache lookups (#13755)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-16 20:41:21 +00:00
james-prysm
2de21eb22f adding headers to post endpoint (#13753) 2024-03-15 18:19:42 +00:00
Nishant Das
58b8c31c93 mark in progress (#13750) 2024-03-15 16:46:26 +00:00
kasey
f343333880 handle special case of batch size=1 (#13646)
* handle special case of batch size=1

* unit test case for backfill batch len=1

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-15 15:19:59 +00:00
kasey
8e0b1b7e1f Backfill min slot flag (#13729)
* flag to set an older backfill slot target

* wire up flag to main cli and usage

* fix deepsource complaints

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-15 15:17:24 +00:00
Manu NALEPA
65f71b3a48 P2P: Simplify code (#13719)
* `subscribeStaticWithSubnets`: Fix docstring.

* `buildOptions`: Avoid `options` mutations.

* `dv5Cfg`: Avoid mutation.

* `RefreshENR`: Use default for all but Phase0.

* `udp4`, `udp6`: Create enum.

* `p2p.Config`: `BootstrapNodeAddr`==> `BootstrapNodeAddrs`.

* `p2p.Config`: `Discv5BootStrapAddr` ==> `Discv5BootStrapAddrs`.

* `TestScorers_BadResponses_Score`: Improve.

* `BeaconNode`: Avoid mutation.

* `TestStore_TrustedPeers`: Remove blankline.

* Remove blank identifiers.

* `privKey`: Keep the majority of code with low indentation.

* `P2PPreregistration`: Return error instead of fatal log.

* `parseBootStrapAddrs` => `ParseBootStrapAddrs` (export)

* `p2p.Config`: Remove `BootstrapNodeAddrs`.

* `NewService`: Avoid mutation when possible.

* `Service`: Remove blank identifier.

* `buildOptions`: Avoid `log.Fatalf` (make deepsource happy).

* `registerGRPCGateway`: Use `net.JoinHostPort` (make deepsource happy).

* `registerBuilderService`: Make deepsource happy.

* `scorers`: Add `NoLock` suffix (make deepsource happy).

* `scorerr`: Add some `NoLock`suffixes (making deepsource happy).

* `discovery_test.go`. Remove init.

Rationale:
`rand.Seed` is deprecated: As of Go 1.20 there is no reason to call Seed with a random value. Programs that call Seed with a known value to get a specific sequence of results should use New(NewSource(seed)) to obtain a local random generator.

This makes deepsource happy as well.

* `createListener`: Reduce cyclomatic complexity (make deepsource happy).

* `startDB`: Reduce cyclomatic complexity (make deepsource happy).

* `main`: Log a FATAL on error.

This way, the error message is very readable.
Before this commit, the error message is the less readable
message in the logs.

* `New`: Reduce cyclomatic complexity (make deepsource happy).

* `main`: Avoid `App` mutation, and make deepsource happy.

* Update beacon-chain/node/node.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* `bootnodes` ==> `BootNodes` (Fix PR comment).

* Remove duplicate `configureFastSSZHashingAlgorithm` since already done in `configureBeacon`. (Fix PR comment)

* Add `TestCreateLocalNode`. (PR comment fix.)

* `startModules` ==> `startBaseServices (Fix PR comment).

* `buildOptions` return errors consistently.

* `New`: Change ordering.

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-03-15 11:08:19 +00:00
kasey
9fcb9b86af fix 1-worker underflow; lower default batch size (#13734)
* fix 1-worker underflow; lower default batch size

* adding test for single item in batcher

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-14 00:59:27 +00:00
terence
aa63c4e7f2 Use correct gossip validation time (#13740) 2024-03-13 16:10:18 +00:00
james-prysm
d6ae838bbf replace receive slot with event stream (#13563)
* WIP

* event stream wip

* returning nil

* temp removing some tests

* wip health checks

* fixing conficts

* updating fields based on linting

* fixing more errors

* fixing mocks

* fixing more mocks

* fixing more linting

* removing white space for lint

* fixing log format

* gaz

* reverting changes on grpc

* fixing unit tests

* adding in tests for health tracker and event stream

* adding more tests for streaming slot

* gaz

* Update api/client/event/event_stream.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* review comments

* Update validator/client/runner.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_api_validator_client.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing radek comments

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing review feedback

* moving things to below next slot ticker

* fixing tests

* update naming

* adding TODO comment

* Update api/client/beacon/health.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comments

* fixing broken linting

* fixing more import issues

* fixing more import issues

* linting

* updating based on radek's comments

* addressing more comments

* fixing nogo error

* fixing duplicate import

* gaz

* adding radek's review suggestion

* Update proto/prysm/v1alpha1/node.proto

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* preston review comments

* Update api/client/event/event_stream.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* addressing some more preston review items

* fixing tests for linting

* fixing missed linting

* updating based on feedback to simplify

* adding interface check at the top

* reverting some comments

* cleaning up intatiations

* reworking the health tracker

* fixing linting

* fixing more linting to adhear to interface

* adding interface check at the the top of the file

* fixing unit tests

* attempting to fix dependency cycle

* addressing radek's comment

* Update validator/client/beacon-api/beacon_api_validator_client.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* adding more tests and feedback items

* fixing TODO comment

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-13 13:01:05 +00:00
terence
d49afb370c Add blob index to file path (#13736)
* Add index to file path

* Add block root

* Add index to file path
2024-03-13 06:36:43 +00:00
terence
4d3a6d84d2 Add gossip blob sidecar verification ms metric (#13737) 2024-03-13 04:41:47 +00:00
Preston Van Loon
9c5d16e161 Clean up some unused beacon state proto types and related code (#13735) 2024-03-13 03:20:36 +00:00
terence
4731304187 Save invalid blob to temp under new flag (#13725) 2024-03-12 17:51:08 +00:00
Potuz
02cbcf8545 only update head at 10 seconds when validating (#13570)
* only update head at 10 seconds when validating

* fix tests
2024-03-12 15:11:40 +00:00
Potuz
4e10734ae4 Compute unrealized checkpoints with pcli (#13692)
* Compute unrealized checkpoints with pcli

* gazelle

* fix lint

* Gazelle

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-12 15:03:21 +00:00
Preston Van Loon
e19c99c3e2 Update bazel, rules_go, gazelle, and go versions (#13724)
* Update go, rules_go, gazelle, bazel

* Update zig toolchain
2024-03-12 14:33:37 +00:00
terence
697bcd418c Save invalid block to temp save-invalid-block-temp (#13722) 2024-03-11 20:34:44 +00:00
terence
ec7949fa4b Use justified checkpoint from head state to build attestation (#13703) 2024-03-11 15:05:40 +00:00
Nishant Das
cb8eb4e955 fix context deadline rejections (#13716) 2024-03-11 04:51:02 +00:00
Chanh Le
800f3b572f chore(execution): Clean up unreachable code; use new(big.Int) instead of big.NewInt(0) (#13715)
* refactor with builtin min/max

* use new(big.Int) for more efficiency
2024-03-11 00:31:55 +00:00
terence
9d3af41acb Remove unused deneb code (#13712)
* Remove unused deneb code

* Gazelle
2024-03-09 00:12:26 +00:00
kasey
07a0a95ee7 Blob verification spectest (#13707)
* use real blob verifier in forkchoice spectest

* wip

* Use real blob sidecar for test

* Set file db correctly

* correctly handle blob cases where valid=false

* work-around spectest's weird Fork in genesis state

* gaz

* revert T-money's log level change

* rm whitespace

* unskip minimal test

* Preston's feedback

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2024-03-08 18:20:38 +00:00
Radosław Kapka
9e7352704c Optimize SubscribeCommitteeSubnets VC action (#13702)
* Optimize `SubscribeCommitteeSubnets` VC action

* test fixes

* remove newline

* review
2024-03-08 18:08:35 +00:00
Nishant Das
2616de1eb1 Check Unrealized Justification Balances In Spectests (#13710)
* add them

* Ensure activation epoch does not overflow

* add them all in

* better check for overflow

* fix tests

* fix tests

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2024-03-08 14:49:34 +00:00
Manu NALEPA
b2e3c29ab3 Improve logging. (#13708)
* Improve logging.

* Make deepsource happy.

* Fix comment.
2024-03-08 12:23:34 +00:00
Preston Van Loon
83538251aa Remove DOMAIN_BLOB_SIDECAR. See https://github.com/ethereum/consensus-specs/pull/3542 (#13706) 2024-03-08 03:55:46 +00:00
Stefan
2442280e37 Fix/race receive block (#13700)
* blob save: add better data checking for empty blob issues (#13647)

(cherry picked from commit daad29d0de)

* avoid part path collisions with mem addr entropy (#13648)

* avoid part path collisions with mem addr entropy

* Regression test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
(cherry picked from commit 4c66e4d060)

* fix error race

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
2024-03-07 19:46:03 +00:00
Stefan
4608569495 fix race condition when pinging peers (#13701) 2024-03-07 19:21:55 +00:00
Preston Van Loon
20d013a30b Unskip merkle proof tests (#13704) 2024-03-07 19:18:25 +00:00
Potuz
b0a2115a26 Fix UJ (#13688)
* Fix UJ

* gate slashed

* don't filter slashed for active balance

* don't overflow

* fix tests

* fix tests
2024-03-06 20:46:16 +00:00
Potuz
102518e106 pass justified=finalized in Prater (#13695)
* pass justified=finalized in Prater

* fix gazelle mess
2024-03-06 18:45:41 +00:00
james-prysm
e49ed4d554 keymanager api: lowercase statuses (#13696)
* cleanup

* adding test

* address small comment

* gaz
2024-03-06 16:30:17 +00:00
Manu NALEPA
21775eed52 Fix VC DB conversion when no proposer settings is defined and add Experimental flag in the --enable-minimal-slashing-protection help. (#13691)
* VC: Allow DB conversion without proposer settings.

* `enable-minimal-slashing-protection` flag: Add `Experimental warning`.
2024-03-06 14:48:18 +00:00
Radosław Kapka
ee9274a9bc REST VC metrics (#13588)
* REST VC metrics

* validator status is not used

* refactor

* remove test

* server-side

* server-side per endpoint

* cleanup

* add config endpoints

* add proper HTTP methods to endpoints

* initialize missing fields
2024-03-05 18:15:46 +00:00
Manu NALEPA
ef21d3adf8 Implement EIP-3076 minimal slashing protection, using a filesystem database (#13360)
* `EpochFromString`: Use already defined `Uint64FromString` function.

* `Test_uint64FromString` => `Test_FromString`

This test function tests more functions than `Uint64FromString`.

* Slashing protection history: Remove unreachable code.

The function `NewKVStore` creates, via `kv.UpdatePublicKeysBuckets`,
a new item in the `proposal-history-bucket-interchange`.

IMO there is no real reason to prefer `proposal` than `attestation`
as a prefix for this bucket, but this is the way it is done right now
and renaming the bucket will probably be backward incompatible.

An `attestedPublicKey` cannot exist without
the corresponding `proposedPublicKey`.

Thus, the `else` portion of code removed in this commit is not reachable.
We raise an error if we get there.

This is also probably the reason why the removed `else` portion was not
tested.

* `NewKVStore`: Switch items in `createBuckets`.

So the order corresponds to `schema.go`

* `slashableAttestationCheck`: Fix comments and logs.

* `ValidatorClient.db`: Use `iface.ValidatorDB`.

* BoltDB database: Implement `GraffitiFileHash`.

* Filesystem database: Creates `db.go`.

This file defines the following structs:
- `Store`
- `Graffiti`
- `Configuration`
- `ValidatorSlashingProtection`

This files implements the following public functions:
- `NewStore`
- `Close`
- `Backup`
- `DatabasePath`
- `ClearDB`
- `UpdatePublicKeysBuckets`

This files implements the following private functions:
- `slashingProtectionDirPath`
- `configurationFilePath`
- `configuration`
- `saveConfiguration`
- `validatorSlashingProtection`
- `saveValidatorSlashingProtection`
- `publicKeys`

* Filesystem database: Creates `genesis.go`.

This file defines the following public functions:
- `GenesisValidatorsRoot`
- `SaveGenesisValidatorsRoot`

* Filesystem database: Creates `graffiti.go`.

This file defines the following public functions:
- `SaveGraffitiOrderedIndex`
- `GraffitiOrderedIndex`

* Filesystem database: Creates `migration.go`.

This file defines the following public functions:
- `RunUpMigrations`
- `RunDownMigrations`

* Filesystem database: Creates proposer_settings.go.

This file defines the following public functions:
- `ProposerSettings`
- `ProposerSettingsExists`
- `SaveProposerSettings`

* Filesystem database: Creates `attester_protection.go`.

This file defines the following public functions:
- `EIPImportBlacklistedPublicKeys`
- `SaveEIPImportBlacklistedPublicKeys`
- `SigningRootAtTargetEpoch`
- `LowestSignedTargetEpoch`
- `LowestSignedSourceEpoch`
- `AttestedPublicKeys`
- `CheckSlashableAttestation`
- `SaveAttestationForPubKey`
- `SaveAttestationsForPubKey`
- `AttestationHistoryForPubKey`

* Filesystem database: Creates `proposer_protection.go`.

This file defines the following public functions:
- `HighestSignedProposal`
- `LowestSignedProposal`
- `ProposalHistoryForPubKey`
- `ProposalHistoryForSlot`
- `ProposedPublicKeys`

* Ensure that the filesystem store implements the `ValidatorDB` interface.

* `slashableAttestationCheck`: Check the database type.

* `slashableProposalCheck`: Check the database type.

* `slashableAttestationCheck`: Allow usage of minimal slashing protection.

* `slashableProposalCheck`: Allow usage of minimal slashing protection.

* `ImportStandardProtectionJSON`: Check the database type.

* `ImportStandardProtectionJSON`: Allow usage of min slashing protection.

* Implement `RecursiveDirFind`.

* Implement minimal<->complete DB conversion.

3 public functions are implemented:
- `IsCompleteDatabaseExisting`
- `IsMinimalDatabaseExisting`
- `ConvertDatabase`

* `setupDB`: Add `isSlashingProtectionMinimal` argument.

The feature addition is located in `validator/node/node_test.go`.
The rest of this commit consists in minimal slashing protection testing.

* `setupWithKey`: Add `isSlashingProtectionMinimal` argument.

The feature addition is located in `validator/client/propose_test.go`.

The rest of this commit consists in tests wrapping.

* `setup`: Add `isSlashingProtectionMinimal` argument.

The added feature is located in the `validator/client/propose_test.go`
file.

The rest of this commit consists in tests wrapping.

* `initializeFromCLI` and `initializeForWeb`: Factorize db init.

* Add `convert-complete-to-minimal` command.

* Creates `--enable-minimal-slashing-protection` flag.

* `importSlashingProtectionJSON`: Check database type.

* `exportSlashingProtectionJSON`: Check database type.

* `TestClearDB`: Test with minimal slashing protection.

* KeyManager: Test with minimal slashing protection.

* RPC: KeyManager: Test with minimal slashing protection.

* `convert-complete-to-minimal`: Change option names.

Options were:
- `--source` (for source data directory), and
- `--target` (for target data directory)

However, since this command deals with slashing protection, which has
source (epochs) and target (epochs), the initial option names may confuse
the user.

In this commit:
`--source` ==> `--source-data-dir`
`--target` ==> `--target-data-dir`

* Set `SlashableAttestationCheck` as an iface method.

And delete `CheckSlashableAttestation` from iface.

* Move helpers functions in a more general directory.

No functional change.

* Extract common structs out of `kv`.

==> `filesystem` does not depend anymore on `kv`.
==> `iface` does not depend anymore on `kv`.
==> `slashing-protection` does not depend anymore on `kv`.

* Move `ValidateMetadata` in `validator/helpers`.

* `ValidateMetadata`: Test with mock.

This way, we can:
- Avoid any circular import for tests.
- Implement once for all `iface.ValidatorDB` implementations
  the `ValidateMetadata`function.
- Have tests (and coverage) of `ValidateMetadata`in
  its own package.

The ideal solution would have been to implement `ValidateMetadata` as
a method with the `iface.ValidatorDB`receiver.
Unfortunately, golang does not allow that.

* `iface.ValidatorDB`: Implement ImportStandardProtectionJSON.

The whole purpose of this commit is to avoid the `switch validatorDB.(type)`
in `ImportStandardProtectionJSON`.

* `iface.ValidatorDB`: Implement `SlashableProposalCheck`.

* Remove now useless `slashableProposalCheck`.

* Delete useless `ImportStandardProtectionJSON`.

* `file.Exists`: Detect directories and return an error.

Before, `Exists` was only able to detect if a file exists.
Now, this function takes an extra `File` or `Directory` argument.
It detects either if a file or a directory exists.

Before, if an error was returned by `os.Stat`, the the file was
considered as non existing.
Now, it is treated as a real error.

* Replace `os.Stat` by `file.Exists`.

* Remove `Is{Complete,Minimal}DatabaseExisting`.

* `publicKeys`: Add log if unexpected file found.

* Move `{Source,Target}DataDirFlag`in `db.go`.

* `failedAttLocalProtectionErr`: `var`==> `const`

* `signingRoot`: `32`==> `fieldparams.RootLength`.

* `validatorClientData`==> `validator-client-data`.

To be consistent with `slashing-protection`.

* Add progress bars for `import` and `convert`.

* `parseBlocksForUniquePublicKeys`: Move in `db/kv`.

* helpers: Remove unused `initializeProgressBar` function.
2024-03-05 15:27:15 +00:00
Potuz
b6ce6c2eba Do not check parent weight on early FCU (#13683)
When a late block arrives and the beacon is proposing the next block, we
perform several checks to allow for the next block to reorg the incoming
late block.

Among those checks, we check that the parent block has been heavily
attested (currently 160% of the committee size).

We perform this check in these circumstances:
- When the late block arrives
- At 10 seconds into the slot
- At 0 seconds into the next slot (at proposing time)

The problem is that for blocks that arrive between 4 seconds and 10
seconds, the parent block will not have yet this expected weight since
attestations from the current committee were not imported yet, and thus
Prysm will send an FCU with payload attributes anyway at this time.

What happens is that Prysm keeps the EL building different blocks based
on different parents at the same time, when later in the next slot it
calls to propose, it will reorg the late block anyway and the EL would
have been computing a second payload uselessly.

This PR enables this check only when calling `ShouldOverrideFCU` after
10 seconds into the slot which we do only after having imported the
current attestations. We may want to actually remove this check entirely
from `ShouldOverrideFCU` and only keep it in `ProposerHead`.

Shout out to Anthithesis for reporting an issue that led to this
discoverly.
2024-03-05 15:07:39 +00:00
kasey
b3caaa9acc exit blob fetching for cp block if outside retention (#13686)
* exit blob fetching for cp block if outside retention

* regression test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-03-05 02:57:06 +00:00
james-prysm
d6fb8c29c9 refactoring how proposer settings load into validator client (#13645)
* refactoring how proposer settings load

* fixing tests and moving test data

* fixing linting and adding comments

* accidently removed a function, adding it back in

* fixing usage of dependency

* gaz

* fixing package visibility

* gaz

* iface config gaz

* adding visibility for db

* fix ineffectual assignment to err

* adding in log for when the builder is set but ignored due to no fee recipient

* Update config/validator/service/proposer_settings.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/validator/service/proposer_settings.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/validator.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/validator/service/proposer_settings.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/proposer/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/proposer/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/proposer/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/proposer/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/proposer/loader.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/validator/service/proposer_settings.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update config/util.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* some of the review feedback

* more review comments

* adding more test coverage

* Update config/proposer/loader.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* Update config/proposer/loader.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* Update config/proposer/loader.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* Update config/proposer/loader.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* updating based on feedback

* renaming variable

* fixing unhandled errors

* fixing tests

* gaz

* adding in gaslimit log

* fixing log

* some more review comments

* renaming and moving proposer settings file

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-03-04 15:12:11 +00:00
Manu NALEPA
3df7a1f067 VC: Improve logging in case of fatal error (#13681)
* VC: Display `FATAL` log in case of uncaught error.

* `initializeFromCLI`: Remove `Println`.
2024-03-04 09:28:49 +00:00
kasey
4c3dbae3c0 tests for origin blob fetching (#13667)
* tests for origin blob fetching

* Update beacon-chain/sync/initial-sync/service.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-03-02 15:11:22 +00:00
Nishant Das
68b78dd520 fix race (#13680) 2024-03-01 09:29:51 +00:00
Potuz
2e2ef4a179 Fix failed reorg log (#13679) 2024-02-29 15:23:29 +00:00
Nishant Das
b61d17731e Downgrade Level DB to Stable Version (#13671)
* downgrade level db

* fix current issues

* update geth

* Fix zstd build. The patch is now longer needed now that https://github.com/bazelbuild/rules_go/issues/3411 is fixed.

* Revert "update geth"

This reverts commit 2a7c51a952.

* change to hex

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-29 04:49:02 +00:00
james-prysm
6d3c6a6331 move setting route handlers to registration from start (#13676)
* moving route registration out of the start function and into registration

* moving where grpc is set
2024-02-28 11:30:31 +00:00
Nishant Das
f1615c4c88 Employ Dynamic Cache Sizes (#13640)
* dynamic cache sizes

* tests

* gosimple

* fix it

* add tests

* comments

* skip test

* Update beacon-chain/blockchain/receive_block_test.go

Co-authored-by: terence <terence@prysmaticlabs.com>

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
2024-02-28 10:46:52 +00:00
Preston Van Loon
87b127365f Update bazel-lib to include https://github.com/aspect-build/bazel-lib/pull/768 (#13675) 2024-02-27 21:24:44 +00:00
Manu NALEPA
5215ed03fd Set the log level for running on <network> as INFO. (#13670)
In case of custom ethereum network, keep the log as `WARN`.
2024-02-27 09:48:22 +00:00
james-prysm
0453d18395 small cleanup on functions (#13666) 2024-02-26 22:23:23 +00:00
kasey
0132c1b17d download checkpoint sync origin blobs in init-sync (#13665)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-02-26 22:00:15 +00:00
Radosław Kapka
d9d2ee75de Do not log zero sync committee messages (#13662) 2024-02-26 17:57:26 +00:00
Nishant Das
ddb321e0ce add changes (#13661) 2024-02-26 13:45:17 +00:00
Nishant Das
5735379963 Use a Validator Reader When Computing Unrealized Balances (#13656)
* employ a val reader to prevent constant copies

* clean it up and fix tests

* gaz

* radek's review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-24 12:10:00 +00:00
Nishant Das
1d5a09c05d Optimize Adding Dirty Indices (#13660)
* add it in

* add in test

* potuz's review
2024-02-24 10:08:17 +00:00
kasey
70e1b11aeb blob save fsync feature flag (#13652)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-02-23 23:14:35 +00:00
Dhruv Bodani
e100fb0c08 Add support for sync committee selections (#13633)
* add support for sync committee selections

* go mod tidy

* remove unused fields

* fix build

* fix build

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-23 13:53:42 +00:00
Nishant Das
789c3f8078 add changes (#13657) 2024-02-23 10:51:46 +00:00
Radosław Kapka
0b261cba5e Unify log fields (#13654)
* unify fields

* fix tests
2024-02-22 22:40:36 +00:00
Radosław Kapka
7a9608ea20 Normalize backfill logs/errors (#13642)
* Normalize backfill logs

* improve flag desc

* review
2024-02-22 12:32:32 +00:00
Nishant Das
f795e09ecf do not store it (#13637) 2024-02-22 02:08:40 +00:00
Nishant Das
e6a6365bdd Use Max Request Limit in Initial Sync (#13641)
* use max limit

* manu's review
2024-02-22 01:12:43 +00:00
kasey
4c66e4d060 avoid part path collisions with mem addr entropy (#13648)
* avoid part path collisions with mem addr entropy

* Regression test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-21 22:47:33 +00:00
Preston Van Loon
daad29d0de blob save: add better data checking for empty blob issues (#13647) 2024-02-21 21:57:43 +00:00
terence
9f67ad9496 Fix blob batch verifier pointer receiver (#13649) 2024-02-21 20:34:55 +00:00
terence
0ee0653a15 Remove unused bolt buckets (#13638) 2024-02-21 19:08:13 +00:00
Sammy Rosso
4ff91bebf8 Switch gomock library (#13639)
* Update gomock

* Update mockgen

* Gaz

* Go mod

* Cleanup

* Regenerate gomock

* Manually fix import
2024-02-21 18:37:17 +00:00
Radosław Kapka
f85e027141 Normalize filesystem/blob logs (#13644) 2024-02-21 17:34:57 +00:00
Manu NALEPA
e09ae75c9f Normalize checkpoint logs. (#13643) 2024-02-21 15:20:44 +00:00
Manu NALEPA
cb80d5ad32 Slasher: Reduce surrounding/surrounded attestations processing time (#13629)
* Improve package documentation.

* `processAttestations`: Improve logging.

* Add `Benchmark_checkSurroundVotes` benchmark.

* Implement `saveChunksToDisk` as remplacement of `saveUpdatedChunks`.

The idea is to open only on DB transaction for all validator chunk indexes instead of
one DB transaction per validator chunk index.

It saves the overhead due to transaction start/stop of the DB.

Result of `Benchmark_checkSurroundVotes`:
- Before this commit: 133 seconds
- After this commit: 5.05 seconds

* `LoadSlasherChunks` and `SaveSlasherChunks`: Batch.

* `loadChunks` ==> `loadChunksFromDisk`

* `updatedChunkByChunkIndex`: Don't update if `latestEpochWritten == currentEpoch `.

* `updatedChunkByChunkIndex`: Load all needed chunks once.

* `latestEpochWritten` ==> `latestEpochUpdated`.

* `checkSurroundVotes`: Dump to disk at most every `25_600` chunks.

* `SaveAttestationRecordsForValidators`: Batch.

* `batchSize`: Use as package const and add comment.
2024-02-21 15:12:37 +00:00
terence
24b029bbef Some clean ups on the sync package (#13636) 2024-02-20 21:32:13 +00:00
kasey
04dd60d1bf Fix pending block/blob zero peer edge case (#13625)
* pending broadcast err if missing blobs and 0 peers

* compute request first for len check

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-02-17 00:29:23 +00:00
Potuz
92303b610d add mainnet fork epoch (#13601)
* add mainnet fork epoch

* add sha

* gazelle

* gazelle screwed up

* Update spec test to v1.4.0-beta.7

* Update spec test to v1.4.0-beta.7-hotfix

* Gazelle

* compensate for lower request limits post-deneb

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-02-16 22:08:06 +00:00
Radosław Kapka
41d97a2a27 Remove /eth/v1/beacon/blocks/{block_id} (#13628) 2024-02-16 15:01:09 +00:00
Radosław Kapka
b842d8ce1d Remove leftover log (#13627) 2024-02-16 14:02:56 +00:00
Manu NALEPA
4030614df0 Slasher: Reduce cold start duration. (#13620)
* `getChunk` ==> `getChunkFromDatabase`.

* `loadChunks`: Rename variables.

* `Update`: Use explicit arguments.

* `detect_attestations.go`: Reduce abstraction layers.

* `loadAndUpdateChunks`: Change arguments order.

* `updatedChunkByChunkIndex`: Update all known validators in the chunk.

* `LastEpochWrittenForValidators`: Avoid avoidable `for`loop.

* `chunks.go`: Ensure implementations respect the interface.

* `LastEpochWrittenForValidators`: Stop considering lack of epoch as genesis epoch.

* `updatedChunkByChunkIndex`: Don't update latest updated epoch.

And add a bunch of tests.

* Improve slasher cold boot duration.

Before this commit, on a slasher cold boot (aka, without any db),
the `updatedChunkByChunkIndex` function looped for all validators
AND for all epochs between the genesis epoch and the current epoch.

This could take several dozen of minutes, and it is useless since the
min/max spans are actually a circular buffer with a limited lenght.
Cells of min/max spans can be overwritten (with the same value)
plenty of times.

After this commit, the `updatedChunkByChunkIndex` function loops
for all validators AND AT most 'historyLength' lenght.
Every cell of min/max spans are written AT MOST once.

Time needed for slasher boot goes from `O(nm)` to "only" `O(m)`, where:
- `n` is the number of epochs since the genesis.
- `m` is the number of validators.
2024-02-16 10:10:26 +00:00
terence
1ec745b88e Check deneb fork transition test folder is empty (#13626) 2024-02-15 22:08:06 +00:00
Dust
05b2795844 add BlockBatchLimit comment: Maximum 128 (#13502)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-02-15 20:00:18 +00:00
james-prysm
2d0fe20917 Blob API - invalid indicies should error instead of ignored (#13616)
* addressing ux issue when using API

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* fixing tests

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-15 19:54:42 +00:00
james-prysm
3a2734f249 removing show deposit data flag (#13618)
* removing show deposit data flag

* removing flag insted of deprecating

* fixing test

* fixing test
2024-02-15 19:49:50 +00:00
terence
50d1961e52 Remove safe slots to import flag (#13624) 2024-02-15 18:59:05 +00:00
terence
0dfe19c3dd Update aggregate atts t1 bucket values (#13607) 2024-02-15 18:36:30 +00:00
kasey
e5394fe081 backfill waits for init-sync before starting (#13623)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-02-15 17:42:36 +00:00
Radosław Kapka
f09fe4f038 Reorganize Prysm custom endpoints (#13621)
* Reorganize Prysm custom endpoints

* add deprecated paths

* review
2024-02-15 17:26:59 +00:00
Radosław Kapka
6c5351c3a2 Remove deprecated items (#13600)
* Remove deprecated items

* missed test

* add back deprecated fields

* Preston's review

* remove tests

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-02-15 15:23:31 +00:00
terence
5a66807989 Update to V5 (#13622)
* First take at updating everything to v5

* Patch gRPC gateway to use prysm v5

Fix patch

* Update go ssz

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-15 05:46:47 +00:00
kasey
bb66201c2c Backfill Blobs (#13595)
* retrieve and save blobs during backfill

* Update beacon-chain/sync/backfill/batch.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/sync/backfill/blobs.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/sync/backfill/metrics.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* make blobSync initialization path a little safer

* use bytes.Equal to avoid extra allocation

* stop using log.Fatal and other little cleanups

* wrap up post blob sync actions in batch mutator

* unit test coverage on verifier

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-14 20:58:51 +00:00
kasey
5de22d22bc metrics for count and size of blob files (#13614)
* metrics for count and size of blob files

* Update beacon-chain/db/filesystem/metrics.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* double check ssz extension

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-02-14 19:28:20 +00:00
Radosław Kapka
1eb373103a Remove /eth/v1/debug/beacon/states/{state_id} (#13619) 2024-02-14 17:52:20 +00:00
terence
b18026fc00 Rm unused import (#13615)
* Remove unused import in events.proto

* Remove unused import in beacon_lightclient.proto
2024-02-13 23:39:49 +00:00
Potuz
8a89173937 Data Race on the background FCU call (#13602) 2024-02-13 16:22:53 +00:00
Manu NALEPA
3d13c69ef3 Slasher: Fix surrounded votes false negative (#13612)
* `LastEpochWrittenForValidators`: Use golang if.

* `SaveLastEpochsWrittenForValidators`: Refactor.

* `SaveLastEpochsWrittenForValidators`: Fix when `epochByValIndex` > `batchSize`.

Before this commit, `TestStore_LastEpochWrittenForValidators` works if `validatorsCount <= 10000`
and stops working if `validatorsCount > 10000`.

* Slasher: Detect surrounded votes in multiple batches.

Fixes https://github.com/prysmaticlabs/prysm/issues/13591.
2024-02-13 15:50:33 +00:00
Joel Rousseau
f2f10e7381 add tests for beaconcha baseURL (#13308)
* add tests for different environments

* sort import

* remove goerli for prater

* clean

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-13 13:44:51 +00:00
Nishant Das
9aa55b47f3 Clean Up Deposit Processing Logs (#13611)
* add it

* increase it
2024-02-13 10:41:07 +00:00
Preston Van Loon
c4c6b47d9b validator/exit: Print other testnet beaconcha.in URLs (#13610) 2024-02-12 19:36:58 +00:00
Manu NALEPA
06a5548424 Slasher: Fixes double votes false positive and attester slashings duplication. (#13596)
* `Test_processAttestations`: Remove duplicated tests.

* Sort indexed attestations by data root.

* `processAttestations`: Don't return duplicate slashings anymore.

Fix https://github.com/prysmaticlabs/prysm/issues/13592.

* `AttesterDoubleVote`: Rename fields.

* Detect double votes in different batches.

In order to do that:
1. Each attestation of the batch is tested against the other attestations of the batch.
2. Each attestation of the batch is tested against the content of the database.
2. Attestations are saved into the database.

Fixes https://github.com/prysmaticlabs/prysm/issues/13590.
2024-02-12 17:35:22 +00:00
terence
5e74c798d4 Use Gwei for value in get header log (#13608) 2024-02-12 15:14:58 +00:00
Nishant Das
7b955c94ec Reduce Lookahead Steps Parameter (#13599)
* reduce lookahead steps

* test
2024-02-12 14:39:42 +00:00
terence
256a05bfd5 Remove deprecated flags: optional engine and registration (#13606) 2024-02-10 04:59:00 +00:00
Preston Van Loon
03068ba781 db: clear blobs when using --clear-db or --force-clear-db (#13605)
* Call Close() as part of ClearDB

* Add method to clear blob storage

* Clear blob storage when clearing DB
2024-02-10 03:52:30 +00:00
Nishant Das
5df8b83a05 Validate Range Availibility (#13587)
* fix it

* check range avail

* add test cases

* add checks

* kasey's review

* gaz
2024-02-09 23:41:08 +00:00
terence
db653b8863 Remove deprecated build block parallel flag (#13539) 2024-02-09 22:13:51 +00:00
Manu NALEPA
af203efa0c Slasher: Refactor and add tests (#13589)
* `helpers.go`: Improve naming consistency.

* `detect_attestations.go`: Improve readability.

* `receive.go`: Add `attsQueueSize` in log message.

* `checkSlashableAttestations`: Improve logging.

`avgBatchProcessingTime` is not displayed any more if not batch is
processed.

* `loadChunks`: Use explicit `chunkKind` and `chunkIndices`.

* `getChunk`: Use specific `chunkIndex` and `chunkKind`.

* `validatorIndicesInChunk` -> `validatorIndexesInChunk`.

* `epochUpdateForValidator`: Use explicit arguments.

* `getChunk`: Change order of arguments.

* `latestEpochWrittenForValidator`: Use `ok` parameter.

So the default value is not any more considered as the absence of
value.

* `applyAttestationForValidator`: Use explicit arguments.

* `updateSpans`: Use explicit arguments.

* `saveUpdatedChunks`: Use explicit arguments.

* `checkSurrounds`: Use explicit arguments.

We see here that, previously, in `checkSlashableAttestations`,
`checkSurrounds` was called with the default value of `slashertypes`: `MinSpan`.

Now, we set it expliciterly at `MinSpan`, which may explicit a bug.

* `epochUpdateForValidator`: Set modified by the function argument first.

* `applyAttestationForValidator`: Set mutated argument `chunksByChunkIdx`first.

* `applyAttestationForValidator`: Rename variables.

* `Test_processQueuedAttestations`: Fix test.

Two tests were actually exactly the same.

* `updateSpans`: Keep happy path in the outer scope.

Even if in this case the "happy" path means slashing.

* `checkSurrounds`: Rename variable.

* `getChunk`: Avoid side effects.

It adds a few lines for callers, but it does not modify any more
arguments and it does what it says: getting a chunk.

* `CheckSlashable`: Flatten.

* `detect_attestations_test.go`: Simplify.

* `CheckSlashable`: Add error log in case of missing attestation.

* `processQueuedAttestations`: Extract a sub function.

So testing will be easier.

* `processAttesterSlashings` and `processProposerSlashings`: Improve.

* `processAttesterSlashings`: Return processed slashings.

* `createAttestationWrapper`: Rename variables.

* `signingRoot` ==> `headerRoot` or `dataRoot`.

Before this commit, there is two typse of `signing root`s floating around.
- The first one is a real signing root, aka a hash tree root computed from an object root and
a domain. This real signing root is the object ready to be signed.
- The second one is a "false" signing root, which is actually just the hash tree root of an object. This object is either the `Data` field of an attestation, or the `Header` field of a block.

Having 2 differents objects with the same name `signing root` is quite confusing.
This commit renames wrongly named `signing root` objects.

* `createAttestationWrapper` => `createAttestationWrapperEmptySig`.

So it's clear for the user that the created attestation wrapper has an empty signature.

* Implement `createAttestationWrapper`.

* `processAttestations`: Return processed attester slashings.

* Test `processAttestations` instead of `processQueuedAttestations`.

By testing `processAttestations` instead of `processQueuedAttestations`, we get rid of a lot of tests fixtures, including the 200 ms sleep.

The whole testing duration is shorter.

* `Test_processAttestations`: Allow multiple steps.

* `Test_processAttestations`: Add double steps tests.

Some new failing tests are commented with a corresponding github issue.

* `NextChunkStartEpoch`: Fix function comment.

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* `chunks.go`: Avoid templating log messages.

* `checkSlashableAttestations`: Simplify duration computation.

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-09 21:02:18 +00:00
terence
5582c558c6 Remove deprecated aggregate parallel flag (#13538) 2024-02-09 20:18:36 +00:00
terence
573d9739ea Remove disable vectorized htr (#13537) 2024-02-09 18:28:13 +00:00
terence
bb18fa3f71 Remove deprecated late reorg flag (#13536) 2024-02-09 17:15:08 +00:00
Aditya Asgaonkar
6a605e6b6d Fork choice filter changes (#13464)
* implement confirmation rule prerequisite - f.c. filter changes

* update tests

* update WORKSPACE for spec v1.4.0-beta.6

* run bazel gazelle

* Fix consensus_spec sha256

* drift also forkchoice time when drifting the service on tests

* update minimal kzg_commitment_inclusion_proof_depth.size

* fix mock engine client

* remove unnecessary helper & revert test changes

* revert change of proof size in minimal preset

* fix tests

* fix loader test

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2024-02-09 17:01:44 +00:00
Nishant Das
a0787e2379 Handle Syncing Executon Client (#13597)
* fix it

* manu's review

* fix failing tests
2024-02-09 09:07:48 +00:00
Nishant Das
621bda068d Suppress Unwanted P2P Errors (#13598)
* supress unwanted errors

* gaz

* mod
2024-02-09 05:13:57 +00:00
Radosław Kapka
91504eb95a Improve vc logs (#13573)
* duties

* atts

* revert some changes

* revert timeTillDuty

* Manu's review

* Revert "Auxiliary commit to revert individual files from 6806ca9fbe18101f58ccb40fe191c61c183735a8"

This reverts commit 0820c870d2627950179b0edf7ce62ee4fa4a03a3.

* remove trash

* more reivew

* making Manu happy

* test fixes
2024-02-08 18:24:03 +00:00
Sammy Rosso
5afb1255fe Add /eth/v1/beacon/deposit_snapshot endpoint (#13514)
* Add endpoint

* Uncomment in InitializeRoutes

* Add test

* Add 404

* Add more checks

* Test improvements

* Ssz

* Add ssz tags

* Add DepositSnapshot to bazel

* Fix tests

* Fix max size

* Resolve conflicts

* Revert untouched code

* Fix test + review

* Lint

* Oops

* Preston + Radek' review

* Only return 3 finalized roots

* Change to deposit contract depth

* Radek' review

* Gaz

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-02-07 22:53:08 +00:00
Manu NALEPA
9d6160e112 Slasher: Remove unused RPC. (#13594) 2024-02-07 21:11:58 +00:00
james-prysm
1383546999 Beacon API: get blob fix retention cases (#13585)
* fixing the handling for certain cases

* fixing tests

* Update beacon-chain/rpc/eth/blob/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* update comment based on review

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-06 22:31:17 +00:00
Justin Traglia
01116f7f82 Fix a few minor nits in protobuf definitions (#13512) 2024-02-06 21:17:32 +00:00
Thabokani
692ebd313f Fix typos in doc (#13583)
Signed-off-by: Thabokani <149070269+Thabokani@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-02-06 10:18:21 +00:00
Nishant Das
6fa656c1ee Add Sync Checker (#13580)
* fix it

* add it in

* typo

* fix tests

* fix tests

* export and add test

* preston's review
2024-02-06 02:34:30 +00:00
Dhruv Bodani
55a29a4670 Implement beacon committee selections (#13503)
* implement beacon committee selections

* fix build

* fix lint

* fix lint

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/beacon_committee_selections.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* move beacon committee selection structs to validator module

* fix bazel build files

* add support for POST and GET endpoints for get state validators query

* add a handler to return error from beacon node

* move beacon committee selection to validator top-level module

* fix bazel

* re-arrange fields to fix lint

* fix TestServer_InitializeRoutes

* fix build and lint

* fix build and lint

* fix TestSubmitAggregateAndProof_Distributed

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-02-05 15:43:51 +00:00
Potuz
e2e7e84a96 Get the right head state when proposing a failed reorg (#13579)
* Get the right head state when proposing a failed reorg

* add unit test

* split logic
2024-02-05 13:40:35 +00:00
terence
91b0a93df7 Enhance EL block height log (#13582) 2024-02-05 01:52:01 +00:00
Preston Van Loon
8839015312 docker: Add coreutils to docker images (#13564)
* Add coreutils to docker images

* add coreutils dependencies

* Add a prysmaticlabs.com/uploads backup of the deb files

* Run gazelle and fix issues

* Remove broken tar, change http_archive deps to debian_archive, remove http mirrors in favor of snapshot

* Add comments about which deps are required by other deps
2024-02-03 19:21:21 +00:00
terence
61ab4bf7ca Rename block by range request log (#13561) 2024-02-03 19:20:04 +00:00
Radosław Kapka
e3ce1bde45 Move API structs to api module (#13577) 2024-02-03 11:57:01 +00:00
Nishant Das
9d1189b222 Do Not Cache For Non Active Public Keys (#13581)
* fix it

* clean up
2024-02-03 05:19:54 +00:00
KeienWang
74f5452a64 Fix typo in [beacon-chain/cache/depositsnapshot/deposit_cache_test.go]: Corrected a spelling error. (#13532)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-02-03 05:14:32 +00:00
Nishant Das
ea1204d3c7 Fix Slashing Gossip Checks (#13574)
* fix it

* add for proposals too
2024-02-02 23:13:22 +00:00
Radosław Kapka
d9ac69752b Return consensus block value in Wei (#13575)
* Return consensus block value in Wei

* Return consensus block value in Wei

* review
2024-02-02 18:17:40 +00:00
terence
52af63f25a Revise blob sidecar not found log (#13571)
* Update blob sidecar not found log

* Use fields
2024-02-01 20:48:59 +00:00
james-prysm
2dad245bc8 handle slice out of range (#13568)
* handle slice out of range

* adding some tests
2024-02-01 16:59:40 +00:00
Potuz
9a9990605c Update Gohashtree to v0.0.4-beta (#13569)
* Update Gohashtree to v0.0.4-beta

* go mod tidy

* go mod tidy

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-01 15:42:56 +00:00
james-prysm
2cddb5ca86 fixing jwt auth checks (#13565) 2024-02-01 15:13:52 +00:00
Nishant Das
73ce28c356 make it the default (#13556) 2024-01-31 10:27:26 +00:00
Manu NALEPA
7a294e861e Beacon node slasher improvement (#13549)
* Slasher: Ensure all gorouting are stopped before running `Stop` actions.

Fixes #13550.
In tests, `exitChan` are now useless since waitgroup are used to wait
for all goroutines to be stopped.

* `slasher.go`: Add comments and rename some variables. - NFC

* `detect_blocks.go`: Improve. - NFC

- Rename some variables.
- Add comments.
- Use second element of `range` when possible.

* `chunks.go`: Remove `_`receivers. - NFC

* `validateAttestationIntegrity`: Improve documentation. - NFC

* `filterAttestations`: Avoid `else`and rename variable. - NFC

* `slasher.go`: Fix and add comments.

* `SaveAttestationRecordsForValidators`: Remove unused code.

* `LastEpochWrittenForValidators`: Name variables consistently. - NFC

Avoid mixes between `indice(s)`and `index(es)`.

* `SaveLastEpochsWrittenForValidators`: Name variables consistently. - NFC

* `CheckAttesterDoubleVotes`: Rename variables and add comments. - NFC

* `schema.go`: Add comments. - NFC

* `processQueuedAttestations`: Add comments. - NFC

* `checkDoubleVotes`: Rename variable. - NFC

* `Test_processQueuedAttestations`: Ensure there is no error log.

* `shouldNotBeSlashable` => `shouldBeSlashable`

* `Test_processQueuedAttestations`: Add 2 test cases:
- Same target with different signing roots
- Same target with same signing roots

* `checkDoubleVotesOnDisk` ==> `checkDoubleVotes`.

Before this commit, `checkDoubleVotes` did two tasks:
- Checking if there are any slashable double votes in the input
  list of attestations with respect to each other.
- Checking if there are any slashable double votes in the input
  list of attestations with respect to our database.

However, `checkDoubleVotes` is called only in
`checkSlashableAttestations`.

And `checkSlashableAttestations` is called only in:
- `processQueuedAttestations`, and in
- `IsSlashableAttestation`

Study of case `processQueuedAttestations`:
---------------------------------------------
In `processQueuedAttestations`, `checkSlashableAttestations`
is ALWAYS called after
`Database.SaveAttestationRecordsForValidators`.

It means that, when calling `checkSlashableAttestations`,
`validAtts` are ALREADY stored in the DB.

Each attestation of `validAtts` will be checked twice:
- Against the other attestations of `validAtts` (the portion of
  deleted code)
- Against the content of the database.

One of those two checks is redundent.
==> We can remove the check against other attestations in `validAtts`.

Study of case `Database.SaveAttestationRecordsForValidators`:
----------------------------------------------------------------
In `Database.SaveAttestationRecordsForValidators`,
`checkSlashableAttestations` is ALWAYS called with a list of
attestations containing only ONE attestation.

This only attestaion will be checked twice:
- Against itself, and an attestation cannot conflict with itself.
- Against the content of the database.

==> We can remove the check against other attestations in `validAtts`.

=========================

In both cases, we showed that we can remove the check of attestation
against the content of `validAtts`, and the corresponding test
`Test_checkDoubleVotes_SlashableInputAttestations`.

* `Test_processQueuedBlocks_DetectsDoubleProposals`: Wrap proposals.

So we can add new proposals later.

* Fix slasher multiple proposals false negative.

If a first batch of blocks is sent with:
- validator 1 - slot 4 - signing root 1
- validator 1 - slot 5 - signing root 1

Then, if a second batch of blocks is sent with:
- validator 1 - slot 4 - signing root 2

Because we have two blocks proposed by the same validator (1) and for
the same slot (4), but with two different signing roots (1 and 2), the
validator 1 should be slashed.

This is not the case before this commit.
A new test case has been added as well to check this.

Fixes #13551

* `params.go`: Change comments. - NFC

* `CheckSlashable`: Keep the happy path without indentation.

* `detectAllAttesterSlashings` => `checkSurrounds`.

* Update beacon-chain/db/slasherkv/slasher.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update beacon-chain/db/slasherkv/slasher.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* `CheckAttesterDoubleVotes`: Keep happy path without indentation.

Well, even if, in our case, "happy path" mean slashing.

* 'SaveAttestationRecordsForValidators': Save the first attestation.

In case of multiple votes, arbitrarily save the first attestation.
Saving the first one in particular has no functional impact,
since in any case all attestations will be tested against
the content of the database. So all but the first one will be
detected as slashable.

However, saving the first one and not an other one let us not
to modify the end to end tests, since they expect the first one
to be saved in the database.

* Rename `min` => `minimum`.

Not to conflict with the new `min` built-in function.

* `couldNotSaveSlashableAtt` ==> `couldNotCheckSlashableAtt`

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-01-31 09:49:14 +00:00
james-prysm
258123341e add a log and update size for promptui (#13542) 2024-01-30 17:19:31 +00:00
Preston Van Loon
224b136737 Revert "set limit to multiple of burst for goerli" (#13552)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-30 06:10:12 +00:00
Nishant Das
3ed4866eec Makes Our New Deposit Trie The Default (#13555)
* make 4881 the default

* fix failed build
2024-01-30 05:15:52 +00:00
kasey
373c853d17 set limit to multiple of burst for goerli (#13544)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-01-27 22:12:08 +00:00
terence
23b0718b5f Add metric for data availability wait time (#13534)
* Add metric for data availability wait time

* Kasey's feedback

* Kasey's feedback
2024-01-26 18:17:25 +00:00
terence
3a9854145c Correct metrics from ns to ms (#13540) 2024-01-26 17:43:30 +00:00
Radosław Kapka
1b70d2b566 Fetch unaggregated atts in GetAggregateAttestation (#13533) 2024-01-26 17:08:58 +00:00
Nishant Das
59b310a221 make it the same (#13531) 2024-01-26 05:35:27 +00:00
Nishant Das
22b6d1751d Enable Backfill in E2E (#13524)
* enable backfill for devmode

* enable backfill

* gaz

* move to its own package

* fix panic

* fix bug

* gaz

* kasey's review
2024-01-26 04:37:41 +00:00
Potuz
9c13d47f4c fix off by one (#13529) 2024-01-26 00:05:56 +00:00
Justin Traglia
835dce5f6e Enable wastedassign linter & fix findings (#13507)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-01-25 17:07:48 +00:00
james-prysm
c4c28e4825 fixing small typo in error messages (#13525) 2024-01-25 04:56:17 +00:00
Radosław Kapka
c996109b3a Return payload value in Wei from /eth/v3/validator/blocks (#13497)
* Add value in Wei to execution payload

* simplify how payload is returned

* test fix

* fix issues

* review

* fix block handlers
2024-01-24 20:58:35 +00:00
terence
e397f8a2bd Skip origin root when cleaning dirty state (#13521)
* Skip origin root when cleaning dirty state

* Clean up
2024-01-24 17:22:50 +00:00
Radosław Kapka
6438060733 Clear cache everywhere in tests of core helpers (#13509) 2024-01-24 16:11:43 +00:00
Nishant Das
a2892b1ed5 clean up validate beacon block (#13517) 2024-01-24 05:48:15 +00:00
Nishant Das
f4ab2ca79f lower it (#13516) 2024-01-24 01:28:36 +00:00
kasey
dbcf5c29cd moving some blob rpc validation close to peer read (#13511)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-01-23 22:54:16 +00:00
james-prysm
c9fe53bc32 Blob API: make errors more generic (#13513)
* make api response more generic

* gaz
2024-01-23 20:07:46 +00:00
terence
8522febd88 Add Holesky Deneb Epoch (#13506)
* Add Holesky Deneb Epoch

* Fix fork version

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Fix config

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-01-23 19:29:17 +00:00
james-prysm
75a28310c2 fixing route to match specs (#13510) 2024-01-23 18:04:03 +00:00
kasey
1df173e701 Block backfilling (#12968)
* backfill service

* fix bug where origin state is never unlocked

* support mvslice states

* use renamed interface

* refactor db code to skip block cache for backfill

* lint

* add test for verifier.verify

* enable service in service init test

* cancellation cleanup

* adding nil checks to configset juggling

* assume blocks are available by default

As long as we're sure the AvailableBlocker is initialized correctly
during node startup, defaulting to assuming we aren't in a checkpoint
sync simplifies things greatly for tests.

* block saving path refactor and bugfix

* fix fillback test

* fix BackfillStatus init tests

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-01-23 07:54:30 +00:00
terence
3187a05a76 Align aggregated att gossip validations (#13490)
* Align aggregated att gossip validations

* Feedback on reusing existing methods

* Nishant's feedback

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-23 04:37:06 +00:00
Justin Traglia
4e24102237 Fix minor issue in blsToExecChange validator (#13498)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-23 03:26:57 +00:00
james-prysm
8dd5e96b29 re-enabling jwt on keymanager API (#13492)
* re-enabling jwt on keymanager API

* adding tests

* Update validator/rpc/intercepter.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* handling error in test

* remove debugging logs

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-01-22 22:16:10 +00:00
james-prysm
4afb379f8d cleanup duties naming (#13451)
* updating some naming to reflect changes to duties

* fixing unit tests

* fixing more tests
2024-01-22 16:58:25 +00:00
Nishant Das
5a2453ac9c Add Debug State Transition Method (#13495)
* add it

* lint
2024-01-22 14:46:20 +00:00
Nishant Das
e610d2a5de fix it (#13496) 2024-01-22 14:26:14 +00:00
Preston Van Loon
233aaf2f9e e2e: Fix multiclient lighthouse flag removal (#13494) 2024-01-21 21:11:11 +00:00
Nishant Das
a49bdcaa1f fix it (#13493) 2024-01-20 16:15:38 +00:00
Gaki
bdd7b2caa9 chore: typo fix (#13461)
* messsage

* cancellation
2024-01-20 01:07:17 +00:00
terence
8de0e3804b Update Sepolia Deneb fork epoch (#13491) 2024-01-19 18:47:07 +00:00
Ying Quan Tan
bfb648067b Re-enable Slasher E2E Test (#13420)
* re-enable e2e slashing test #12415

* refactored slashing evaluator

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-19 04:44:27 +00:00
terence
852db1f3eb Remove debug setting highest slot log (#13488) 2024-01-19 04:25:15 +00:00
Nishant Das
5d3663ef8d update lighthouse and tests (#13470) 2024-01-19 03:46:36 +00:00
Radosław Kapka
a608630727 Add Inactivity field ro attestation rewards (#13382) 2024-01-18 18:51:35 +00:00
Mario Vega
37739b4193 fix blobsidecar json tag for commitment inclusion proof (#13475)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-18 17:43:43 +00:00
james-prysm
4d2067dbae bugfix: ssz post-requests should check content type not accept (#13482)
* updating post requests that accept ssz to check content type instead of accept header

* radek's review comments to make things more clear
2024-01-18 17:41:31 +00:00
Nishant Das
fc05e306dd Allow Pcli to Run State Transitions Easily (#13484)
* add all this in

* gaz

* add flag
2024-01-18 14:44:06 +00:00
Radosław Kapka
204de13c86 REST VC: Subscribe to Beacon API events (#13453)
* Revert "Revert "REST VC: Subscribe to Beacon API events  (#13354)" (#13428)"

This reverts commit 8d092a1113.

* change logic

* review

* test fix

* fix critical error

* merge flag check

* change error msg

* return on errors

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-18 14:27:41 +00:00
terence
f3ef1b64d6 Enhance block by root log (#13472) 2024-01-18 13:43:10 +00:00
terence
c3dbfa66d0 Change blob latency metrics to ms (#13481) 2024-01-17 23:28:42 +00:00
terence
93aba997f4 Move checking of attribute empty earlier (#13465) 2024-01-17 18:42:56 +00:00
Potuz
79bb7efbf8 Check init sync before getting payload attributes (#13479)
* Check init sync before getting payload attributes

This PR adds a helper to forkchoice to return the delay of the latest
imported block. It also adds a helper with an heuristic to check if the
node is during init sync. If the highest imported node was imported with
a delay of less than an epoch then the node is considered in regular
sync. If on the other hand, in addition the highest imported node is
more than two epochs old, then the node is considered in init Sync.

The helper to check this only uses forkchoice and therefore requires a
read lock. There are four paths to call this

1) During regular block processing, we defer a function to send the
   second FCU call with attributes. This function may not be called at
all if we are not regularly syncing
2) During regular block processing, we check in the path
   `postBlockProces->getFCUArgs->computePayloadAttributes` the payload
attributes if we are syncing a late block. In this case forkchoice is
already locked and we add a call in `getFCUArgs` to return early if not
regularly syncing
3) During handling of late blocks on `lateBlockTasks` we simply return
   early if not in regular sync (This is the biggest change as it takes
a longer FC lock for lateBlockTasks)
4) On Attestation processing, in UpdateHead, we are already locked so we
   just add a check to not update head on this path if not regularly
syncing.

* fix build

* Fix mocks
2024-01-17 15:39:28 +00:00
terence
87b53db3b4 Capitalize Aggregated Unaggregated Attestations Log (#13473) 2024-01-17 13:30:31 +00:00
terence
fe431b9201 Use correct HistoricalRoots (#13477) 2024-01-17 08:14:32 +00:00
james-prysm
790a09f9b1 Improve wait for activation (#13448)
* removing timeout on wait for activation, instead switched to an event driven approach

* fixing unit tests

* linting

* simplifying return

* adding sleep for the remaining slot to avoid cpu spikes

* removing ifstatement on log

* removing ifstatement on log

* improving switch statement

* removing the loop entirely

* fixing unit test

* fixing manu's reported issue with deletion of json file

* missed change around writefile at path

* gofmt

* fixing deepsource issue with reading file

* trying to clean file to avoid deepsource issue

* still getting error trying a different approach

* fixing stream loop

* fixing unit test

* Update validator/keymanager/local/keymanager.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* fixing linting

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-01-16 17:04:54 +00:00
Manu NALEPA
46387a903a getLegacyDatabaseLocation: Change message. (#13471)
* `getLegacyDatabaseLocation`: Change message.

* Update validator/node/node.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-16 11:29:36 +00:00
Nishant Das
6a65e07684 Add Spans to Core Validator Methods (#13467)
* add traces

* gaz
2024-01-16 07:52:46 +00:00
Potuz
abef94d7ad do not check optimistic status if cached attestation (#13462)
* do not check optimistic status if cached attestation

* Gazelle

* Gazelle again

* fix nil panics

* more nil checks

* more nil checks

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2024-01-15 18:50:33 +00:00
Manu NALEPA
99a8d0bac6 Validator client - Improve readability - NO FUNCTIONAL CHANGE (#13468)
* Improve `NewServiceRegistry` documentation.

* Improve `README.md`.

* Improve readability of `registerValidatorService`.

* Move `log` in `main.go`.

Since `log` is only used in `main.go`.

* Clean Tos.

* `DefaultDataDir`: Use `switch` instead of `if/elif`.

* `ReadPassword`: Remove unused receiver.

* `validator/main.go`: Clean.

* `WarnIfPlatformNotSupported`: Add Mac OSX ARM64.

* `runner.go`: Use idiomatic `err` handling.

* `waitForChainStart`: Avoid `chainStartResponse`mutation.

* `WaitForChainStart`: Reduce cognitive complexity.

* Logs: `powchain` ==> `execution`.
2024-01-15 14:46:54 +00:00
Preston Van Loon
b585ff77f5 Fix port logging in bootnode (#13457) 2024-01-15 04:38:22 +00:00
Nishant Das
1ff5a43385 Add the Abillity to Defragment the Beacon State (#13444)
* Defragment head state

* change log level

* change it to be more efficient

* add flag

* add tests and clean up

* fix it

* gosimple

* Update container/multi-value-slice/multi_value_slice.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's review

* unlock it

* remove from fc lock

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2024-01-13 05:44:02 +00:00
dependabot[bot]
0cfbddc980 Bump github.com/quic-go/quic-go from 0.39.3 to 0.39.4 (#13445)
* Bump github.com/quic-go/quic-go from 0.39.3 to 0.39.4

Bumps [github.com/quic-go/quic-go](https://github.com/quic-go/quic-go) from 0.39.3 to 0.39.4.
- [Release notes](https://github.com/quic-go/quic-go/releases)
- [Changelog](https://github.com/quic-go/quic-go/blob/master/Changelog.md)
- [Commits](https://github.com/quic-go/quic-go/compare/v0.39.3...v0.39.4)

---
updated-dependencies:
- dependency-name: github.com/quic-go/quic-go
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Ran gazelle

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-01-12 18:12:19 +00:00
Manu NALEPA
22a484c45e Fixes issues when running validator client with the --web flag and non existing validator.db file AND/OR prysm-wallet-v2 directory. (#13460)
* `getLegacyDatabaseLocation`: Add tests.

* `getLegacyDatabaseLocation`: Handle `c.wallet == nil`.

* `saveAuthToken`: Create parent directory if needed.
2024-01-12 15:53:27 +00:00
terence
6ddafe1159 Delete invalid blob at block processing (#13456)
* Delete invalid blob at block processing

* Fix test
2024-01-12 08:09:45 +00:00
qinlz2
b8c5af665f [3/5] light client events (#13225)
* add http streaming light client events

* expose ForkChoiceStore

* return error in insertFinalizedDeposits

* send light client updates

* Revert "return error in insertFinalizedDeposits"

This reverts commit f7068663b8c8b3a3bf45950d5258011a5e4d803e.

* fix: lint

* fix: patch the wrong error response

* refactor: rename the JSON structs

* fix: LC finalized stream return correct format

* fix: LC op stream return correct JSON format

* fix: omit nil JSON fields

* chore: gazzle

* fix: make update by range return list directly based on spec

* chore: remove unneccessary json annotations

* chore: adjust comments

* feat: introduce EnableLightClientEvents feature flag

* feat: use enable-lightclient-events flag

* chore: more logging details

* chore: fix rebase errors

* chore: adjust data structure to save mem

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* refactor: rename config EnableLightClient

* refactor: rename feature flag

* refactor: move helper functions to helper pkg

* test: fix broken unit tests

---------

Co-authored-by: Nicolás Pernas Maradei <nicolas@polymerlabs.org>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-01-11 18:38:59 +00:00
Radosław Kapka
2875ce6ee1 Use a single rest handler (#13446) 2024-01-11 16:03:35 +00:00
Manu NALEPA
a883ae2a76 BN: Move --db-backup-output-dir as a deprecated flag. (#13450) 2024-01-11 14:11:36 +00:00
Preston Van Loon
3a2b486bde Bazel 7.0.0 (#13321) 2024-01-10 15:34:11 +00:00
terence
283e09569d Remove old blob types (#13438)
* Remove old types

* Gen

* Remove old types

* Gen

* Fix lint

* Rm unused key

* Kasey's comment
2024-01-10 09:38:06 +00:00
Preston Van Loon
69723b4a77 Update go to 1.21.6 (#13440) 2024-01-10 09:37:40 +00:00
psr
4fe6834ba5 http endpoint cleanup (#13432)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-01-09 23:48:43 +00:00
Preston Van Loon
98e3f2b80f sort static analyzers, add more, fix violations (#13441) 2024-01-09 23:29:36 +00:00
Enrico Del Fante
2aef7a3ec5 Update teku's bootnode (#13437) 2024-01-09 22:28:41 +00:00
Brandon Liu
c41a54be9d fix metric for exited validator (#13379)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 22:15:53 +00:00
Justin Traglia
7e65378f63 Check sidecar index in BlobSidecarsByRoot response (#13180)
* Check sidecar index in BlobSidecarsByRoot response

* Remove unnecessary MaxBlobsPerBlock check
2024-01-09 22:14:56 +00:00
Justin Traglia
cf606e3766 Only process blocks which haven't been processed (#13442)
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-01-09 22:14:03 +00:00
Justin Traglia
703cfc5819 Initialize exec payload fields and enforce order (#13372)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 21:49:35 +00:00
GoodDaisy
c6ebe157a6 Fix typos (#13435) 2024-01-09 21:03:36 +00:00
Preston Van Loon
a3cc81a048 Add nil check for head in IsOptimistic (#13439) 2024-01-09 19:40:26 +00:00
Nishant Das
75bbeb61cc Add Detailed Multi Value Metrics (#13429)
* add it

* pingo

* gaz

* remove pingo

* fix for old forks
2024-01-09 05:16:03 +00:00
kasey
5cea6bebb8 minimize syscalls in pruning routine (#13425)
* minimize syscalls in pruning routine

* terence feedback

* Update beacon-chain/db/filesystem/pruner.go

Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>

* pr feedback

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-01-08 22:31:16 +00:00
Potuz
28596d669b Use proposer index cache for blob verification (#13423)
* Use proposer index cache for blob verification

* add unit test

* Fix test

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2024-01-07 03:24:07 +00:00
kasey
0e043d55b4 VerifiedROBlobs in initial-sync (#13351)
* Use VerifiedROBlobs in initial-sync

* Update beacon-chain/das/cache.go

Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>

* Apply suggestions from code review

comment fixes

Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>

* fix lint error from gh web ui

* deepsource fixes

* more deepsource

* fix init wiring

* mark blobless blocks verified in batch mode

* move sig check after parent checks

* validate block commitment length at start of da check

* remove vestigial locking

* rm more copy-locksta

* rm old comment

* fail the entire batch if any sidecar fails

* lint

* skip redundant checks, fix len check

* assume sig and proposer checks passed for block

* inherits most checks from processed block

* Assume block processing handles most checks

* lint

* cleanup unused call and gaz

* more detailed logging for e2e

* fix bad refactor breaking non-finalized init-sync

* self-review cleanup

* gaz

* Update beacon-chain/verification/blob.go

Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>

* terence and justin feedback

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-01-06 23:47:09 +00:00
Radosław Kapka
8d092a1113 Revert "REST VC: Subscribe to Beacon API events (#13354)" (#13428)
This reverts commit e68b2821c1.
2024-01-06 21:36:42 +00:00
kasey
073c4edc5f use ROForkchoice in blob verifier (#13426)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2024-01-06 19:39:03 +00:00
terence
d055db1c31 Unlock forkchoice store if attribute is empty (#13427)
* Unlock forkchoice store if attribute is empty

* Better version
2024-01-06 07:32:56 +00:00
Nishant Das
a974627258 Make Aggregating In Parallel The Permanent Default (#13407)
* make it the permanent default

* gaz
2024-01-06 07:29:06 +00:00
Potuz
67dccc5e43 Break out several helpers from postBlockProcess (#13419)
* Break out several helpers from `postBlockProcess`

In addition fix a bug found by @terencechain where we should use a slot
context instead of the parent context in the second FCU call.

* Remove calls for tracked proposer

getPayloadAttribute already takes care of this
Also compute correctly the time into voting window

* call with attributes only when incoming block is canonical

* check for empty payload instead of only nil

* add unit tests

* move log for non-canonical block

* return early if the incoming block does not change head

* Pass fcuArgs as arguments

* lint
2024-01-06 02:29:07 +00:00
terence
ff06e08274 Prune dangling blob (#13424)
* Prune dangling blob

* Fix test

* Kasey's feedback

* Preston's feedback

* Use warning, fix test
2024-01-05 22:29:57 +00:00
james-prysm
d3d25e3ae5 proposer and attester slashing sse (#13414)
* wip

* adding in event notifiers for slashing events

* fixing tests
2024-01-05 15:27:50 +00:00
Nishant Das
929e9ddf4c enable it (#13421) 2024-01-05 05:23:22 +00:00
Nishant Das
7c0e79d432 Make New Engine Methods The Permanent Default (#13406)
* make them the default

* gaz

* fix tests
2024-01-05 04:38:04 +00:00
terence
3c1c0b3c00 Update blob pruning log (#13417) 2024-01-04 18:02:19 +00:00
james-prysm
d439e6da74 adding builder boost factor to get block v3 (#13409)
* adding builder boost factor to functions

* gaz

* fixing linting

* fixing unit tests

* gaz

* addressing review comments

* fixing tests

* addressing review feedback

* gaz

* changing log based on review
2024-01-04 17:25:18 +00:00
Radosław Kapka
e68b2821c1 REST VC: Subscribe to Beacon API events (#13354)
* Initial code for head event streaming

* handle events and error

* keepalive event

* tests

* generate new mock

* remove single case select

* cleanup

* explain eventByteLimit

* use 2 channels in test

* review

* more review

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-04 17:14:45 +00:00
Potuz
cfef8f4676 Don't hardcode 4 seconds in forkchoice (#13416) 2024-01-04 16:49:16 +00:00
terence
9709412511 Use Afero Walk for Pruning Blob (#13410)
* Use Afero walk

* Return err

* Use wrap

* More err to the end

* Fix loop
2024-01-04 16:41:00 +00:00
james-prysm
7781eb60f4 Add rpc trigger for blob sidecar event (#13411)
* adding missed rpc trigger for blob sidecar event

* fixing unit tests

* moving event feed to after receive blob call to prioritize db
2024-01-04 14:22:24 +00:00
Potuz
396b8bf970 Simplify fcu 4 (#13403)
* send two FCU when proposing

* compute voting window at runtime
2024-01-04 13:43:57 +00:00
Nishant Das
d5107942a1 update it (#13415) 2024-01-04 11:22:23 +00:00
terence
bd4a520013 Initialize blob storage without pruning (#13412) 2024-01-04 05:56:38 +00:00
Sammy Rosso
a0ff1351a0 Fix batch pruning errors (#13355)
* Add compareAndSwap

* Update lastPrunedEpoch before prune

* Fix and test

* Remove debug log

* Kasey' review

* Fix tests

* Address Kasey's comments

* Fix prune before slot

* Rename

* Fix bad test

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2024-01-03 20:52:07 +00:00
Nishant Das
7e6fd5fd8b Make Reorging Of Late Blocks The Permanent Default (#13405)
* make it the permanent default

* gaz

* fix merge conflicts
2024-01-03 14:46:58 +00:00
Nishant Das
d984210baa fix it (#13404) 2024-01-03 13:54:18 +00:00
Potuz
31c72672d7 Remove the getPayloadAttribute call from updateForkchoiceWithExecution (#13402)
* Remove the getPayloadAttribute call from updateForkchoiceWithExecution

* Move log
2024-01-03 12:43:40 +00:00
Potuz
8c1e180dd1 Simplify fcu 2 (#13400)
* change getPayloadAttribute signature

* Unify different FCU arguments
2024-01-02 22:45:55 +00:00
Manu NALEPA
886d76fe7c Refactor validator client help. (#13401)
* Define `cli.App` without mutation.

No functional change.

* `usage.go`:  Clean `appHelpTemplate`.

No functional change is added.
Modifications consist in adding prefix/suffix `-` to improve readability of
the template without adding new lines in template inference.

We now see some inconsistencies of the template:
- `if .App.Version` is around the `AUTHOR` section.
- `if .App.Copyright` is around both `COPYRIGHT` and `VERSION` sections.
- `if len .App.Authors` is around nothing.

* `usage.go`: Surround version and author correctly.

* `usage.go`: `AUTHOR` ==> `AUTHORS`

* `usage.go`: `GLOBAL` --> `global`.

* `--grpc-max-msg-size`: Remove double default.

* VC: Standardize help message.

- Flags help begin with a capital letter and end with a period.
- If a flag help begins with a verb, it is conjugated.
- Expermitemtal, danger etc... mentions are between parenthesis.

* VC help message: Wrap too long lines.
2024-01-02 18:02:28 +00:00
Potuz
a602acf492 Remove getPayloadAttributes from FCU call (#13399) 2024-01-02 17:37:18 +00:00
terence
1b6547de6a Add Goerli Deneb Fork Epoch (#13390)
* Add deneb fork epoch

* Fix test
2024-01-02 15:31:57 +00:00
Nishant Das
88685bb3bd Fix Up Builder Evaluator (#13395)
* fix it up

* fix evaluator

* fix evaluator again

* fix it

* gaz
2024-01-02 10:40:26 +00:00
Nishant Das
2319b7d4bd increase params (#13398) 2024-01-02 10:19:59 +00:00
Manu NALEPA
82b2840d68 --validatorS-registration-batch-size (add s) (#13396) 2024-01-02 09:52:14 +00:00
Manu NALEPA
cf221d0f4c Validator client: Always use the --datadir value. (#13392)
Fix https://github.com/prysmaticlabs/prysm/issues/13391
2024-01-02 09:24:24 +00:00
Preston Van Loon
0956e3a657 Update libp2p/go-libp2p-asn-util to v0.4.1 (#13370)
* Update go-libp2p-asn-util to v0.4.1

* fix go mod

---------

Co-authored-by: nisdas <nishdas93@gmail.com>
2024-01-02 07:30:50 +00:00
terence
351ed1c511 Check kzg commitment count from builder (#13394) 2024-01-02 06:50:23 +00:00
Potuz
9809f5ac77 Simplify fcu 1 (#13387)
* Remove unsafe proposer indices cache

* Simplify FCU #1

This PR starts the process of gradually simplifying FCU
It removes the responsibility of getting the state and block from this
function and informing if head has changed. It is only called when the
imported block has actually become head.

* Add a call to FCU in edge cases
2023-12-30 12:20:20 +00:00
Potuz
cff5e2b5fe Remove unsafe proposer indices cache (#13385) 2023-12-30 12:20:02 +00:00
terence
dd15f9e0cc Rewrite ProposeBlock endpoint (#13380)
* Init

* Tests

* Init

* Tests

* Radek's feedback

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* More Radek's feedback

* Potuz feedback

* Use inline copy

* Fix conflict

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-29 23:32:58 +00:00
terence
1c9ded4684 Remove blind field from block type (#13389)
* Init

* Init

* Fix tests
2023-12-29 21:28:19 +00:00
Potuz
d4cc6fcf4a update shuffling caches before calling FCU on epoch boundaries (#13383)
* update shuffling caches before calling FCU on epoch boundaries

* Terence's review
2023-12-28 15:19:09 +00:00
Radosław Kapka
49c16f1a71 Return SignedBeaconBlock from ReadOnlySignedBeaconBlock.Copy (#13386) 2023-12-28 08:55:45 +00:00
terence
e70b606e78 Replace validator count with validator indices in update fee recipient log (#13384)
* Add validator count to updated fee recipient address log

* Add validator count to updated fee recipient address log

* Replace
2023-12-27 16:46:15 +00:00
Potuz
0e8b37c317 Log value of local payload when proposing (#13381) 2023-12-27 14:43:32 +00:00
Potuz
e80db9554d Use advanced epoch cache when preparing proposals (#13377) 2023-12-27 12:42:51 +00:00
Radosław Kapka
d0bf03e863 Simplify error handling for JsonRestHandler (#13369)
* Simplify error handling for `JsonRestHandler`

* POST

* reduce complexity

* review feedback

* uncomment route

* fix rest of tests
2023-12-22 22:39:20 +00:00
Potuz
b7e0819f00 refactor Payload Id caches (#12987)
* init

- getLocalPayload does not use the proposer ID from the cache but takes
  it from the block

- Fixed tests in blockchain package
- Fixed tests in the RPC package
- Fixed spectests

EpochProposers takes 256 bytes that can be avoided to be copied, but
this optimization is not clear to be worth it.

assginmentStatus can be optimized to use the cached version from the
TrackedValidatorsCache

We shouldn't cache the proposer duties when calling getDuties but when
we update the epoch boundary instead

* track validators on prepare proposers

* more rpc tests

* more rpc tests

* initialize grpc caches

* Add back fcu log

Also fix two existing bugs wrong parent hash on pre Capella and wrong
blockhashes on altair

* use beacon default fee recipient if there is none in the vc

* fix validator test

* radek's review

* push always proposer settings even if no flag is specified in the VC

* Only register with the builder if the VC flag is set

Great find by @terencechain

* add regression test

* Radek's review

* change signature of registration builder
2023-12-22 18:47:51 +00:00
Radosław Kapka
7d64104003 block publishing (#13376) 2023-12-22 18:15:00 +00:00
Nishant Das
b1e8a9ea3d fix it with regression (#13375) 2023-12-22 12:33:23 +00:00
Radosław Kapka
cc1028ca3c Use deneb key for deneb state in saveStatesEfficientInternal (#13374)
* Use deneb key for deneb state in saveStatesEfficientInternal

* move reset out of inner loop
2023-12-21 18:14:04 +00:00
Nishant Das
233f4d99a2 Update Libp2p To v0.32.1 and Go to v1.21.5 (#13304)
* update libp2p

* fix tests

* fix tests

* fix build

* update to go v1.21

* workflow

* workflow again

* update ci

* update golangci

* disable quic
2023-12-21 16:09:54 +00:00
terence
a068f3877e Use block value correctly when proposing a block (#13368)
* Use block value correctly

* Fix the function
2023-12-21 03:00:34 +00:00
james-prysm
856907d760 Small encoding fixes on logs and http error code change (#13345)
* fixing some bad encodings

* changing http error to align with other clients

* fixing unit test
2023-12-20 18:18:55 +00:00
Sammy Rosso
c6801df05a Fix total pruned metric + add to logging (#13367) 2023-12-19 16:15:01 +00:00
SQL TRIGGER
bc7b15b04e typo fix (#13357) 2023-12-19 16:03:40 +00:00
Nishant Das
eb713d1177 Refactor Network Config Into Main Config (#13364)
* change parameters to main config

* add more changes

* change to accepted format

* fix changes in config

* gaz

* fix test

* fix test again
2023-12-19 14:59:30 +00:00
Preston Van Loon
844b2c6602 Add error wrapping to blob initialization errors (#13366) 2023-12-19 14:55:26 +00:00
Potuz
9efaa832cd use different keys for the proposer indices cache (#13272)
* use different keys for the proposer indices cache

* Add a way to get the proposer indices from a checkpoint

* fix fuzzing tests

* use htr instead of body root

* move comment
2023-12-19 13:14:55 +00:00
Radosław Kapka
e9d26c61d7 Do not skip mev boost in v3 block production endpoint (#13365) 2023-12-19 12:46:17 +00:00
Sammy Rosso
374d77f437 Blob filesystem metrics (#13316)
* Add metrics

* Replace counter with gauge

* Preston's comments

* Remove hardcoded number

* Count blob files

* Fix count order

* Fixes

* Cleanup

* Add blob bucket

* Update beacon-chain/node/node.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Rename

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-18 18:24:07 +00:00
Justin Traglia
1f6d1d1852 For golangci-lint, enable all by default (#13353)
* For golangci-lint, enable all by default

* Use latest golangci-lint here too

* Use v1.55.2 instead of latest

* Remove usestdlibvars from list

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
2023-12-18 18:20:55 +00:00
terence
0eff83cb9d Use a cache of one entry to build attestation (#13300)
* Use a cache of one entry to build attestation

* Gazelle

* Enforce on RPC side

* Rm unused var

* Potuz feedback, dont use pointer

* Fix tests

* Init fetcher

* Add in-progress

* Add back missing lock

* Potuz feedback

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-18 16:12:43 +00:00
Justin Traglia
ffe2f6b732 Enable mirror linter and fix findings (#13342)
* Enable mirror linter and fix findings

* Use latest version of golangci-lint

* Use v1.55.2 instead of latest

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-18 12:16:26 +00:00
terence
d57bca97a5 Check builder header kzg commitment (#13358) 2023-12-18 06:14:59 +00:00
Nishant Das
b45a6664be Enable Deneb For E2E Scenario Tests (#13317)
* fix all cases

* update web3signer

* current progress

* fix it finally

* push it back to capella

* remove hard-coded forks

* fix failing tests

* gaz

* fix dumb bug

* fix bad test setup

* change back
2023-12-16 11:37:44 +00:00
Justin Traglia
c56abfb840 Enable usestdlibvars linter and fix findings (#13339)
Co-authored-by: terence <terence@prysmaticlabs.com>
2023-12-15 19:21:54 +00:00
Preston Van Loon
d70f477b1e Fix docker image version strings in CI (#13356) 2023-12-15 19:15:51 +00:00
Preston Van Loon
db096488b0 fixing sa4006 (#13350) 2023-12-15 16:49:27 +00:00
Radosław Kapka
344e68b81b Use SkipMevBoost properly during block production (#13352)
* fix bugs

* tests

* name fix
2023-12-15 16:14:42 +00:00
Justin Traglia
1962cca69e Fix error string generation for missing commitments (#13338) 2023-12-15 04:03:45 +00:00
Justin Traglia
4a374435c0 Enable errname linter and fix findings (#13341) 2023-12-15 03:26:48 +00:00
David Theodore
0fde4a22e1 reordered blob validation (#13347) 2023-12-15 02:46:12 +00:00
terence
62ecc0d177 Add more color to sending blob by range req log (#13349) 2023-12-15 02:43:16 +00:00
Justin Traglia
97dfec84f6 Handle potential error from newBlockRangeBatcher (#13344) 2023-12-15 02:28:07 +00:00
terence
53bc96844e Move pruning log to after retention check (#13348) 2023-12-15 00:49:29 +00:00
terence
ddcf0c18dc Excluse DA wait time for chain processing time (#13335)
* Excluse DA wait time for chain processing time

* Rename
2023-12-14 22:46:48 +00:00
james-prysm
45a2746d0e Builder API: Fix max field check on toProto function (#13334)
* fixing field param used in ToProto function

* fxing test to pass

* making blobs empty in test
2023-12-14 03:03:00 +00:00
Preston Van Loon
09f3df309d Remove rules_docker, make multiarch images canonical (#13324)
* Remove rules_docker

* Update base image
2023-12-13 23:31:58 +00:00
Potuz
96df81d5c5 Hook to slot stream instead of block stream on the VC (#13327)
* Hook to slot stream instead of block stream on the VC

* Implement StreamSlots in the BN

* mock update

* fix tests

* don't return from stream

* Terence's review

* deepsource second complain

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2023-12-13 23:13:56 +00:00
terence
c47c52152b Enhance Pruning Logs (#13331)
* Log prunning info

* Added start log

* Log prunning info

* Added start log
2023-12-13 19:24:47 +00:00
james-prysm
4cbe144a6c CLI: fixing account import ux bugs (#13328)
* fixing account import checking wallet twice, and adding sub folder search with a depth of 2

* removing uneeded check

* fixing unit test

* adding reset cache to fix potential flake

* improving test based on feedback
2023-12-13 17:11:32 +00:00
Justin Traglia
52b9b65adb Add sanity checks for bundle from builder (#13319)
* Add sanity checks for bundle from builder

* Add more checks to BlobsBundle.ToProto()

* Fix minor typo

* Fix tests & add new ones

* Add tests for ToProto

* Add "not" to error message

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-13 15:54:00 +00:00
Radosław Kapka
ea59b1ec71 Increase buffer of events channel (#13329) 2023-12-13 15:37:45 +00:00
Radosław Kapka
175c484c44 Uncomment e2e flakiness (#13326) 2023-12-13 12:50:13 +00:00
Nishant Das
8aaab86987 fix it (#13325) 2023-12-13 11:01:01 +00:00
Preston Van Loon
381116a3e8 Fix missing testnet versions. Issue #13288 (#13323) 2023-12-12 21:44:14 +00:00
Sammy Rosso
3d61fd0436 Blob filesystem add pruning during blob write (#13275)
* Add prune during write

* Fix merge errors

* Add test

* Add test timeout

* Gaz

* Check prune at midpoint

* Fix slot number

* More checks
2023-12-12 21:27:15 +00:00
james-prysm
b19d24c581 Remove signed block requirement from no-verify functions (#13314)
* removing fake wrappers

* fixing conficts and missed tests

* fixing more conflicts

* addressing missed unit test

* fixing nogo error

* fixing more unit tests

* fixing more tests
2023-12-12 20:18:40 +00:00
Radosław Kapka
8387088a52 Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-12 19:46:19 +00:00
Potuz
ce7452c97a update spectests to 1.4.0-beta.5 (#13318)
* update spectests to 1.4.0-beta.5

* add spec config
2023-12-12 18:27:48 +00:00
james-prysm
5e56b5fdd7 Beacon APIs: re enabling blob events (#13315)
* re enabling blob events

* terence's comments

* Update beacon-chain/rpc/eth/events/events_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-12 15:58:11 +00:00
Nishant Das
bfaba378f6 activate deneb (#13311) 2023-12-12 08:04:55 +00:00
Sammy Rosso
3bd116db16 Blob filesystem add pruning at startup (#13253)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Add pruning at startup

* Add nil check

* Fix merge errors

* Fix test

* Fix test

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-11 17:08:52 +00:00
terence
7d2ddaee43 Test improvement TestValidateVoluntaryExit_ValidExit (#13313) 2023-12-11 08:01:56 +00:00
terence
122a7782ff Initialize blob storage for initial sync service (#13312) 2023-12-11 07:52:07 +00:00
terence
9b1b6f9be6 Use verified blob for gossip checks (#13294)
* Use blob verifier for gossip rules

* Fixing tests

* Fix lint

* Mocks

* Trying Kasey's rec

* mock verifier init workaround

* Add more tests

* Reset deneb epoch for exit test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-11 00:37:45 +00:00
Delweng
0eb08a4f96 beacon-chain/rpc: use BalanceAtIndex instead of Balances to reduce memory copy (#13279)
* beacon-chain/rpc: use BalanceAtIndex instead of Balances

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: stream use BalanceAtIndex is sufficient

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: fix commit review

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: http2 -> httputil

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-10 00:04:45 +00:00
Radosław Kapka
bdfa06ed65 Simplify post-evaluation in Beacon API evaluator (#13309)
* Simplify post-evaluation in Beacon API evaluator

* revert slottime

* remove unused error
2023-12-09 07:54:31 +00:00
Manu NALEPA
a94f2b93e3 filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
* `filterAndCacheActiveKeys`: Add test cases

- Validator is in unknown status (to be fitered out)
- Validator is in pending status,
   with activation period > current period (to be filtered out)
- Validator is in pending status,
   with activation period == current period (to be kept)

* `filterAndCacheActiveKeys`: Keep exiting keys

Initially:
-------
If a validator is in exiting (so, with status==EXITING != ACTIVE) state,
it will be filtered out by the `filterAndCacheActiveKeys` function.
The validator won't be registered to the beacon node.

If this exiting validator has to propose a block:
- the block will be proposed using local block building only.
- the fee recipient will be the one set in the beacon node.

(Additionally, if the beacon node Prysm without any
fee recipient defined at the beacon node level, the fee recipient
will default on the `0x00000...` burn address.)

This commit modifies the `filterAndCacheActiveKeys` function
by stopping filtering out exiting validators.
2023-12-09 07:53:08 +00:00
Radosław Kapka
4c47756aed HTTP endpoints cleanup (#13251)
* remove validation package

* structs cleanup

* merge with apimiddleware removal

* more validation and Bls capitalization

* builder test fix

* use strconv for uint->str conversions

* use DecodeHexWithLength

* use exact param names

* rename http package to httputil

* change conversions to fmt.Sprintf

* handle query paramsd and route variables

* spans and receiver name

* split structs, move bytes helper

* missing ok check

* fix reference to indexed failure

* errors fixup

* add godoc to helper

* fix BLS casing and chainhead ref

* review

* fix import in tests

* gzl
2023-12-08 20:37:20 +00:00
Nishant Das
440841d565 only run it in the middle of an epoch (#13303) 2023-12-08 15:14:01 +00:00
Preston Van Loon
ff99616833 Fix staticcheck violations (#13301)
* Fix violations of sa2002

* Fix violations of sa4005

* Fix violations of sa4010

* Fix violations for sa4023

* Comment on commented static checks
2023-12-08 13:07:52 +00:00
Preston Van Loon
f537a98fcd Add staticchecks to bazel builds (#13298)
* Update staticcheck to latest

* Add static checks while ignoring for third party / external stuff

* Added a hack to keep go mod happy.

* disable SA2002

* Pin go mod tidy checker image to golang:1.20-alpine
2023-12-08 05:42:55 +00:00
Radosław Kapka
cee38660c7 Gracefully handle unknown validator index in the REST VC (#13296)
* Gracefully handle unknown validaor index in the REST VC

* add apostrophes
2023-12-08 04:30:50 +00:00
james-prysm
481d77bfde APIs: reusing grpc cors middleware for rest (#13284)
* reusing grpc cors middleware for rest

* addressing radek's comments

* Update api/server/middleware.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing to recommended name

* fixing naming

* fixing rename on test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-08 04:24:18 +00:00
Nishant Das
590317553c Support New Subnet Backbone (#13179)
* add in changes

* fix it up

* fix test

* gaz

* lint

* add back

* fix tests

* fix it

* fix tests

* add lib

* fix it
2023-12-08 04:07:48 +00:00
terence
68b7d1009e Update README.md (#13302) 2023-12-08 04:07:10 +00:00
james-prysm
b5b8825cc8 Beacon API: fix get blob returns 500 instead of empty (#13297)
* fix blob api, should return empty if no indicies were found

* fixing small bug with slice
2023-12-07 22:33:26 +00:00
Justin Traglia
382b8b23c2 Ensure partial blob is deleted if there's an error (#13292)
* Ensure partial blob is deleted if there's an error

* Add debug log if file is removed
2023-12-07 20:52:16 +00:00
kasey
40a3ebab91 initialize sig cache for verification.Initializer (#13295)
* initialize sig cache for verification.Initializer

* gaz

* lint

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 20:14:01 +00:00
james-prysm
83af9a5694 Beacon API: update Deneb endpoints after removing blob signing (#13235)
* making needed changes to beacon API based on removal of blobsidecar from block contents

* fixing tests and reverting some changes to be addressed later

* fixing generated code from protos

* gaz

* fixing get blob handler and adding blob storage to the blob service

* updating unit tests

* WIP

* wip tests

* got tests passing but needs cleanup

* removing gomod and gosum changes

* fixing more tests

* fixing more tests

* fixing more tests

* gaz

* moving some proto types around

* removing unneeded unit test

* fixing proposer paths

* adding more tests

* fixing more tests

* improving more unit tests

* updating one blob only unit test

* changing arguments of buildBlobSidecar

* reverting a change based on feedback

* terence's review items

* fixing test based on new develop changes

* radek's comments

* addressed more comments from radek

* adding in blobs to test data

* fixing casing in test

* removing extra line

* fixing issue from bad merge

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* moving core getblob business logic to blocker based on radek's comment

* fixing mock blocker

* gaz

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-07 17:37:11 +00:00
Nishant Das
6a45323ab7 only run metrics for canonical blocks (#13289) 2023-12-07 11:03:23 +00:00
kasey
4008ea736f Verify roblobs (#13245)
* scaffolding for verification package

* WIP blob verification methods

* lock wrapper for safer forkchoice sharing

* more solid cache and verification designs; adding tests

* more test coverage, adding missing cache files

* clearer func name

* remove forkchoice borrower (it's in another PR)

* revert temporary interface experiment

* lint

* nishant feedback

* add comments with spec text to all verifications

* some comments on public methods

* invert confusing verification name

* deep source

* remove cache from ProposerCache + gaz

* more consistently early return on error paths

* messed up the test with the wrong config value

* terence naming feedback

* tests on BeginsAt

* lint

* deep source...

* name errors after failure, not expectation

* deep sooource

* check len()==0 instead of nil so empty lists work

* update test for EIP-7044

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 02:36:25 +00:00
Justin Traglia
4e4fb9ad52 Split blob pruning into two funcs (#13285) 2023-12-06 23:39:02 +00:00
kasey
737e0e0d3a Use functional options for --blob-retention-epochs (#13283)
* blob retention period functional opts

* missed unstaged change

* missed other init after cleardb

* fix ineffassign

* fix dup import

* config failsafe for tests

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-06 20:20:34 +00:00
Potuz
604c82626f Allow requests for old target roots (#13281) 2023-12-06 17:11:34 +00:00
Nishant Das
e1a3852f08 push up the defaults (#13278) 2023-12-06 16:46:46 +00:00
Preston Van Loon
a40cc40edf CI: Add merge queue events trigger for github workflows (#13282) 2023-12-06 16:13:13 +00:00
james-prysm
0e3c1d42f6 Beacon API: routes unit test (#13276)
* refactoring to add a routes unit test for rest handlers

* gaz

* updating names for functions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 15:17:13 +00:00
Radosław Kapka
f41e603e5a Simplify Beacon API evaluator (#13265)
* Simplify Beacon API evaluator

* remove comment

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 14:54:33 +00:00
Radosław Kapka
28c3330375 Don't fetch duties for unknown keys (#13269)
* Don't fetch duties for unknown keys

* test
2023-12-06 14:18:45 +00:00
Nishant Das
cb465086e8 Fix Optimistic Sync Evaluator (#13262)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 13:41:47 +00:00
Nishant Das
cdd7958739 Fix Domain Data Caching (#13263)
* fix domain data caching

* fix locking of domain data

* preston's review
2023-12-06 12:04:19 +00:00
Nishant Das
97a522827b Bump Up Gossip Queue Size (#13277) 2023-12-06 09:49:26 +00:00
Preston Van Loon
b84b795f23 Relax file permissions check on existing directories (#13274) 2023-12-05 19:39:45 -06:00
Sammy Rosso
f40b8583f7 Blob filesystem: prune blobs (#13147)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Fix linter errors

* Fix retentionSlot calculation

* Move + use MaxEpochsToPersistBlobs

* Remove unused ctx

* Prestons suggestion

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Rename

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-05 21:07:34 +00:00
Preston Van Loon
a6b6a938de blobstorage: Improve mkdirall error (#13271) 2023-12-05 19:57:08 +00:00
Brandon Liu
c78d698d89 Add --jwt-id flag (#13218)
* add jwt-id flag

* optimize unit test for jwt-id

* Add jwt-id to help text

* gofmt

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-05 19:02:25 +00:00
Nishant Das
705e98e3c3 no need to hash it again (#13261) 2023-12-05 07:59:01 -03:00
kasey
ce2344301c forkchoice.Getter wrapper with locking wrappers (#13244)
* forkchoice.Getter wrapper with locking wrappers

* comments

* lint

* only expose fast fc getters

* potuz feedback re rlock

* update mocks for new fc method

* appease deepsource

* add missing exported func comment

* yeet errors to make the linter happy

* even more devious _discard

* rm TargetRoot

* derp

* handle nil error in _discard

* deep source

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 21:01:39 +00:00
Manu NALEPA
1112e01c06 Make Prysm VC compatible with the version v5.3.0 of the slashing protections interchange tests. (#13232)
* `TestStore_GenesisValidatorsRoot_ReadAndWrite`: Make all test cases independents.

In a test with multiple test cases, each test case should be independents.
(aka: Removing test case `A` should not impact test case `B`)

* `SaveGenesisValidatorsRoot`: Allow to overwrite the genesis validator root if the root is the same.

* `ProposalHistoryForSlot`: Add `signingRootExists`

Currently, it is not possible with `ProposalHistoryForSlot` to know if a
proposal is stored with and `0x00000....` signing root or with an empty
signing root. Both cases result to `proposalExists == true` and
`signingRoot == 0x00000`.

This commit adds a new return boolean: `signingRootExists`.

If a proposal has been saved with a `0x00000...` signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `true`, and
- `signingRoot` is set to `0x00000...`

If a proposal has been saved with an empty signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `false`, and
- (`signingRoot` is set to `0x00000...`)

* `ImportStandardProtectionJSON`: When importing EIP-3076 Slashing Protection Interchange Format, do not filter any more slashable keys.
Note: Those keys are still saved into the black-listed public keys list.

There is two reason not to do so:
- The EIP-3076 test cases do not know about Prysm's internal black-listed public keys list.
  Tests will expect, without looking into this internal black-listed public keys list,
  to deny a further signature. If we filter these keys from the DB (even if we keep them
  into the black-listed keys list), then some tests will fail.
- If we import a interchange file containing slashable keys and we filter them, then,
  if we re-export the DB, those slashing offences won't appear in the exported interchange
  file.

* `transformSignedBlocks`: Store an 0-len byte slice

When importing an EIP-3076 interchange format, and when no
signing root is specified into the file, we currently store a
`0x00000.....` signing root.

In such a case, instead storing `0x00000...`, this commit stores
a 0-len byte array, so we can differentiate real `0x000.....` signing
root and no signing-root at all.

* `slashableProposalCheck`: Manage lack of sign root

Currently, `slashableProposalCheck` does not really make a difference
between a `0x0000.....` signing root and a missing signing root.

(Signing roots can be missing when importing an EIP-3076 interchange
file.)

This commit differentiate, for  `slashableProposalCheck`, `0x0000....`
signing root and a missing signing root.

* `AttestationRecord.SigningRoot`: ==> `[]byte`

When importing attestations from EIP-3076 interchange format,
the signing root of an attestation may be missing.

Currently, Prysm consider any missing attestation signing root as
`0x000...`.
However, it may conflict with signing root which really are equal to
`0x000...`.

This commit transforms `AttestationRecord.SigningRoot` from `[32]byte` to
`[]byte`, and change the minimal set of functions (sic) to support this
new type.

* `CheckSlashableAttestation`: Empty signing root

Regarding slashing roots, 2 attestations are slashable, if:
- both signing roots are defined and differs, or
- one attestation exists, but without a signing root

* `filterSlashablePubKeysFromAttestations`: Err sort

Rergarding `CheckSlashableAttestation`, we consider that:
- If slashable == NotSlashable and err != nil, then CheckSlashableAttestation
failed.
- If slashable != NotSlashable, then err contains the reason why the attestation
is slashable.

* `setupEIP3076SpecTests`: Update to `v5.3.0`

This commit:
- Updates the version of EIP-3076 tests to `v.5.2.1`.
- Setups on anti-slashing DB per test case, instead per step.

* `ImportStandardProtectionJSON`: Reduce cycl cmplxt

* `AttestationHistoryForPubKey`: copy signing root

BoltDB documentation specifies:
| Byte slices returned from Bolt are only valid during a transaction.
| Once the transaction has been committed or rolled back then the memory
| they point to can be reused by a new page or can be unmapped
| from virtual memory and you'll see an unexpected fault address panic
| when accessing it.
2023-12-04 17:10:32 +00:00
terence
243bcb03ce Fix FFG LMD Consistency Check (Option 2) (#13258)
* Fix FFG LMD Consistency Check with TargetRootForSlot

* Add test, removed  implementation

* convert to epoch and fix self target

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-04 15:31:18 +00:00
Radosław Kapka
a0ca4a67b0 Remove API Middleware (#13243)
* remove api/gateway/apimiddleware

* fix errors in api/gateway

* remove beacon-chain/rpc/apimiddleware

* fix errors in api/client/beacon

* fix errors in validator/client/beacon-api

* fix errors in beacon-chain/node

* fix errors in validator/node

* fix errors in cmd/prysmctl/validator

* fix errors in testing/endtoend

* fix all other code

* remove comment

* fix tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 11:55:21 +00:00
Preston Van Loon
b68a4e12aa Update bazel and other CI improvements (#13246)
* Update bazel to 6.4.0, review flags

* Remove problematic/slow targets

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-01 22:20:54 +00:00
kasey
c010601f3b Initialize cancellable root context in main.go (#13252)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-01 21:16:06 +00:00
james-prysm
394bd1786a HTTP validator API: beacon and account endpoints (#13191)
* fixing squashing changes, migrates beacon , account, and auth endpoints on validator client

* adding accounts endpoints

* fixing tests and query endpoints

* adding auth endpoint and fixing unit tests

* removing unused files and updating node file to skip gRPC

* ineffectual assignment fix

* rolling back a change to fix e2e

* fixing issues with ui

* updating with webui version 2.0.5

* updating package name flag in readme

* removing restore assets functions

* adding nomemcopy flag to see if vulenerability scan passes

* making data non compressed to avoid copy vulnerability

* Update beacon-chain/rpc/eth/shared/structs_validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating site_data, and skipping static analysis on file

* adding back deprecation comment notice

* updating workflows to ignore generated

* addressing radek comments

* missed a conversion

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2023-12-01 20:40:09 +00:00
Potuz
461af4baa6 Add test helpers to produce commitments and proofs (#13242)
* Add test helpers to produce commitments and proofs

* go mod tidy

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-01 19:51:19 +00:00
Sammy Rosso
7a70305935 Blob filesystem: delete blobs (#13233)
* Add deletion

* Gaz

* Return on removal

* Test cleanup

* Simply blob deletion

* Add test case to prove that deleting a root that doesn't exist will not return an error

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-01 19:03:47 +00:00
Potuz
83ce7e3607 Verify lmd without ancestor (#13250) 2023-12-01 17:31:27 +00:00
Potuz
cf8e554981 track target in forkchoice (#13249) 2023-12-01 16:30:34 +00:00
Nishant Das
59aa978223 Optimize Multivalue Slice For Trie Recomputation (#13238)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-01 12:01:28 +01:00
Radosław Kapka
e4a5711c8f Redesign of Beacon API evaluator (#13229)
* redesign

* ssz

* small fixes

* capitalize json and ssz

* rename and split files

* clearer names and comments

* bazel fix

* one more simplification
2023-11-30 16:53:51 +00:00
Nishant Das
d8b38cf230 Drop Transaction Count for Transaction Generator (#13228)
* reduce

* comment
2023-11-30 10:55:18 +00:00
Nishant Das
ca36634de6 Improve Gossipsub Rejection Metric (#13236)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-30 10:16:26 +00:00
Nishant Das
1c35b66132 Add Gossipsub Queue Flag (#13237)
* add it

* remove var

* fix tests

* terence's comments

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-30 14:22:59 +08:00
terence
56c1f9aab5 Update Prysm Proposer end points for Builder API (#13240) 2023-11-29 13:07:57 -08:00
Radosław Kapka
5ecb4d62a9 REST VC: Use POST to fetch validators (#13239) 2023-11-29 18:53:26 +01:00
james-prysm
bc107a61e3 builder API: remove blinded blob sidecar (#13202) 2023-11-29 06:28:37 -08:00
Nishant Das
9f41375550 remove subscriber checker (#13234) 2023-11-29 10:11:10 +08:00
Radosław Kapka
6a638bd148 HTTP handler for Beacon API events (#13207)
* in progress

* implementation done

* bzl

* fixes

* tests in progress

* tests

* go mod tidy

* Update beacon-chain/rpc/eth/events/events.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fix config test

* fix unreachable code issue

* remove proto service dir

* test fix

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-11-28 23:20:02 +00:00
Nishant Das
52f1b3f958 add in changes (#13226) 2023-11-28 22:12:13 +08:00
Nishant Das
80526a1899 Verify Block Signatures On Insertion Into Pending Queue (#13183)
* add check for bad signatures via gossip

* edge case handled
2023-11-28 03:13:59 +00:00
Manu NALEPA
da2212f6cc Allow validators registration batching on Builder API /eth/v1/builder/validators (#13178)
* builder `NewClient`: Simplify + fix some typos.

* Validator client: Implement `validator-registration-batch-size` option

* Address Potuz comments

* Address Potuz's comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-28 00:23:48 +00:00
terence
7cc05401ca Update proposer RPC to new blob sidecar format (#13189) 2023-11-27 15:44:52 -08:00
Radosław Kapka
cd8d499198 Move weak subjectivity endpoint to HTTP (#13220)
* Move weak subjectivity endpoint to HTTP

* remove server test file

* remove deprecation
2023-11-27 14:44:26 +00:00
Radosław Kapka
2fbda536b0 Fix handling POST requests in the REST VC (#13215)
* Fix handling POST requests in the REST VC

* tests for decodeResp
2023-11-25 23:25:14 +00:00
Nishant Das
0498e0a4d5 Fix Blob Storage Path (#13222)
* fix the path

* gaz
2023-11-25 01:57:22 +00:00
Radosław Kapka
098d6a3c0b Handle non-JSON responses from Beacon API (#13213)
* Run Beacon API evaluator at slot 3

* Revert "Auxiliary commit to revert individual files from f80b444688ed1acb267ee8bf00ba602d1f890cc7"

This reverts commit 0d3d7a4113533ac0516efe12d09cc3b9d78793f1.
2023-11-24 23:30:21 +00:00
Potuz
67d0b26a21 Accept block when error is only in logging (#13223)
* Accept block when error is only in logging

* linter shutup

* ignore nilerr on the linter
2023-11-24 19:00:53 +00:00
terence
6c85587d14 Update broadcast method to use BlobSidecar instead of SingedBlobSidecar (#13221)
* Update broadcast method to use BlobSidecar instead of SingedBlobSidecar

* Fix test
2023-11-24 07:18:00 +00:00
terence
6daa72634d Fix forkchoice pkg's comments grammar (#13217) 2023-11-22 17:27:42 -08:00
Potuz
07ee42660a lock RecentBlockSlot (#13212)
* lock RecentBlockSlot

* Kasey's fix
2023-11-22 16:58:00 -03:00
hzysvilla
4b5db8003b Comment typo (#13209)
* Update config.go

* Update flags.go

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-11-22 18:54:19 +00:00
Radosław Kapka
4b3c511a26 POST version of GetValidators and GetValidatorBalances (#13199)
* POST versions of GetValidators and GetValidatorBalances

* post statuses

* balances test

* group params

* test error cases
2023-11-22 17:30:52 +00:00
terence
8902ad3a20 Implement Slot-Dependent Caching for Blobs Bundle (#13205) 2023-11-22 07:23:50 -08:00
kasey
1123df7432 Verified roblobs (#13190)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-21 18:44:38 +00:00
Radosław Kapka
a7edec9b98 Better error handling in REST VC (#13203) 2023-11-21 17:42:55 +01:00
Nicolás Pernas Maradei
10ccf1840f [2/5] light client http api (#12984)
Co-authored-by: Lizhang <lizhang@polymerlabs.org>
2023-11-21 13:26:39 +01:00
terence
d035be29cd Optimize ReplayBlocks for Zero Diff (#13198)
* Stategen: replay block return early when zero diff

* Fix test setup
2023-11-17 18:19:05 +00:00
Preston Van Loon
bba8dd6f1e bazel: Run buildifier, general cleanup (#13193)
* Run buildifier

* Other BUILD.bazel cleanup, rm swagger stuff.

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-16 18:41:37 +00:00
terence
8f5ae760ee Add concurrency test for getting attestation state (#13196)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-16 17:52:59 +00:00
terence
5ba91a5216 Add construct_generic_block_test to build file (#13195)
* Add construct_generic_block_test test to build file

* Use the right require library

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-15 20:33:56 +00:00
james-prysm
4c381938e1 HTTP validator API: wallet endpoints (#13171)
* converting wallet calls to pure http

* fixing proto and gaz

* adding routes and fixing test

* fixing error handling

* fixing protos after conflict with develop

* adding deprecation notice

* fixing route test

* review feedback

* addressing more comments

* updating comment to be more clear

* fixing web_api proto
2023-11-15 19:40:14 +00:00
james-prysm
d4726f2866 HTTP Validator API: slashing protection import and export (#13165)
* adding migration for import and export slashing protection

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comments

* fixing unit test errors after view comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-11-15 17:35:22 +00:00
terence
4e3419e870 Enhance Validation for Block by Root RPC Requests (#13184)
* blk-by-root-check-root

* Account for gap

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-14 17:13:01 +00:00
terence
ac06362baf Add a helper for max request block (#13173)
* Add a helper for max request block

* Add test

* Use deneb fork epoch from config

* Fix comment
2023-11-14 05:50:51 +00:00
Radosław Kapka
28aa11c976 Config HTTP endpoints (#13168)
* Config HTTP endpoints

* error on unsupported type

* type assertion
2023-11-13 23:38:23 +00:00
Radosław Kapka
798d5ec585 Remove default value of circuit breaker flags (#13186)
* Update default value of `max-builder-epoch-missed-slots`

* remove the default value

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-13 18:55:55 +00:00
Radosław Kapka
9b97f3fd92 Return 404 from eth/v1/beacon/headers when there are no blocks (#13185)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-13 18:06:26 +00:00
Radosław Kapka
0946b5853f Pool slashings HTTP endpoints (#13148)
* Pool slashings HTTP endpoints

* e2e fix

* commit

* remove pb files

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-13 17:22:39 +00:00
Nishant Das
1530d17977 Fix Withdrawals (#13181)
* fix withdrawals

* disable it
2023-11-09 13:50:57 +00:00
Potuz
e46f9c5631 KZG Commitment inclusion proof verifier (#13174)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-08 19:04:21 +00:00
Preston Van Loon
3097601530 pgo: Enable pgo behind release flag (#13158)
* Revert a54e61ecb0

* Configure the use of pgo profiles behind the release config flag (--config=release)
2023-11-08 13:33:26 +00:00
Nishant Das
4a515c36e6 Deneb E2E (#13040)
* save changes

* add dep

* add changes

* add latest changes

* push changes

* hack it for mainnet

* fix deps

* update it

* add changes

* fix e2e

* revert it

* gaz

* remove log

* preston's review

* clear up

* add more logs

* fix nonce gaps

* make it better

* fix blobs

* set value

* add support for deneb scenario paths

* update to fix scenario

* go mod

* clean up

* fix up

* reduce cog complexity

* lint

* remove

* go sec

* Update testing/endtoend/evaluators/fork.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/ssz_proto_library.bzl

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix

* radek's review

* make it atomic

* gaz

* add deneb case

* remove deneb activation

* change e2e yaml

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-11-08 12:24:23 +00:00
Potuz
afaeff9d4c Merkle Proofs of KZG commitments (#13159)
* Merkle Proofs of KZG commitments

* fix mock

* Implement Merkle proof spectests

* Check Proof construction in spectests

* fix Merkle proof generator

* Add unit test

* add ssz package unit tests

* add benchmark

* fix typo in comment

* ProposerSlashing was repeated

* Terence's review

* move to consensus_blocks

* use existing error
2023-11-06 08:49:35 -03:00
terence
f663f605d2 Add blob getters (#13170)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-04 14:35:56 +00:00
Terence
12f7143c4f Validator client: remove blob signing (#13169) 2023-11-03 12:10:15 -07:00
Radosław Kapka
1f250f7e89 Validator HTTP endpoints (#13167)
* HTTP validator endpoints

* Sammy's review

* capitalize errors

* test fix

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-03 16:59:04 +00:00
Sammy Rosso
0f65e51d1e Blob filesystem: Save Blobs (#13129)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Save full sidecar + reviews

* Use path builder in test

* Use other BlobSidecar

* Cleanup

* Fix gosec

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-03 16:24:30 +00:00
Radosław Kapka
d1dd8471a3 Debug HTTP endpoints (#13164)
* Debug HTTP endpoints

* register endpoints

* tests

* small fixes

* config test fix
2023-11-03 15:33:46 +00:00
Terence
7a6487b746 Remove pending blobs queue (#13166) 2023-11-03 07:07:43 -07:00
Potuz
daa6d2e741 Implement Merkle proof spectests (#13146)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-02 23:08:16 +00:00
Raul Jordan
8a743a6430 Update Terms of Service (#13163)
* tos updates

* fixes
2023-11-02 17:11:11 +00:00
james-prysm
c0fb16a96f HTTP validator API: health endpoints (#13149)
* updating health endpoints

* updating tests

* updating tests

* moving where the header is written and adding allow origin header

* removing header

* Update validator/rpc/handlers_health.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handlers_health.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handlers_health.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's comments

* Update handlers_health.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* adding the correct errors to handle error

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-11-02 15:51:21 +00:00
Terence
57eda1de63 Add RO blob sidecar (#13144) 2023-11-01 10:03:49 -07:00
Preston Van Loon
a54e61ecb0 pgo: remove default pprof profile (#13150) 2023-10-31 21:43:41 +00:00
james-prysm
27b4e32e1c HTTP Validator API: /eth/v1/keystores (#13113)
* WIP

* fixing tests

* fixing bazel

* fixing api client

* fixing tests

* fixing more tests and bazel

* fixing trace and more bazel issues

* fixing router path function definitions

* fixing more tests and deep source issues

* adding delete test

* if a route is provided, reregister before the catch all on the middleware.

* fixing linting

* fixing deepsource complaint

* gaz

* more deepsource issues

* fixing missed err check

* changing how routes are registered

* radek reviews

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing unit test after sammy's review

* adding radek's comments

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2023-10-31 16:33:54 +00:00
Nishant Das
b56bf00682 Fix Pending Queue Deadline Bug (#13145)
* rearrange deadline

* naming
2023-10-31 06:40:41 +00:00
Preston Van Loon
b24b60dbd8 zig: Update zig to recent main branch commit (#13142)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 23:48:45 +00:00
Potuz
dc9d34b41b fix head slot in log (#13139) 2023-10-30 16:56:20 -03:00
Radosław Kapka
2ef0b3526d Fix block proposals in the REST validator client (#13116)
* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 18:15:45 +00:00
Sammy Rosso
047613069e Rename Blob retention epoch flag (#13124)
* Rename flag and add alias

* Update cmd/beacon-chain/flags/base.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Fix sentence

* Fix TestConfigureBlobRetentionEpoch

* Fix silly mistake

* Reviews

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-10-30 17:35:31 +00:00
Justin Traglia
159a5dd69d Check that blobs count is correct when unblinding (#13118)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-30 16:49:32 +01:00
hyunchel
470ea6d717 Remove no-op cancel func (#13069)
This cancel function is currently a no-op due to the blank identifier.
One might argue that the cancel func should be restored from no-op by
replacing the blank identifier with the proper variable. When the parent
context is cancelled, however, all the functions down the call tree with
the context will be notified of the cancellation anyway. Removing the
cancel function would not change any outcome under the current
implementation.

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-30 14:43:25 +00:00
Radosław Kapka
b441f20e6a Remove /node/peers/{peer_id} from Beacon API evaluator (#13138) 2023-10-30 14:21:07 +00:00
Potuz
2ea5bff9c0 Log when sending FCU with payload attributes (#13137)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 12:30:20 +00:00
terencechain
c2433ff854 Update spectest and changed minimal preset for field elements (#13090)
* update trusted setup

* update dependencies

* Update workspace

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 11:41:58 +00:00
Preston Van Loon
82640b3d88 Enable profile guided optimization for beacon-chain (#13035)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 05:52:36 +00:00
Radosław Kapka
f925aded66 Allow unknown fields in Beacon API responses (#13131) 2023-10-27 16:47:50 +00:00
james-prysm
10a89fef13 DEPRECTATION: Remove exchange transition configuration call (#13127)
* wip removing call to execution client for transition configuration

* updating bazel and execution engine proto

* removing more spots where the call was added

* removing unused metric
2023-10-27 15:43:00 +00:00
Nishant Das
56c65b8527 Return Error Gracefully When Removing 4881 Flag (#13096)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-27 13:24:43 +00:00
Radosław Kapka
022ee17af9 Better Beacon API evaluator part 1 (#13084)
* Better Beacon API evaluator part 1

* rename package

* more endpoints

* rename package back

* more endpoints

* small improvements

* remove the need for `params`

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-27 11:57:49 +00:00
terencechain
203dc5f63b Check blob index duplication for blob notifier (#13123)
* Check blob index duplication for blob notifier

* Better locks and test

* Better locks and test

* Kasey's feedback

* Fix init
2023-10-27 03:26:34 +00:00
Stefan
6f941b8138 fix segmentation fork when Capella for epoch is MaxUint64 (#13126)
* fix segmentation fork when Capella for epoch is MaxUint64

fix segmentation

fix segmentation

* Update cmd/prysmctl/testnet/generate_genesis.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update cmd/prysmctl/testnet/generate_genesis.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

---------

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2023-10-27 00:46:08 +00:00
dependabot[bot]
ac412259eb Bump google.golang.org/grpc from 1.53.0 to 1.56.3 (#13119)
* Bump google.golang.org/grpc from 1.53.0 to 1.56.3

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.53.0 to 1.56.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.53.0...v1.56.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* gazelle

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-26 16:54:57 +00:00
Radosław Kapka
3d78a52980 Fill state attestations (#13121) 2023-10-26 15:35:01 +00:00
Preston Van Loon
5de8ec4600 Update go to 1.20.10 (#13120) 2023-10-26 02:31:12 +00:00
Preston Van Loon
7e88eefc60 Add zero length check on indices during NextSyncCommitteeIndices (#13117)
* Add zero length check on indices during NextSyncCommitteeIndices computation. Fixes #13051

* Move the error further up the stack

* Fix TestSubmitAggregateAndProof_IsAggregatorAndNoAtts

* Delete TestServer_ListAssignments_NoResults. That is an impossible scenario that now returns an error
2023-10-25 21:42:17 +00:00
terencechain
cabf3476e7 Add context deadline for pending queue's receive block (#13114)
* Add context dead like for pending queue's receive block

* Use timeout
2023-10-25 19:40:17 +00:00
Radosław Kapka
5a01eecc50 HTTP state endpoints (#13099)
* slowly plowing through

* implementation ready

* wrong epoch particip

* fix epoch participation

* tests

* fix e2e

* error handling in tests

* review from James

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-10-25 18:12:58 +00:00
terencechain
b608c9f711 Log blob's kzg commmitment at sync (#13111)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-25 16:27:19 +00:00
Justin Traglia
671bf00c98 Fix bug in Beacon API getBlobs (#13100)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-25 03:33:59 +00:00
terencechain
cbf6a2752d Reject Blob Sidecar Incorrect Index (#13094)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-25 01:57:54 +00:00
Nishant Das
642458f037 Fix Pending Queue Expiration Bug (#13104)
* fix bug

* make test better

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-24 20:28:57 +00:00
james-prysm
2a067d5d03 HTTP Validator API: /eth/v1/validator/{pubkey}/feerecipient (#13085)
* migrating fee recipient endpoints to pure http implementation

* fixing linting

* fixing type name

* fixing after merging develop

* fixing linting and tests

* Update validator/rpc/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-24 16:55:45 +00:00
terencechain
a2f60364ae Check return and request lengths for blob sidecar by root (#13106) 2023-10-24 15:02:44 +00:00
Justin Traglia
45f68fa8d5 Replace MAX_BLOB_EPOCHS usages with more accurate terms (#13098)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-24 03:28:50 +00:00
terencechain
f55708b995 Fix blob sidecar subnet check (#13102)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-23 23:08:25 +00:00
Delweng
00826e8858 beacon-chain/blockchain: fix some datarace in go test (#13036)
* beacon-chain/blockchain: mockBeaconNode with mutex

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-node/forkchoice: bool -> atomic.Bool

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/blockchain: datarace in concurrent postBlock

Signed-off-by: jsvisa <delweng@gmail.com>

* Revert "beacon-node/forkchoice: bool -> atomic.Bool"

This reverts commit 4aad095b0f.

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-23 21:55:47 +00:00
terencechain
76fec1799e Replace Empty Slice Literals with Nil Slices (#13093)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-23 16:36:11 +00:00
james-prysm
9c938d354d HTTP Validator API: /eth/v1/validator/{pubkey}/gas_limit (#13082)
* WIP

* more WIP

* fixing unit tests

* gaz

* gofmt

* adding routes

* adding tests for validator routes

* adding in missed comment

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* adding log and removing unneeded type

* fixing casing on tests

* adding more tests

* Update server.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing radek's comments

* handling error

* fixing naming on validator struct for mock

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-23 15:49:28 +00:00
terencechain
83932d8e05 Refactor Error String Formatting According to Go Best Practices (#13092)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-23 07:57:25 +00:00
Nishant Das
beebb56c8e Fix Builder Testing For Multiclient Runs (#13091) 2023-10-23 06:33:46 +00:00
Potuz
0920fb1f61 Return early from ReceiveBlock if already sycned (#13089)
* Return early from ReceiveBlock if already sycned

* Fix bad setup test
2023-10-22 18:31:50 -03:00
Delweng
29f8880638 beacon-node/rpc: fix go test datarace (#13018)
* beacon-chain/p2p: ust atomic.Bool instead of bool

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/p2p,rpc: read mock.BroadcastMessages with lock

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/p2p,rpc: read attestation with lock

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: fix typo

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/p2p: typo

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-22 15:12:55 +00:00
Nishant Das
f91efafe24 Fix Multivalue Slice Deadlock (#13087)
* fix deadlock

* gofmt

* lint
2023-10-21 17:08:52 +00:00
terencechain
9387a36b66 Refactor Exported Names to Follow Golang Best Practices (#13075)
* Fix exported names that start with a package name

* A few more renames

* Fix exported names that start with a package name

* A few more renames

* Radek's feedback

* Fix conflict

* fix keymanager test

* Fix comments

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 16:45:33 +00:00
Potuz
65ce27292c sync only up to previous epoch on phase 1 (#13083)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-21 00:05:14 +08:00
terencechain
823f8ee3a2 Fix redundant type converstion (#13076)
* Fix redundant type converstion

* Revert generated changes

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 15:07:10 +00:00
vuittont60
88e1b9edb3 docs: fix typo (#13023)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-20 14:55:16 +00:00
Nishant Das
c7e28908f5 Add Clarification To Sync Committee Cache (#13067)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 13:57:07 +00:00
james-prysm
7143fe80bc HTTP VALIDATOR API: remote keymanager api /eth/v1/remotekeys (#13059)
* WIP migrating keymanager api changes

* gaz

* fixing more tests

* fixing unit tests

* fixing deepsource

* fixing visibility of package

* fixing more package visability issues

* gaz

* fixing test

* moving routes to proper location

* removing whitespae for linting

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-19 16:17:42 +00:00
Potuz
e231d88ca0 Remove sidecars with invalid proofs (#13070)
* Remove sidecars with invalid proofs

* unhandled error

* Add missing delete
2023-10-19 12:33:19 -03:00
Nishant Das
0486b64dcc Set Better Slice Capacities in the State (#13068)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-19 14:45:39 +00:00
shuoer86
b4847ac9ad Fix typos (#13053)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-19 12:23:11 +00:00
Nishant Das
bc125a95ae Set Verbosity of Goodbye Logs to Trace (#13077) 2023-10-19 03:49:34 +00:00
Preston Van Loon
f592bf7f07 rpc/apimiddleware: Test all paths can be created (#13073) 2023-10-18 21:12:45 +00:00
Nishant Das
bcc23d2ded Fix Withdrawals Marshalling (#13066)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-18 17:34:12 +00:00
Radosław Kapka
6e0715e92a HTTP Beacon APIs for blocks (#13048)
* handlers

* blinded block handlers and protos

* revert SSZ and register endpoints

* remove vars

* e2e

* handler for v1 endpoint

* tests

* register GetBlock

* disable security linter

* remove unused structs

* more review comments
2023-10-18 15:09:38 +00:00
james-prysm
71fa70ce40 CLEANUP: validator exit prompt (#13057)
* removing check special phrase in url from validator exit

* fixing missed tests

* fixing unit test and addressing preston/sammy's comments

* fixing test

* sammy's comments

* deepsource recommendation

* Update validator/accounts/accounts_helper.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* sammy's review

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-18 14:00:25 +00:00
terencechain
4809da62cc Passing block as arugment for sidecar validation (#13062) 2023-10-18 13:42:15 +00:00
terencechain
e10dbaa8b4 Use correct context for sendBatchRootRequest (#13061) 2023-10-18 05:58:44 -07:00
Delweng
71b08a50b7 beacon-chain/sync: fix some datarace in go test (#13039)
* beacon-chain/sync: adjust dataflow incase of datarace

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/p2p: use atomic..Bool instead of bool

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/sync: use channel to control concurrent

Signed-off-by: jsvisa <delweng@gmail.com>

* Revert "beacon-chain/sync: use channel to control concurrent"

This reverts commit 500d5b1ecdeef175e3a5970867d8176abd0763d7.

Signed-off-by: jsvisa <delweng@gmail.com>

* Revert "beacon-chain/sync: adjust dataflow incase of datarace"

This reverts commit 0c819e96cafb89ccba314ca98a8a0fdfcbb70b8c.

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/sync: waitForChain is invoked inside registerHandlers

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/sync: no need to registerHandler for waitForChainStart testcase

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/sync: set p2p.Digest before goroutine

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-17 17:30:10 +00:00
terencechain
cb5ce74a23 Add pending blobs queue for missing parent block (#13005)
* Add pending blobs queue for missing parent block

* Prune sidecars older than previous slot

* Prune sidecar based on time

* Tests

* Fix state notifier

* Wait for chain to start

* Remove logs

* Remove bad logs

* James feedback

* Fix conflict

* Rm outdated check

* Potuz's feedback

* Kasey's feedback

* Use 11s mark

* Use secs

* Add pending blobs queue for missing parent block

* Prune sidecars older than previous slot

* Prune sidecar based on time

* Tests

* Fix state notifier

* Wait for chain to start

* Remove logs

* Remove bad logs

* James feedback

* Fix conflict

* Rm outdated check

* Potuz's feedback

* Kasey's feedback

* Use 11s mark

* Use secs

* Add test case for duplicates

* Radek's feedback

* Fix test
2023-10-17 14:42:15 +00:00
Andrew Davis
cc81444e13 Fix blob_sidecar SSE payload (#13050)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-17 11:07:03 +00:00
james-prysm
bfae7f3c9f HTTP VALIDATOR API: /eth/v1/validator/{pubkey}/voluntary_exit (#13032)
* migrating set validator exit to http only and removing from api middleware

* fixing ineffassign error

* cleaning up middleware

* fixing linting

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* adding more tests based on sammy's comments

* radek's feedback

* adjusting error codes

* one more status change

* fixing unit test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-16 22:16:20 +00:00
Potuz
2fc5011091 remove bad comment (#13056)
* remove bad comment

* rename function

* rename function
2023-10-16 19:40:51 +00:00
james-prysm
493a7179d7 Deneb - web3signer (#12767)
* wip

* adding deneb block

* adding in support for blobs and fixing unit tests for deneb

* fixing linting

* gaz

* adding support for new web3signer version

* fixing tag name

* addressing feedback

* fixing tests

* adding unit test for review

* updating test name

* updating unit tests and length logic

* adding in lengthfor root

* adjusting max blob length

* fixing mock

* fixing another mock

* gaz

* adding network configs

* removing duplicate

* changing based on nishant's feedback

* Update validator/keymanager/remote-web3signer/v1/requests.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/keymanager/remote-web3signer/metrics.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* sammy's suggestions

* removing temp file

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-16 18:06:36 +00:00
Radosław Kapka
b52baba2f1 Register sync subnet when fetching sync committee duties through Beacon API (#12972)
* Register sync subnet when fetching sync committee duties through Beacon API

* review

* oops

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-16 17:20:27 +00:00
terencechain
2f378a045a Proposer: better handling of blobs bundle (#12956)
* Proposer better handling of blobs bundle

* Reset bundles after conversion

* Reset earlier

* Proposer better handling of blobs bundle

* Reset bundles after conversion

* Reset earlier

* Fix conflict

* use correct blindBlobsBundle

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-16 16:41:24 +00:00
terencechain
58cdb29ef3 Verify blobs that arrived from by root request (#13044)
* Verify blobs that arrived from by root request

* Add tests

* Add tests

* Fix test

* Fix lint

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-16 05:30:33 +00:00
hyunchel
cc2b4db582 Add state not found test case (#13034)
* beacon-chain:rpc/eth/shared: prevent mutiple error messages

This commit prevents the error writing function from writing multiple
JSON objects. An error message with more than one JSON object will not
unmarshal into the default error response.

* beacon-chain/rpc/eth/beacon: add a test case for missing state

This commit adds a test on beacon states finality checkpoints endpoint
to cover a case when state is not found.

* beacon-chain/rpc/eth: update error response to meet the spec

This commit updates error message on beacon states finality
checkpoints endpoint to ensure that the response complies to
Ethereum Beacon-API specification.

* beacon-chain/rpc/eth/shared: add build dependency

* beacon-chain/rpc/eth/shared: update test on state not found

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-13 17:21:38 +00:00
Andrew Davis
be9b6ea837 fix(beacon-chain/rpc): blob_sidecar event stream handler (#12999)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2023-10-13 13:51:16 +00:00
Potuz
806a394c89 Reject blobs with invalid parent (#13047)
* Reject blobs with invalid parent

* remove unused comment
2023-10-13 13:15:40 +00:00
Potuz
97a99874e8 Remove confusing comment (#13045) 2023-10-13 00:36:03 +00:00
terencechain
945b087ca9 Add more blob and block checks for by range (#13043)
* Add more blob and block checks for by range

* Update beacon-chain/sync/initial-sync/blocks_fetcher.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/sync/initial-sync/blocks_fetcher.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update beacon-chain/sync/initial-sync/blocks_fetcher_test.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-10-12 19:35:32 +00:00
Radosław Kapka
b57effd096 HTTP Beacon APIs: 3 state endpoints (#13001)
* HTTP Beacon APIs: 3 state endpoints

* remove API middleware e2e evaluator

* remove evaluator registrations

* review feedback

* compilation fix
2023-10-12 17:53:33 +00:00
terencechain
867db1aeee Exit early if blob by root request is empty (#13038) 2023-10-12 02:27:26 +00:00
terencechain
99843688cd Request missing blobs while processing pending queue (#13015)
* Request missing blobs while processing pending queue

* Fix build

* Lower complexity

* Fix conflict

* Clean up per James's comment

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-11 22:07:11 +00:00
Radosław Kapka
a536612c39 HTTP Beacon APIs for node (#13010) 2023-10-11 22:18:49 +02:00
terencechain
c5501f8775 Refactor Pending Block Queue Logic in Sync Package (#13026)
* Clean up pending block queue

* Kasey's feedback

* Kasey's feedback on validateBeaconBlock err handling

* Clean up handleBlockProcessingError

* Clean up old comments

* James feedback

* has peer helper

* Reuse parent reoot

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-11 14:41:24 +00:00
james-prysm
55e4c6e1db HTTP API: /eth/v1/beacon/pool/bls_to_execution_changes (#12963)
* wip

* wip changes for beacon handlers

* wip

* updating protos

* fixing conflict

* fixing protos

* fixing more tests

* migrating tests and removing one that is irrelevant

* fixing e2e

* fixing unit tests

* missed adding routes

* gaz

* fixing more tests

* Update beacon-chain/rpc/eth/beacon/handlers_pool.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's comments

* fixing protos after conflict

* accidently removed a proto type

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-11 14:08:06 +00:00
Dhruv Bodani
2806326155 integrate validator count endpoint in validator client (#12912)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-11 15:23:02 +02:00
Delweng
d7318ea485 beacon-chain/execution: fix a data race in testcase (#13016)
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-11 03:37:25 +00:00
Radosław Kapka
e183d1dff4 Improvements to produceBlockV3 (#13027) 2023-10-10 18:06:57 +00:00
Radosław Kapka
a3868e7fc6 Fix comment of BlobSidecarsBySlot (#13019)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-10 15:54:29 +00:00
james-prysm
af70677778 Deneb: Produce Block V3 - adding consensus block value (#12948)
* adding in block rewards to represent consensus payload

* Update beacon-chain/rpc/eth/validator/handlers_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's comments

* more review changes

* adding more tests for forks

* gaz

* updating names

* gaz

* fixing imports

* fixing variable name

* gaz

* fixing test

* renaming variables to match data

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-10 15:12:20 +00:00
Preston Van Loon
8eb82dd378 Update rules_go and gazelle to 0.42 & 0.33 (latest releases) (#13021)
* Provide @go_googleapis, update gazelle

* fix pb build failures

Fix build issues with grpc-gateway
2023-10-10 12:50:29 +08:00
Preston Van Loon
0bd232667b Fix image deps (#13022) 2023-10-09 17:21:49 -05:00
terencechain
39072e1b74 Check block exists in pending queue before requesting from peer (#13013) 2023-10-09 10:35:15 -07:00
terencechain
66011d5d9c Check blob exists before requesting from peer (#13012)
* Check blob exists before requesting from peer

* Potuz's feedback

* Fix err

* Add index check
2023-10-09 22:55:24 +08:00
Delweng
419dbd57f7 beacon-chain/execution: no need to reread and unmarshal the eth1Data twice (#12826)
* beacon-chain/execution: no need to reread and unmarshal the eth1Data multitimes

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/execution: rename to validPowchainData

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/execution: no return eth1Data if error

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/execution: return eth1data even if genstate is nil

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-09 13:51:52 +08:00
Nishant Das
da6ae3c204 use read only head state (#13014) 2023-10-07 14:44:52 -05:00
Radosław Kapka
44973b0bb3 Using the multi value slice for the beacon state (#12549)
* in progress...

* in progress...

* remove log

* log root

* Revert "Auxiliary commit to revert individual files from f12a609ea2a2f1e87e97321f3a717cd324b5ae97"

This reverts commit 5ae35edb6477d8d0ea4e94b273efc6590484da85.

* cleanup

* remove log

* remove whitespace

* remove logs

* more stuff

* copy

* always rebuild trie

* revert

* add state

* init state

* fix all

* uintptr

* move slice to new package

* lock in `Detach`

* remove constraint

* reorder

* blockroots and stateroots

* fill roots in empty()

* fix hasher

* implement slice for balances and inactivity scores

* detach in setters

* Revert "implement slice for balances and inactivity scores"

This reverts commit 59eb9df8d7.

# Conflicts:
#	beacon-chain/state/state-native/setters_validator.go

* use counter to track states

* typos

* rename interface

* balances

* gauge

* some improvements

* first try with map

* fix

* inactivity scores in progress

* fix test

# Conflicts:
#	beacon-chain/state/state-native/helpers_test.go

* test fixes

* ToProto fix

* copy roots

* validators

* build fixes

* fix bug in `ToProto`

* fix fuzz test

* fix bug in slice getters

* fix state equality checks

* make tests pass

* make tests pass

* more test updates

* Revert "Auxiliary commit to revert individual files from 34e7344bff08a589e6341bb1829e3cb74159e878"

This reverts commit ecd64efa8917f37ca41460e0356ff007fe55dd9d.

* Revert "make tests pass"

This reverts commit 0cf00f19ee.

* Revert "make tests pass"

This reverts commit 521b65e1d2.

* pass tests

* deepequal identifiable types

* Deflake `cloners_test.go`

* feature flag for block roots

* feature flag

* remove recursive locks

* reduce complexity of rootSelector

* fix randao mixes root

* some fixes

* revisit tests

* revert change to field trie helpers

* initialize field map for tests

* remove whitespace

* initialize roots with proper length

* more fixes

* out of bounds message fix

* optimize length calculation

* remove call to Len in PubkeyAtIndex

* don't log deposits

* unit tests

* unit tests

* fix

* comments

* test fixes

* id

* remove Enumerator interface

* review feedback

* simplify field trie

* bring back fieldtrie package

* fix bazel file

* use handle32ByteArrays for root computation

* fix locks

* metrics

* bzl

* simplify some things

* use htr in state test

* remove code from require package

* gzl

* more htr

* Fuzzing of the multi-value slice

* assert values

* getter optimizations

* use At when reading from validators

* Nishant's review

* restore safe copy

* remove empty line

* build fix

* restore how we get root at index for deafult mode

* more review comments

* optimize default behavior

* simplify Slice calls

* test fix

* remove unnecessary package

* remove unused setter

* make fieldMap unexported

* some improvements in state package

* call `Slice` instead of manually copying

* unlock in ReadFromEveryValidator

* Potuz's comments

* lock the state when reading from all validators

# Conflicts:
#	beacon-chain/state/state-native/getters_validator.go

* add back preston's changes

* add index

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-10-06 01:10:05 +00:00
Preston Van Loon
de0c7e6256 update go 1.20.9 (#13009) 2023-10-05 22:11:05 +00:00
Nishant Das
c1c0cd040c Fix Initial Sync PreProcessing (#13007)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-05 15:18:53 +00:00
Nishant Das
734eb98941 Fix Long Running E2E (#13000) 2023-10-05 14:37:11 +00:00
Nishant Das
ffaef83634 Build Blocks in Parallel Permanently (#13008) 2023-10-05 06:45:11 -07:00
terencechain
f9a40ef111 Handle wrapped invalid block error (#12982)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-04 18:16:35 +00:00
Radosław Kapka
7454041356 Use middleware to handle comma-separated query params (#12995) 2023-10-04 15:49:42 +02:00
Radosław Kapka
f37301c0c0 Remove remote slashing protection feature (#12989)
* Remove remote slashing protection feature

* test fix

* remove mock from tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-04 04:15:06 +00:00
Nishant Das
cf1bfb9d67 Make New Engine Methods The Default (#12997) 2023-10-04 01:44:31 +00:00
Potuz
9d8dd5c9ad Don't prune proposer ID cache in a loop (#12996)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-03 23:25:53 +00:00
Delweng
f812bdcf60 beacon-node/state: alloc 1more item for append case (#12832)
* beacon-chain/state/attenstation: alloc +1 items for append

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/state/eth1: alloc +1 items for append

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/state/misc: alloc +1 items for append

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/state/participation: alloc +1 items for append

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/state/validator: alloc +1 items for append

Signed-off-by: jsvisa <delweng@gmail.com>

* Add some benchmarks

* Evaluate append vs copy. Apply results

* fix copy issue

* revert copy changes from a5ba8d4352

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-10-03 20:41:55 +00:00
terencechain
58c0899676 Don't mark block as bad in validateBeaconBlock for pending queue (#12983)
* Don't mark block as bad in validateBeaconBlock for pending queue

* Fix tests
2023-10-03 16:36:23 +00:00
terencechain
4628c19f51 Refactor churn limit helpers (#12971) 2023-10-03 14:14:41 +00:00
Radosław Kapka
58f23d2302 Handle hex in Blocker, Stater and optimistic mode checker (#12979)
* Handle hex in `Blocker`

* hex in stater

* ineff fix

* optimistic mode

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-03 05:52:29 +00:00
Potuz
9e33723b26 Revert "Do not cache proposer ID on GetProposerDuties (#12939)" (#12986)
This reverts commit e2591f7c5b.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-03 04:04:17 +00:00
Potuz
2c32a87d17 Tweak LastRoot to return head root (#12985) 2023-10-03 02:22:52 +00:00
Potuz
56f3dafb54 Lock forkchoice on late block tasks (#12978)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-02 17:07:03 +00:00
Nicolás Pernas Maradei
70380660b3 [1/5] Light client sync protocol (#12853)
* generate SyncCommittee SSZ

* refactor error handling

* rewards: use http2.HandleError

* add light client protobuf files

* add light client helpers
2023-10-02 15:34:34 +00:00
Radosław Kapka
ecd55e5462 Remove go-playground/validator from Beacon API (#12973)
* Remove go-playground/validator from Beacon API

* gzl

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-02 13:21:59 +00:00
Nishant Das
46b2442127 Do Not Calculate Churn With 0 Exits (#12976) 2023-10-02 01:46:58 +00:00
Radosław Kapka
d8b2d9060f Return RpcError from core service's SubmitSyncMessage (#12974)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-30 14:19:09 +00:00
Justin Traglia
b667c68c87 Default to the portable version of blst (#12720)
* Default to the portable version of blst

* Update based on review comments

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-09-29 15:35:15 -05:00
Lucas Saldanha
fb19ee8895 Updating Teku mainnet bootnodes ENRs (#12962)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-09-29 10:11:58 +00:00
terencechain
51b8075474 Use skip mev-boost flag for GetBlock RPC (#12969)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-09-29 16:48:40 +08:00
Radosław Kapka
367504f403 Support comma-separated query params in Beacon API (#12966) 2023-09-29 15:40:23 +08:00
Potuz
b4e72f1e19 Deneb spectests release v1.4.0-beta.2-hotfix (#12959)
* Update and use max per epoch churn limit

* Update spec tests

* Fix e2e test

* deneb fork epoch condition

* Fix lint and better casting

* fix ordering

* fix check

* gaz

* Fix more tests

* Apply proposer boost to first block in equivocation

* Increase timeout

* Don't increase timeout, it's not the reason

* implement deneb forkchoice spectests

expose ReceiveBlob from the blockchain package

* spin_off_helper

* remove minimal tests

* Terence's review

* Add process register test for Deneb

* Terence's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* fix forkchoice minimal

* fix minimal sha

* general sha

* different repos

* different repos

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2023-09-28 21:05:23 +00:00
Preston Van Loon
a7361cd5ab Docker: Add "manual" build tag to reduce build times in CI (#12967)
* Add manual tags to oci images. This reduces build times for build and test in CI.

* Ran buildifier on tools dir
2023-09-28 17:49:26 +00:00
Preston Van Loon
02ee6897bb Multiarch docker containers (#12428)
* Add bazel-zig-cc for a hermetic cc toolchain

* gazelle

* Remove llvm

* remove wl

* Add new URLs for renamed repo

* gazelle

* Update to v2.0.0-rc1

* bump to rc2

* Proof of concept multi-arch containers for beacon-chain

* TODO and gaz

* Refactor to starlark macro. Use a version of bash that actually works

* progress

* gaz

* multirun to use multiple repositories, but doesn't work with tag stamping

* Revert "multirun to use multiple repositories, but doesn't work with tag stamping"

This reverts commit 93afa76f65.

* Add targets for all supported docker images and temporarily set the repository to prysm-dev for testing

* use a temporary fix to see if it works on buildkite

* Revert "use a temporary fix to see if it works on buildkite"

This reverts commit ddc79283ca.

* testing a cURL fix

* try fix with my fix

* Revert "try fix with my fix"

This reverts commit bb7521bf47.

* Revert "testing a cURL fix"

This reverts commit 8a4782110f.

* try tip of main branch for rules_oci

* update to 1.2.0

* Update rules_oci to v1.3.0

* Update rule_oci to 1.3.4

* Disable experimental_remote_downloader

* Remove extra zig bazelrc

* Move image deps to its own file

* PR feedback

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-28 15:24:16 +00:00
terencechain
c20f188966 Deprecate safe slots to import flag (#12964) 2023-09-28 17:40:58 +08:00
terencechain
5870536dca Add a flag to configure blob retention epoch period (#12941)
* Add a flag to configure blob retention epoch period

* Add test

* Kasey's feedback

* More Kasey's feedback

* Fix lint and tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-27 14:59:52 +00:00
james-prysm
c8b39e08ef cleanup on api types (#12961)
* removing code duplication using shared types

* gaz

* fixing test

* fixing tests
2023-09-27 09:26:32 -05:00
Sammy Rosso
b0b4e42436 HTTP Beacon API: /eth/v1/beacon/blocks and /eth/v1/beacon/blinded_blocks (#12827)
* Add PublishBlock

* Add endpoints

* Cleanup

* Modify publishBlock to work for v1 and v2

* Remove v2 + add server receiver

* Cleanup remaining endpoints

* Remove error from SszRequested

* Remove unused functions
2023-09-27 12:51:37 +00:00
terencechain
b0caea3fae Refactor getLocalPayloadAndBlobs with New Helper getParentBlockHash (#12951)
* Implement getParentBlockHash helper

* Specify errors
2023-09-27 03:08:17 +00:00
Roberto Bayardo
a46370f5bf clean up code around setting geth client headers (#11748) 2023-09-26 15:15:28 +00:00
Nishant Das
0919b2245f Avoid Public Key Copies During Aggregation (#12944)
* Add in optimization

* add better test

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-26 11:13:19 +00:00
Nishant Das
42c192d97d Fix More Racy Tests in Blockchain (#12957) 2023-09-26 09:17:11 +00:00
Potuz
3394bbe359 forkchoice return last canonical root of epoch (#12954)
* forkchoice return last canonical root of epoch

* move inside loop

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-25 16:36:43 +00:00
Nishant Das
70225186ff Fix TestService_ReceiveBlock (#12953)
* fix tests

* radek's review

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-25 15:24:32 +00:00
terencechain
942d63fcc1 Refactor construct generic beacon block for proposer (#12943) 2023-09-25 07:07:04 -07:00
Sammy Rosso
90fb2325db /eth/v1/config/deposit_contract return string instead of uint (#12952)
* Convert uint to string

* Add test

* Rename test
2023-09-25 13:02:47 +00:00
Preston Van Loon
57a63f37f7 Update holesky config for new genesis (#12919)
* Update holesky config with new genesis. See https://github.com/eth-clients/holesky/pull/73

* Update bootnodes

* Update bootnodes

* Use latest commit

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-23 16:09:28 +00:00
Preston Van Loon
9a2c2470c6 db: Log a warning if the genesis state is smaller than 1Kb (#12897)
* Add warning with a small genesis state

* 1<<10 is 1Kb

* Add genesis state loading with hash information

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-23 15:29:15 +00:00
james-prysm
723f73795f Deneb: blob sidecar events (#12928)
* adding in deneb blob event triggers

* fixing linting

* kasey's feedback

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-22 21:54:10 +00:00
Raul Jordan
6454081577 Amend Bug Report Template Requirements (#12937)
* requirements

* amend

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-22 20:26:24 +00:00
Potuz
489b34a01f Use headstate when requesting current epochs (#12940)
* Use headstate when requesting current epochs

* gazelle

* review requests
2023-09-22 19:48:24 +00:00
Radosław Kapka
e22258caa9 HTTP validator APIs (#12887)
* GetValidators in progress

* in progress

* completed implementation of GetValidators + some tests

* typo

* completed tests for GetValidators

* GetValidator

* GetValidatorBalances

* register

* more tests and e2e fix

* middleware cleanup

* remove struct

* handle all ignored

* test fixes

* more test fixes

* even more test fixes

* remove unused structs

* docs

* fix validator count test

* build fix

* add length checks

* lint fix

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-21 18:16:59 +00:00
Radosław Kapka
3bc9ac37f6 Add Content-Type header to VC POST requests (#12942) 2023-09-21 17:21:06 +00:00
Nishant Das
7247b8bd3c Fix Unhandled Error (#12938)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-21 16:23:56 +00:00
Potuz
e2591f7c5b Do not cache proposer ID on GetProposerDuties (#12939) 2023-09-21 15:22:41 +00:00
Radosław Kapka
b40729f6c0 Return correct root from GetBlockHeader (#12935)
* Return correct root from `GetBlockHeader`

* variable rename
2023-09-21 12:02:14 +00:00
Nishant Das
b7cb5c81be fix it (#12932) 2023-09-21 11:40:03 +08:00
Radosław Kapka
e76aedf1ae HTTP API: GetLiveness and GetBlockHeader (#12916)
* GetLiveness

* GetBlockHeader

* simplify request objects

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-20 19:06:11 +00:00
terencechain
0e5d299d02 Remove unused pending blobs queue (#12913)
* Remove pending blobs queue

* Fix tests

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-20 17:00:01 +00:00
Radosław Kapka
14f040de48 GetFinalityCheckpoints and GetGenesis HTTP endpoints (#12902)
* GetFinalityCheckpoints and GetGenesis

* bzl

* remove unused func

* use const as base path

* bring back headers

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-19 17:53:45 +00:00
terencechain
1a1a30591e Handle Specific Blob Retrieval Error in DB (#12889)
* log db blob by root error

* Add comments
2023-09-19 15:29:13 +00:00
terencechain
3070878d59 Add Blind Blob Sidecar Signing in Validator Client (#12922)
* Add validator signing blind blob sidecar

* Fix tests

* Refactor

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-19 00:36:42 +00:00
terencechain
f59307358e Copy kzg commitments when using builder block (#12923) 2023-09-18 23:54:20 +00:00
Potuz
ef1f5e6dbe remove unused function (#12920)
* remove unused function

* remove the actual method

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-18 21:30:30 +00:00
terencechain
a22ca3fecb Update geth to v1.13.1 (#12911)
* Update geth

* Fix builder

* Update geth to v1.13.1.

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-18 21:12:07 +00:00
james-prysm
14ce051668 Deneb builder fix (#12921)
* fixing conversion for deneb

* updating unit test to catch this in the future

* gaz
2023-09-18 20:34:28 +00:00
terencechain
998a493ee2 Fix builder blind block namings (#12910)
* Fix builder blind block namings

* Fix

* Fix tests

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-18 15:32:10 +00:00
Nishant Das
398f44bb53 fix it (#12917) 2023-09-18 22:30:34 +08:00
Nishant Das
4098b3a1d2 fix it (#12915) 2023-09-18 21:42:54 +08:00
Radosław Kapka
d8e6d2cb2e Fix proposer duties sorting (#12909)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-16 14:11:57 +00:00
Potuz
6b915bab26 Dont process blocks twice (#12905)
* keep track of block being synced

* gazelle

* use maps

* shutup deepsource

* change godoc

* Radek's review

* Do not process block twice if it's already being processed

* add unit test
2023-09-15 20:11:02 +00:00
Potuz
dd73f762ec keep track of block being synced (#12903)
* keep track of block being synced

* gazelle

* use maps

* shutup deepsource

* change godoc

* Radek's review
2023-09-15 17:13:13 +00:00
anukul
4d120b53ae HTTP Beacon API: /eth/v1/beacon/headers (#12817)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-09-15 12:05:35 +02:00
terencechain
4d6b3252ae Fix deneb builder bid HTR (#12906)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-15 00:14:03 +00:00
terencechain
9bb81537c8 feat: add blob arrival gossip metric (#12888)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-14 21:35:18 +00:00
Potuz
0fdf63b565 default to 7 seconds for first aggregation (#12876)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-14 17:46:28 +00:00
Sammy Rosso
bd85b0e4e1 Update bug report template (#12891)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2023-09-14 16:41:49 +00:00
terencechain
d1562bab53 Update blind blobs bundle max commitment size (#12901)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-14 14:21:56 +00:00
terencechain
1e29877406 don't save blob sidecar syncing to head if 0 (#12892) 2023-09-14 13:17:49 +00:00
Sammy Rosso
8c39c55f05 HTTP Beacon API: /eth/v1/beacon/states/{state_id}/committees (#12879)
* Initial setup

* Gaz

* Remove original implementation

* Fix protos

* Cleanup + tests

* Cleanup

* Radek' review + fixes

* Fix merge errors

* Fix imports

* Rename error handling

* Fix broken e2e

* Gaz

* Update comment
2023-09-13 17:29:32 +00:00
Preston Van Loon
0ccfc74e86 Fix build @com_github_ethereum_c_kzg_4844//... (#12890)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-12 16:59:42 +00:00
james-prysm
9ecf4d34f5 Deneb api cleanup (#12852)
* cleanup wip

* updating tests, and updating logs

* fixing unit tests

* address radek's comments

* fixing bad alias name
2023-09-12 16:22:20 +00:00
kasey
a8793c9f21 Simplify DA check to avoid blob/block timing inconsistencies (#12882)
* collect unique idxs and avoid races

* fix init

* remove unused type (lint error)

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-12 15:32:21 +00:00
Radosław Kapka
79445d2bf6 HTTP Beacon API: /eth/v1/validator/duties (#12810)
* impl

* protos

* remove apimiddleware from e2e

* register endpoint

* get epoch from vars

* tests

* test fixes

* remove unused function

* GetProposerDuties

* proto

* fix compilation

* register

* tests

* GetSyncCommitteeDuties

* protos

* register

* tests in progress

* complete tests

* check altair epoch

* more testing

* create variable

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-12 13:57:04 +00:00
terencechain
35fc1c976f Improved Logging for Blob Sidecar (#12883)
* feat: improve blob sidecar logs

* Kasey's feedback
2023-09-11 20:46:06 +00:00
james-prysm
b0423a94af REST implementation of /eth/v1/beacon/states/{state_id}/fork (#12835)
* fork state endpoint

* removing generated files

* fixing linting

* fixing tests using old type

* moving the response object to a new file under shared

* fixing test

* gaz

* fixing generated code

* gaz

* fixing linting

* rolling back some changes

* reverting generated changes

* fixing spacing

* linting

* updating protos after develop merged

* addressing radek's comments

* addressing more radek comments
2023-09-11 18:57:17 +00:00
james-prysm
6c16e90fe9 deneb - validator beacon rest apis (#12871)
* wip

* adding tests for deneb

* adding deneb to get block

* Update validator/client/beacon-api/get_beacon_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/propose_beacon_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/propose_beacon_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/propose_beacon_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/client/beacon-api/propose_beacon_block.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-09-11 16:50:55 +00:00
kasey
c0a01bb859 Allow equivocating blobs (#12880)
* add eq test

* allow blob at diff roots; defer pruning

* avoid deferred pruning controversy

* lint: already a slice

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-09-10 02:23:37 +00:00
Bharath Vedartham
2a408a0dd8 don't send pre-genesis signed validator registration objects to relayers (#12847)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-09 18:53:02 +00:00
Raul Jordan
af16c71d6e Add Delay Option for Genesis to Prysmctl (#12878)
* add delay option to prysmctl for genesis time

* revert bazel
2023-09-08 20:44:09 +00:00
kasey
ec954ec9a6 function to merge and validate saved/new blobs (#12868)
* function to merge and validate saved/new blobs

* Make it an error to call save with an empty slice

* rename func to match type, undo var rename

* satisfy deepsource; cheap len check before compare

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-08 18:33:51 +00:00
Sammy Rosso
7781a3186b HTTP implementation of /eth/v1/config/deposit_contract (#12872)
* Add endpoint

* Protos

* Forgot to add endpoint

* fix proto double import

* Remove old test

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-08 17:05:29 +00:00
terencechain
440cf32966 fix: correct blob sidecar count (#12865)
* fix: update blob sidecar count

* fix: use beacon config instead of network config

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-08 15:37:53 +00:00
terencechain
13c69af717 fix: use correct blob root in blobs sidecar (#12866)
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
2023-09-08 14:58:33 +00:00
kasey
ca88e59ee3 Tests for init-sync DA (#12873)
* test coverage for commitmentsToCheck

* test for WithinDAPeriod

* gaz

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-09-08 10:47:34 +00:00
kasey
809a67ebcc reject blocks with more commitments than spec (#12863)
* reject blocks with more commitments than spec

* add details of excess commitments to error

* good catch, linter

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-09-07 20:21:03 +00:00
Stefan
f17898f658 add a RW lock for duties (#12861)
* add a RW lock for duties

* remove superfluous rlock

* remove defer from if

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-09-07 18:23:10 +00:00
Nishant Das
d506f9b2da Fix Execution Block Unmarshalling in Deneb (#12860)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-07 16:52:02 +00:00
Sammy Rosso
d55757500f Integrate EIP-4881 Deposit Tree Into Prysm via a Feature Flag (#11942)
* Initial spec rewrite

* Finish adding merkle tree implementation

* Last bits

* Move reverse function

* Add comments

* Add deposit tree snapshot

* Add deposit tree

* Add comments + cleanup

* Fixes

* Add missing errors

* Small fixes

* Add unhandled error

* Cleanup

* Fix unsafe file.Close

* Add missing comments

* Small fixes

* Address some of deepSource' compaints

* Add depositCount check

* Add finalizedDeposit check

* Replace pointer magic with copy()

* Add test for slice reversal

* add back bytes method

* Add package level description

* Remove zerohash gen and add additional checks

* Add additional comments

* Small lint fixes

* Forgot an error

* Small fixes

* Move Uint64ToBytesLittleEndian32 + test

* Fix uint subtraction issue

* Move mixInLength below error handling

* Fix

* Fix deposit root

* integrate 4881

* edits

* added in deposit tree fetcher

* add file

* Add remaining fetcher functions

* Add new file for inserter functions

* Fixes and additional funcs

* Cleanup

* Add

* Graph

* pushed up edits

* fix up

* Updates

* Add EIP4881 toggle flag

* Add interfaces

* Fix tests

* More changes

* Fix

* Remove generated graph

* Fix spacing

* Changes

* Fixes

* Changes

* Test Fix

* gaz

* Fix a couple tests

* Fix last tests

* define protos

* proto methods

* pushed

* regen

* Add proto funcs

* builds

* pushin up

* Fix and cleanup

* Fix spectest

* General cleanup

* add 4881 to e2e

* Remove debug statements + remove test skip

* Implement first set of  missing methods

* Replace Zerohashes + cleanup

* gazelle

* fmt

* Put back defensive check

* Add error logs

* InsertFinalizedDeposits: return an error

* Remove logging

* Radek' Review

* Lint fixes

* build

* Remove cancel

* Update beacon-chain/deterministic-genesis/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/cache/depositsnapshot/deposit_inserter.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Cleanup

* Fix panic when DepositSnapshot is nil on init

* Gofmt

* Fix RootEquivalence test

* Gofmt

* Add missing comments

* Nishant' review

* Add Insert benchmarks

* fix up copy method

* Fix deep copy

* Fix conflicts

* Return error

* Fix linter issues

* add in migration logic

* Cleanup + tests

* fix

* Fix incorrect index in test

* Fix linter

* Gofmt

* fix it

* fixes for off by 1

* gaz

* fix cast

* fix it

* remove ErrZeroIndex

* Fix merkle_tree_test

* add fallback

* add fix for insertion bug

* add many fixes

* fix empty snapshot

* clean up

* use feature

* remove check

* fix failing tests

* skip it

* fix test

* fix test again

* fix for the last time

* Apply suggestions from code review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix it

* remove cancel

* fix for voting

* addressing more comments

* fix err

* potuz's review

* one more test

* fix bad test

* make 4881 part of dev mode

* add workaround for new trie

* comment

* preston's review

* james's review

* add comment

* james review

* preston's review

* remove skipped test

* gaz

---------

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-07 03:19:32 +00:00
james-prysm
41dff74e90 fix phase0 block parsing on submitblock (#12857)
* adding fixes to address new changes in deneb

* removing logs

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-09-07 01:49:18 +00:00
kasey
4f48c551da Fix 12784 (#12822)
* add DA check to batched sync path

also fixes a da period calculation bug in the reg sync path.

* update comment

function original took a slot instead of an epoch, but was rewritten to use epoch to avoid the need to import the slot math package

Co-authored-by: terencechain <terence@prysmaticlabs.com>

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-09-07 00:26:28 +00:00
terencechain
2f6dcb34b6 fix(builder): handle no blob bundle (#12838)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-06 20:59:42 +00:00
kasey
a81518ca91 wrap ctx.Err, not the random last value of err (#12859)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-09-06 15:12:09 -05:00
Preston Van Loon
2b4c83b739 db: Use blobRotatingKey alias for readability (#12855) 2023-09-06 15:45:14 +00:00
terencechain
e3527e1e48 db: fix save blob sidecar using wrong prefix (#12849)
* db: fix save blob sidecar using wrong prefix

* tests: add more

* tests: check blobs no longer there

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-09-06 15:25:26 +00:00
terencechain
2e3b013ae7 fix(sync/rpc): use blobs by root when requesting block (#12837)
* fix(sync/rpc): use blobs by root when requesting block

Fix: use blobs by root for pending block queue (option B)

* fix: skip htr if block is older than deneb
2023-09-06 13:52:42 +00:00
Radosław Kapka
34f507f4b2 HTTP implementation of /eth/v1/beacon/pool/sync_committees (#12782)
* impl

* protos

* tests

* register endpoint

* test fix

* test fix

* remove path

* more test fixes

* cleanup

* bzl

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-01 17:58:50 +00:00
terencechain
3c98c27a32 chore(engine): remove debug log (#12836)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-01 13:22:05 +00:00
james-prysm
9a7393a2e3 Deneb produce blockv3 (#12708)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-09-01 12:51:27 +02:00
ICHINOSE Shogo
3a8be9fcf8 fix fuzz testing (#12790)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-08-31 21:08:27 +02:00
Preston Van Loon
a3212a5428 config/params: Require placeholderFields to be sorted and distinct (#12834)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-08-31 18:06:45 +00:00
james-prysm
bfecbf97db REST implementation of /eth/v1/validator/prepare_beacon_proposer (#12781)
* implementing prepare beacon proposer endpoint in full rest

* fixing type naming on validator client

* fixing linting

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/validator/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/validator/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing feedback

* fixing unit tests

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-08-31 17:02:39 +00:00
terencechain
d08e85b818 chore: improve att error (#12813) 2023-08-31 08:41:57 -05:00
terencechain
3d74ecb6d4 fix: add back non parallel proposer (#12811) 2023-08-31 08:41:57 -05:00
kasey
d0ae692f4c give deneb it's own field count config value (#12806)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
terencechain
b335eba625 fix: pbv2 condition (#12812) 2023-08-31 08:41:57 -05:00
terencechain
cd340cb0c8 fix: set current and prev epoch participations (#12814) 2023-08-31 08:41:57 -05:00
kasey
34b995d0de only initialize kzg if it will be used (#12803)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
kasey
5cf7b0458f re-use metadata bucket, group db constants (#12804)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
kasey
65aaf4878c Nishant's feedback, consistent tmp var (#12805)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
kasey
c32fe2e71b Save blob last stage init sync (#12764)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2023-08-31 08:41:57 -05:00
kasey
7068f97245 binary search is the wrong tool for the job, derp (#12757)
* binary search is the wrong tool for the job, derp

* restore protection from array index panics

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
Potuz
3a1b4a19c2 Blob channel (#12753)
* Add a new blob channel

* fix mock

* reset the channel

* keep a map of channels

* gazelle

* do not overwrite map

* remove pre-declaration
2023-08-31 08:41:57 -05:00
terencechain
7ed2a731d3 test: skip deneb e2e test with tracking (#12751)
* test: skip deneb e2e test with an issue

* Unskip more stuff
2023-08-31 08:41:57 -05:00
terencechain
f5d5a506ff fix: validator returning the right # of blobs (#12738) 2023-08-31 08:41:57 -05:00
terencechain
f49b2b9657 Update gossip_scoring_params.go (#12732) 2023-08-31 08:41:57 -05:00
terencechain
5179305071 feat: save blobs to db for subscriber (#12734) 2023-08-31 08:41:57 -05:00
james-prysm
726601221e fixing route handler and small cleanup items (#12726) 2023-08-31 08:41:57 -05:00
terencechain
83aaf043e4 Fix: payload attribute v3 return nil (#12736) 2023-08-31 08:41:57 -05:00
terencechain
96a9a6fc16 feat: add validator blob signing (#12730) 2023-08-31 08:41:57 -05:00
terencechain
6314f7fcbf feat: log blob commitment count (#12723) 2023-08-31 08:41:57 -05:00
terencechain
d210a2368f feat: implement eip4788 (#12570) 2023-08-31 08:41:57 -05:00
terencechain
708aee0fcb test:beta.1 deneb tests (#12680) 2023-08-31 08:41:57 -05:00
Potuz
57b8da08d1 Validate blobs feature (#12574) 2023-08-31 08:41:57 -05:00
james-prysm
70997e2489 renaming data to blobs in execution payload (#12664) 2023-08-31 08:41:57 -05:00
james-prysm
f602a44bee implementing publishblockv2 for deneb (#12662)
fixing broken test (#12673)
2023-08-31 08:41:57 -05:00
terencechain
6402301871 Test: add deneb spec test version beta.0 (#12610) 2023-08-31 08:41:57 -05:00
kasey
3476112d63 sync blobs in initial-sync (#12522) 2023-08-31 08:41:57 -05:00
Preston Van Loon
48f1f69695 Deneb: Implement EIP-7044 (#12577) 2023-08-31 08:41:57 -05:00
terencechain
f3ca945bb4 feat: add builder override (#12601) 2023-08-31 08:41:57 -05:00
terencechain
2bc91d20f9 feat: EIP7045 increase max att inclusion slot (#12565) 2023-08-31 08:41:57 -05:00
kasey
30ccff4465 support deneb in fullPayloadFromPayloadBody (#12649)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
james-prysm
4c1f398a96 adding ssz handling to produce block endpoint for deneb (#12632) 2023-08-31 08:41:57 -05:00
james-prysm
5a62aa4c07 adding ssz capabilities for deneb publish block v1 (#12622) 2023-08-31 08:41:57 -05:00
terencechain
b8fb602fc4 fix: use hexutil bytes for blobs bundle (#12623) 2023-08-31 08:41:57 -05:00
terencechain
a40640c9b9 fix: add blob batch flags to main/usage (#12614) 2023-08-31 08:41:57 -05:00
james-prysm
a41c01cb5e migrating get blob from eip4844 and deneb-integration (#12603) 2023-08-31 08:41:57 -05:00
james-prysm
6050501eb1 migrating Beacon API code from eip4844 to deneb-integration (#12568) 2023-08-31 08:41:57 -05:00
terencechain
590309e3fc feat(deneb): proposer rpc to handle builder flow (#12554) 2023-08-31 08:41:57 -05:00
terencechain
03fb7ed20c fix: devnet6 interop issues (#12545) 2023-08-31 08:41:57 -05:00
terencechain
95106a7533 feat(prysmctl): request blobs (#12544) 2023-08-31 08:41:57 -05:00
kasey
642988f977 ssz detection support for deneb (#12537)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
kasey
2121d1f4f4 Integrate generate genesis (#12523)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
kasey
87cd96afd3 BlobSidecarsByRoot (#12420)
* BlobSidecarsByRoot RPC handler

* BlobSidecarsByRange rpc handler (#12499)

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-31 08:41:57 -05:00
james-prysm
8a67f616f0 introducing deneb changes and blobs to builder (#12477) 2023-08-31 08:41:57 -05:00
terencechain
bcdf1c92a7 feat(config): update max blobs per block to 6 (#12512) 2023-08-31 08:41:57 -05:00
terencechain
ff6b670d95 feat(blockchain): update payload attribute for deneb (#12509) 2023-08-31 08:41:57 -05:00
terencechain
2be87f623b feat(rpc/validator): stream and use deneb block (#12510) 2023-08-31 08:41:57 -05:00
terencechain
5b1634b335 Update ProposeBeaconBlock Prysm RPC for Deneb (Non builder) (#12495) 2023-08-31 08:41:57 -05:00
terencechain
2dadf36fa9 Add data gas used field to execution payload (#12488) 2023-08-31 08:41:57 -05:00
terencechain
c10e56513d Proposer RPC: GetBlock for Deneb (#12481) 2023-08-31 08:41:57 -05:00
terencechain
7670fe7d6f Update block's commitment size (#12470) 2023-08-31 08:41:57 -05:00
terencechain
5deb1f755b Deneb: implements engine API end points (#12384) 2023-08-31 08:41:57 -05:00
james-prysm
e2750646fa validator signing deneb (#12449)
* validator signing feature migrated from eip4844 branch, added unit tests

* Update proto/prysm/v1alpha1/beacon_block.proto

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-08-31 08:41:57 -05:00
terencechain
e0c69396ed Add Deneb upgrade function (#12433) 2023-08-31 08:41:57 -05:00
terencechain
c03571138c P2P: broadcast blob (#12419) 2023-08-31 08:41:57 -05:00
terencechain
e3ac8b7745 Add Blob Gossip (#12413) 2023-08-31 08:41:57 -05:00
Radosław Kapka
319daa7f6c Deneb integration: Beacon API getStateV2 (#12424)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Fix Migration Of State (#12423)
2023-08-31 08:41:57 -05:00
terencechain
43e8ae4dca Deneb DB methods (#12379) 2023-08-31 08:41:57 -05:00
terencechain
61283e51e1 Add Deneb state (#12375) 2023-08-31 08:41:57 -05:00
terencechain
8e91df9155 Add deneb block to consensus types (#12368) 2023-08-31 08:41:57 -05:00
terencechain
bd86c7c8f3 Add deneb protobufs (#12363) 2023-08-31 08:41:57 -05:00
Raul Jordan
7bc44d9cfe Include Validation Error in ForkchoiceUpdatedResponse (#12828)
* include validation error in engine resp

* add to new payload
2023-08-29 11:28:44 -07:00
Preston Van Loon
63126fd51f Holesky support (#12821)
* Add holesky config


Copy config/params from deneb-integration

* Add --holesky flag

* Handle no genesis block error and suggest to user a workaround
2023-08-29 14:27:50 +00:00
Delweng
0ea0afb494 beacon-chain/execution: use pointer instead of value to reduce copy (#12818)
* beacon-chain/execution: use pointer instead of value to reduce value copy

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/execution: fix unit test

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
2023-08-29 13:24:54 +00:00
m3diumrare
286f6a9931 Minor typo in CLI flag (#12808)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-08-25 21:14:08 +00:00
Dhruv Bodani
4ef29a24e4 Add REST implementation of node version endpoint (#12809) 2023-08-25 13:14:08 +00:00
Dhruv Bodani
30eaddf48f Add support for validator count endpoint (#12752) 2023-08-24 23:11:41 +02:00
Nishant Das
bcf728b9ff Fix Sepolia Version (#12792) 2023-08-24 10:07:41 +00:00
terencechain
6f6c06a95b chore: remove unused proto import (#12787) 2023-08-24 01:09:06 +00:00
Preston Van Loon
5433502055 Add deneb config/params from deneb-integration (#12783)
Co-authored-by: Terence Tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-08-23 20:07:25 +00:00
3244 changed files with 225376 additions and 150493 deletions

View File

@@ -6,6 +6,12 @@ import %workspace%/build/bazelrc/debug.bazelrc
import %workspace%/build/bazelrc/hermetic-cc.bazelrc
import %workspace%/build/bazelrc/performance.bazelrc
# hermetic_cc_toolchain v3.0.1 required changes.
common --enable_platform_specific_config
build:linux --sandbox_add_mount_pair=/tmp
build:macos --sandbox_add_mount_pair=/var/tmp
build:windows --sandbox_add_mount_pair=C:\Temp
# E2E run with debug gotag
test:e2e --define gotags=debug
@@ -15,7 +21,7 @@ coverage --define=coverage_enabled=1
# Stamp binaries with git information
build --workspace_status_command=./hack/workspace_status.sh
build --define blst_disabled=false --define blst_modern=true
build --define blst_disabled=false
run --define blst_disabled=false
build:blst_disabled --define blst_disabled=true
@@ -27,6 +33,7 @@ build:minimal --@io_bazel_rules_go//go/config:tags=minimal
# Release flags
build:release --compilation_mode=opt
build:release --stamp
build:release --define pgo_enabled=1
# Build binary with cgo symbolizer for debugging / profiling.
build:cgo_symbolizer --copt=-g

View File

@@ -1 +1 @@
6.3.2
7.1.0

View File

@@ -9,8 +9,8 @@
#build:remote-cache --strategy=Genrule=standalone
# Prysm specific remote-cache properties.
#build:remote-cache --disk_cache=
build:remote-cache --remote_download_toplevel
build:remote-cache --remote_download_minimal
build:remote-cache --remote_build_event_upload=minimal
build:remote-cache --remote_cache=grpc://bazel-remote-cache:9092
build:remote-cache --experimental_remote_downloader=grpc://bazel-remote-cache:9092
build:remote-cache --remote_local_fallback
@@ -28,7 +28,10 @@ build --experimental_use_hermetic_linux_sandbox
# Import workspace options.
import %workspace%/.bazelrc
startup --host_jvm_args=-Xmx4g --host_jvm_args=-Xms2g
# Enable blake3 once it is supported in remote cache. See: https://github.com/buchgr/bazel-remote/issues/710
# startup --digest_function=blake3
startup --host_jvm_args=-Xmx8g --host_jvm_args=-Xms4g
build --experimental_strict_action_env
build --sandbox_tmpfs_path=/tmp
build --verbose_failures
@@ -38,6 +41,7 @@ build --curses=no --color=no
build --keep_going
build --test_output=errors
build --flaky_test_attempts=5
build --build_runfile_links=false # Only build runfile symlink forest when required by local action, test, or run command.
# Disabled race detection due to unstable test results under constrained environment build kite
# build --features=race

View File

@@ -11,7 +11,7 @@ name = "go"
enabled = true
[analyzers.meta]
import_paths = ["github.com/prysmaticlabs/prysm/v4"]
import_paths = ["github.com/prysmaticlabs/prysm/v5"]
[[analyzers]]
name = "test-coverage"

View File

@@ -1,59 +0,0 @@
---
name: "\U0001F41EBug report"
about: Report a bug or problem with running Prysm
---
<!--💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎
Hellooo! 😄
To help us tend to your issue faster, please search our currently open issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.
💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎💎-->
# 🐞 Bug Report
### Description
<!-- ✍️--> A clear and concise description of the problem...
### Has this worked before in a previous version?
<!-- Did this behavior use to work in the previous version? -->
<!-- ✍️--> Yes, the previous version in which this bug was not present was: ....
## 🔬 Minimal Reproduction
<!--
Please let us know how we can reproduce this issue. Include the exact method you used to run Prysm along with any flags used in your beacon chain and/or validator. Make sure you don't upload any confidential files or private keys.
-->
## 🔥 Error
<pre><code>
<!-- If the issue is accompanied by an error, please share the error logs with us below. If you have a lot of logs, place make a paste bin with your logs and share the link with us here: -->
<!-- ✍️-->
</code></pre>
## 🌍 Your Environment
**Operating System:**
<pre>
<code>
</code>
</pre>
**What version of Prysm are you running? (Which release)**
<pre>
<code>
</code>
</pre>
**Anything else relevant (validator index / public key)?**

66
.github/ISSUE_TEMPLATE/bug_report.yml vendored Normal file
View File

@@ -0,0 +1,66 @@
name: 🐞 Bug report
description: Report a bug or problem with running Prysm
labels: ["Bug"]
body:
- type: markdown
attributes:
value: |
To help us tend to your issue faster, please search our currently open issues before submitting a new one.
Existing issues often contain information about workarounds, resolution, or progress updates.
- type: textarea
id: what-happened
attributes:
label: Describe the bug
description: |
A clear and concise description of the problem...
validations:
required: true
- type: textarea
id: previous-version
attributes:
label: Has this worked before in a previous version?
description: Did this behavior use to work in the previous version?
render: text
- type: textarea
id: reproduction-steps
attributes:
label: 🔬 Minimal Reproduction
description: |
Please let us know how we can reproduce this issue.
Include the exact method you used to run Prysm along with any flags used in your beacon chain and/or validator.
Make sure you don't upload any confidential files or private keys.
placeholder: |
Steps to reproduce:
1. Start '...'
2. Then '...'
3. Check '...'
4. See error
- type: textarea
id: errors
attributes:
label: Error
description: |
If the issue is accompanied by an error, please share the error logs with us below.
If you have a lot of logs, place make a paste bin with your logs and share the link with us here:
render: text
- type: dropdown
id: platform
attributes:
label: Platform(s)
description: What platform(s) did this occur on?
multiple: true
options:
- Linux (x86)
- Linux (ARM)
- Mac (Intel)
- Mac (Apple Silicon)
- Windows (x86)
- Windows (ARM)
- type: input
attributes:
label: What version of Prysm are you running? (Which release)
description: You can check your Prysm version by running your beacon node or validator with the `--version` flag.
- type: textarea
attributes:
label: Anything else relevant (validator index / public key)?

View File

@@ -10,6 +10,7 @@
in review.
4. Note that PRs updating dependencies and new Go versions are not accepted.
Please file an issue instead.
5. A changelog entry is required for user facing issues.
-->
**What type of PR is this?**
@@ -28,3 +29,9 @@
Fixes #
**Other notes for review**
**Acknowledgements**
- [ ] I have read [CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [ ] I have made an appropriate entry to [CHANGELOG.md](https://github.com/prysmaticlabs/prysm/blob/develop/CHANGELOG.md).
- [ ] I have added a description to this PR with sufficient context for reviewers to understand this PR.

View File

@@ -1,4 +1,4 @@
FROM golang:alpine
FROM golang:1.22-alpine
COPY entrypoint.sh /entrypoint.sh

33
.github/workflows/changelog.yml vendored Normal file
View File

@@ -0,0 +1,33 @@
name: CI
on:
pull_request:
branches:
- develop
jobs:
changed_files:
runs-on: ubuntu-latest
name: Check CHANGELOG.md
steps:
- uses: actions/checkout@v4
- name: changelog modified
id: changelog-modified
uses: tj-actions/changed-files@v45
with:
files: CHANGELOG.md
- name: List all changed files
env:
ALL_CHANGED_FILES: ${{ steps.changelog-modified.outputs.all_changed_files }}
run: |
if [[ ${ALL_CHANGED_FILES[*]} =~ (^|[[:space:]])"CHANGELOG.md"($|[[:space:]]) ]];
then
echo "CHANGELOG.md was modified.";
exit 0;
else
echo "CHANGELOG.md was not modified.";
echo "Please see CHANGELOG.md and follow the instructions to add your changes to that file."
echo "In some rare scenarios, a changelog entry is not required and this CI check can be ignored."
exit 1;
fi

View File

@@ -16,9 +16,11 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.22.3'
- id: list
uses: shogo82148/actions-go-fuzz/list@v0
with:
tags: fuzz,develop
outputs:
fuzz-tests: ${{steps.list.outputs.fuzz-tests}}
@@ -34,9 +36,10 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.22.3'
- uses: shogo82148/actions-go-fuzz/run@v0
with:
packages: ${{ matrix.package }}
fuzz-regexp: ${{ matrix.func }}
fuzz-time: "20m"
fuzz-time: "20m"
tags: fuzz,develop

View File

@@ -5,6 +5,8 @@ on:
branches: [ master ]
pull_request:
branches: [ '*' ]
merge_group:
types: [checks_requested]
jobs:
formatting:
@@ -12,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Go mod tidy checker
id: gomodtidy
@@ -25,34 +27,34 @@ jobs:
GO111MODULE: on
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Go 1.20
uses: actions/setup-go@v3
uses: actions/checkout@v4
- name: Set up Go 1.22
uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.22.6'
- name: Run Gosec Security Scanner
run: | # https://github.com/securego/gosec/issues/469
export PATH=$PATH:$(go env GOPATH)/bin
go install github.com/securego/gosec/v2/cmd/gosec@v2.15.0
gosec -exclude=G307 -exclude-dir=crypto/bls/herumi ./...
go install github.com/securego/gosec/v2/cmd/gosec@v2.19.0
gosec -exclude-generated -exclude=G307 -exclude-dir=crypto/bls/herumi ./...
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Set up Go 1.20
uses: actions/setup-go@v3
- name: Set up Go 1.22
uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.22.6'
id: go
- name: Golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v5
with:
version: v1.52.2
version: v1.55.2
args: --config=.golangci.yml --out-${NO_FUTURE}format colored-line-number
build:
@@ -60,13 +62,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: '1.22.6'
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Get dependencies
run: |
@@ -75,14 +77,14 @@ jobs:
- name: Build
# Use blst tag to allow go and bazel builds for blst.
run: go build -v ./...
env:
CGO_CFLAGS: "-O -D__BLST_PORTABLE__"
env:
CGO_CFLAGS: "-O2 -D__BLST_PORTABLE__"
# fuzz leverage go tag based stubs at compile time.
# Building and testing with these tags should be checked and enforced at pre-submit.
- name: Test for fuzzing
run: go test -tags=fuzz,develop ./... -test.run=^Fuzz
env:
CGO_CFLAGS: "-O -D__BLST_PORTABLE__"
env:
CGO_CFLAGS: "-O2 -D__BLST_PORTABLE__"
# Tests run via Bazel for now...
# - name: Test

3
.gitignore vendored
View File

@@ -41,3 +41,6 @@ jwt.hex
# manual testing
tmp
# spectest coverage reports
report.txt

View File

@@ -6,21 +6,82 @@ run:
- proto
- tools/analyzers
timeout: 10m
go: '1.19'
go: '1.22.6'
linters:
disable-all: true
enable:
- gofmt
- goimports
- unused
- errcheck
- gosimple
- gocognit
- dupword
- nilerr
- whitespace
- misspell
enable-all: true
disable:
# Deprecated linters:
- deadcode
- exhaustivestruct
- golint
- govet
- ifshort
- interfacer
- maligned
- nosnakecase
- scopelint
- structcheck
- varcheck
# Disabled for now:
- asasalint
- bodyclose
- containedctx
- contextcheck
- cyclop
- depguard
- dogsled
- dupl
- durationcheck
- exhaustive
- exhaustruct
- forbidigo
- forcetypeassert
- funlen
- gci
- gochecknoglobals
- gochecknoinits
- goconst
- gocritic
- gocyclo
- godot
- godox
- goerr113
- gofumpt
- gomnd
- gomoddirectives
- gosec
- inamedparam
- interfacebloat
- ireturn
- lll
- maintidx
- makezero
- musttag
- nakedret
- nestif
- nilnil
- nlreturn
- noctx
- nolintlint
- nonamedreturns
- nosprintfhostport
- perfsprint
- prealloc
- predeclared
- promlinter
- protogetter
- revive
- staticcheck
- stylecheck
- tagalign
- tagliatelle
- thelper
- unparam
- varnamelen
- wrapcheck
- wsl
linters-settings:
gocognit:

View File

@@ -26,7 +26,6 @@ approval_rules:
only_changed_files:
paths:
- "*pb.go"
- "*pb.gw.go"
- "*.bazel"
options:
ignore_commits_by:
@@ -69,7 +68,6 @@ approval_rules:
changed_files:
ignore:
- "*pb.go"
- "*pb.gw.go"
- "*.bazel"
options:
ignore_commits_by:

View File

@@ -4,6 +4,7 @@ load("@com_github_atlassian_bazel_tools//goimports:def.bzl", "goimports")
load("@io_kubernetes_build//defs:run_in_workspace.bzl", "workspace_binary")
load("@io_bazel_rules_go//go:def.bzl", "nogo")
load("@bazel_skylib//rules:common_settings.bzl", "string_setting")
load("@prysm//tools/nogo_config:def.bzl", "nogo_config_exclude")
prefix = "github.com/prysmaticlabs/prysm"
@@ -11,7 +12,7 @@ exports_files([
"LICENSE.md",
])
# gazelle:prefix github.com/prysmaticlabs/prysm/v4
# gazelle:prefix github.com/prysmaticlabs/prysm/v5
# gazelle:map_kind go_library go_library @prysm//tools/go:def.bzl
# gazelle:map_kind go_test go_test @prysm//tools/go:def.bzl
# gazelle:map_kind go_repository go_repository @prysm//tools/go:def.bzl
@@ -54,13 +55,6 @@ alias(
visibility = ["//visibility:public"],
)
# Protobuf gRPC gateway compiler
alias(
name = "grpc_gateway_proto_compiler",
actual = "@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-grpc-gateway:go_gen_grpc_gateway",
visibility = ["//visibility:public"],
)
gometalinter(
name = "gometalinter",
config = "//:.gometalinter.json",
@@ -82,38 +76,117 @@ workspace_binary(
cmd = "@com_github_golang_lint//golint",
)
STATICCHECK_ANALYZERS = [
# Enabled static checks. See https://staticcheck.dev/docs/checks/
# Please. keep this list sorted. Don't be a bad person by inserting stuff randomly.
"sa1000",
"sa1001",
"sa1002",
"sa1003",
"sa1004",
"sa1005",
"sa1006",
"sa1007",
"sa1008",
"sa1010",
"sa1011",
"sa1012",
"sa1013",
"sa1014",
"sa1015",
"sa1016",
"sa1017",
"sa1018",
# "sa1019", # TODO: Fix all uses of deprecated things.
"sa1020",
"sa1021",
"sa1023",
"sa1024",
"sa1025",
"sa1026",
"sa1027",
"sa1028",
"sa1029",
"sa1030",
"sa2000",
"sa2001",
"sa2002",
"sa2003",
"sa3000",
"sa3001",
"sa4000",
"sa4001",
"sa4003",
"sa4004",
"sa4005",
"sa4006",
"sa4008",
"sa4009",
"sa4010",
"sa4011",
"sa4012",
"sa4013",
"sa4014",
"sa4015",
"sa4016",
"sa4017",
"sa4018",
"sa4019",
"sa4020",
"sa4021",
"sa4022",
"sa4023",
"sa4024",
"sa4025",
"sa4026",
"sa4027",
"sa4028",
"sa4029",
"sa4030",
"sa4031",
"sa4032",
"sa5000",
"sa5001",
"sa5002",
"sa5003",
"sa5004",
"sa5005",
"sa5007",
"sa5008",
"sa5009",
"sa5010",
"sa5011",
"sa5012",
"sa6000",
"sa6001",
"sa6002",
"sa6003",
"sa6005",
"sa6006",
"sa9001",
"sa9002",
#"sa9003", # Doesn't build. See https://github.com/dominikh/go-tools/pull/1483
"sa9004",
"sa9005",
"sa9006",
"sa9007",
"sa9008",
]
nogo_config_exclude(
name = "nogo_config_with_excludes",
checks = [sa.upper() for sa in STATICCHECK_ANALYZERS],
exclude_files = [
"external/.*",
],
input = "nogo_config.json",
)
nogo(
name = "nogo",
config = "nogo_config.json",
config = ":nogo_config_with_excludes",
visibility = ["//visibility:public"],
deps = [
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_default_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_default_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_default_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/shadow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_default_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_default_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_default_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_default_library",
# "@org_golang_x_tools//go/analysis/passes/cgocall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_default_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_default_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_default_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_default_library",
"//tools/analyzers/comparesame:go_default_library",
"//tools/analyzers/cryptorand:go_default_library",
"//tools/analyzers/errcheck:go_default_library",
@@ -129,6 +202,54 @@ nogo(
"//tools/analyzers/shadowpredecl:go_default_library",
"//tools/analyzers/slicedirect:go_default_library",
"//tools/analyzers/uintcast:go_default_library",
"@org_golang_x_tools//go/analysis/passes/appends:go_default_library",
"@org_golang_x_tools//go/analysis/passes/asmdecl:go_default_library",
"@org_golang_x_tools//go/analysis/passes/assign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomic:go_default_library",
"@org_golang_x_tools//go/analysis/passes/atomicalign:go_default_library",
"@org_golang_x_tools//go/analysis/passes/bools:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildssa:go_default_library",
"@org_golang_x_tools//go/analysis/passes/buildtag:go_default_library",
# cgocall disabled
#"@org_golang_x_tools//go/analysis/passes/cgocall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/copylock:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ctrlflow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/deepequalerrors:go_default_library",
"@org_golang_x_tools//go/analysis/passes/defers:go_default_library",
"@org_golang_x_tools//go/analysis/passes/directive:go_default_library",
"@org_golang_x_tools//go/analysis/passes/errorsas:go_default_library",
# fieldalignment disabled
#"@org_golang_x_tools//go/analysis/passes/fieldalignment:go_default_library",
"@org_golang_x_tools//go/analysis/passes/findcall:go_default_library",
"@org_golang_x_tools//go/analysis/passes/framepointer:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpmux:go_default_library",
"@org_golang_x_tools//go/analysis/passes/httpresponse:go_default_library",
"@org_golang_x_tools//go/analysis/passes/ifaceassert:go_default_library",
"@org_golang_x_tools//go/analysis/passes/inspect:go_default_library",
"@org_golang_x_tools//go/analysis/passes/loopclosure:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilfunc:go_default_library",
"@org_golang_x_tools//go/analysis/passes/nilness:go_default_library",
"@org_golang_x_tools//go/analysis/passes/pkgfact:go_default_library",
"@org_golang_x_tools//go/analysis/passes/printf:go_default_library",
"@org_golang_x_tools//go/analysis/passes/reflectvaluecompare:go_default_library",
# shadow disabled
#"@org_golang_x_tools//go/analysis/passes/shadow:go_default_library",
"@org_golang_x_tools//go/analysis/passes/shift:go_default_library",
"@org_golang_x_tools//go/analysis/passes/sigchanyzer:go_default_library",
"@org_golang_x_tools//go/analysis/passes/slog:go_default_library",
"@org_golang_x_tools//go/analysis/passes/sortslice:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stdmethods:go_default_library",
"@org_golang_x_tools//go/analysis/passes/stringintconv:go_default_library",
"@org_golang_x_tools//go/analysis/passes/structtag:go_default_library",
"@org_golang_x_tools//go/analysis/passes/testinggoroutine:go_default_library",
"@org_golang_x_tools//go/analysis/passes/tests:go_default_library",
"@org_golang_x_tools//go/analysis/passes/timeformat:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unmarshal:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unreachable:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unsafeptr:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unusedresult:go_default_library",
"@org_golang_x_tools//go/analysis/passes/unusedwrite:go_default_library",
"@org_golang_x_tools//go/analysis/passes/usesgenerics:go_default_library",
] + select({
# nogo checks that fail with coverage enabled.
":coverage_enabled": [],
@@ -136,7 +257,7 @@ nogo(
"@org_golang_x_tools//go/analysis/passes/composite:go_default_library",
"@org_golang_x_tools//go/analysis/passes/lostcancel:go_default_library",
],
}),
}) + ["@co_honnef_go_tools//staticcheck/%s:go_default_library" % c for c in STATICCHECK_ANALYZERS],
)
config_setting(
@@ -144,6 +265,11 @@ config_setting(
values = {"define": "coverage_enabled=1"},
)
config_setting(
name = "pgo_enabled",
values = {"define": "pgo_enabled=1"},
)
common_files = {
"//:LICENSE.md": "LICENSE.md",
"//:README.md": "README.md",

2817
CHANGELOG.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,9 @@ Excited by our work and want to get involved in building out our sharding releas
You can explore our [Open Issues](https://github.com/prysmaticlabs/prysm/issues) in-the works for our different releases. Feel free to fork our repo and start creating PRs after assigning yourself to an issue of interest. We are always chatting on [Discord](https://discord.gg/CTYGPUJ) drop us a line there if you want to get more involved or have any questions on our implementation!
> [!IMPORTANT]
> Please, **do not send pull requests for trivial changes**, such as typos, these will be rejected. These types of pull requests incur a cost to reviewers and do not provide much value to the project. If you are unsure, please open an issue first to discuss the change.
## Contribution Steps
**1. Set up Prysm following the instructions in README.md.**
@@ -120,15 +123,19 @@ $ git push myrepo feature-in-progress-branch
Navigate to your fork of the repo on GitHub. On the upper left where the current branch is listed, change the branch to your feature-in-progress-branch. Open the files that you have worked on and check to make sure they include your changes.
**16. Create a pull request.**
**16. Add an entry to CHANGELOG.md.**
Navigate your browser to https://github.com/prysmaticlabs/prysm and click on the new pull request button. In the “base” box on the left, leave the default selection “base master”, the branch that you want your changes to be applied to. In the “compare” box on the right, select feature-in-progress-branch, the branch containing the changes you want to apply. You will then be asked to answer a few questions about your pull request. After you complete the questionnaire, the pull request will appear in the list of pull requests at https://github.com/prysmaticlabs/prysm/pulls.
If your change is user facing, you must include a CHANGELOG.md entry. See the [Maintaining CHANGELOG.md](#maintaining-changelogmd) section for more information.
**17. Respond to comments by Core Contributors.**
**17. Create a pull request.**
Navigate your browser to https://github.com/prysmaticlabs/prysm and click on the new pull request button. In the “base” box on the left, leave the default selection “base master”, the branch that you want your changes to be applied to. In the “compare” box on the right, select feature-in-progress-branch, the branch containing the changes you want to apply. You will then be asked to answer a few questions about your pull request. After you complete the questionnaire, the pull request will appear in the list of pull requests at https://github.com/prysmaticlabs/prysm/pulls. Ensure that you have added an entry to CHANGELOG.md if your PR is a user-facing change. See the [Maintaining CHANGELOG.md](#maintaining-changelogmd) section for more information.
**18. Respond to comments by Core Contributors.**
Core Contributors may ask questions and request that you make edits. If you set notifications at the top of the page to “not watching,” you will still be notified by email whenever someone comments on the page of a pull request you have created. If you are asked to modify your pull request, repeat steps 8 through 15, then leave a comment to notify the Core Contributors that the pull request is ready for further review.
**18. If the number of commits becomes excessive, you may be asked to squash your commits.**
**19. If the number of commits becomes excessive, you may be asked to squash your commits.**
You can do this with an interactive rebase. Start by running the following command to determine the commit that is the base of your branch...
@@ -136,7 +143,7 @@ Core Contributors may ask questions and request that you make edits. If you set
$ git merge-base feature-in-progress-branch prysm/master
```
**19. The previous command will return a commit-hash that you should use in the following command.**
**20. The previous command will return a commit-hash that you should use in the following command.**
```
$ git rebase -i commit-hash
@@ -160,13 +167,30 @@ squash hash add a feature
Save and close the file, then a commit command will appear in the terminal that squashes the smaller commits into one. Check to be sure the commit message accurately reflects your changes and then hit enter to execute it.
**20. Update your pull request with the following command.**
**21. Update your pull request with the following command.**
```
$ git push myrepo feature-in-progress-branch -f
```
**21. Finally, again leave a comment to the Core Contributors on the pull request to let them know that the pull request has been updated.**
**22. Finally, again leave a comment to the Core Contributors on the pull request to let them know that the pull request has been updated.**
## Maintaining CHANGELOG.md
This project follows the changelog guidelines from [keepachangelog.com](https://keepachangelog.com/en/1.1.0/).
All PRs with user facing changes should have an entry in the CHANGELOG.md file and the change should be categorized in the appropriate category within the "Unreleased" section. The categories are:
- `Added` for new features.
- `Changed` for changes in existing functionality.
- `Deprecated` for soon-to-be removed features.
- `Removed` for now removed features.
- `Fixed` for any bug fixes.
- `Security` in case of vulnerabilities. Please see the [Security Policy](SECURITY.md) for responsible disclosure before adding a change with this category.
### Releasing
When a new release is made, the "Unreleased" section should be moved to a new section with the release version and the current date. Then a new "Unreleased" section is made at the top of the file with the categories listed above.
## Contributor Responsibilities

View File

@@ -55,7 +55,7 @@ bazel build //beacon-chain --config=release
## Adding / updating dependencies
1. Add your dependency as you would with go modules. I.e. `go get ...`
1. Run `gazelle update-repos -from_file=go.mod` to update the bazel managed dependencies.
1. Run `bazel run //:gazelle -- update-repos -from_file=go.mod` to update the bazel managed dependencies.
Example:

View File

@@ -30,9 +30,6 @@ You can use `bazel run //tools/genesis-state-gen` to create a deterministic gene
- **--output-ssz** string: Output filename of the SSZ marshaling of the generated genesis state
- **--config-name=interop** string: name of the beacon chain config to use when generating the state. ex mainnet|minimal|interop
**deprecated flag: use --config-name instead**
- **--mainnet-config** bool: Select whether genesis state should be generated with mainnet or minimal (default) params
The example below creates 64 validator keys, instantiates a genesis state with those 64 validators and with genesis unix timestamp 1567542540,
and finally writes a ssz encoded output to ~/Desktop/genesis.ssz. This file can be used to kickstart the beacon chain in the next section. When using the `--interop-*` flags, the beacon node will assume the `interop` config should be used, unless a different config is specified on the command line.

6
MODULE.bazel Normal file
View File

@@ -0,0 +1,6 @@
###############################################################################
# Bazel now uses Bzlmod by default to manage external dependencies.
# Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel.
#
# For more details, please check https://github.com/bazelbuild/bazel/issues/18958
###############################################################################

1653
MODULE.bazel.lock generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -2,12 +2,12 @@
[![Build status](https://badge.buildkite.com/b555891daf3614bae4284dcf365b2340cefc0089839526f096.svg?branch=master)](https://buildkite.com/prysmatic-labs/prysm)
[![Go Report Card](https://goreportcard.com/badge/github.com/prysmaticlabs/prysm)](https://goreportcard.com/report/github.com/prysmaticlabs/prysm)
[![Consensus_Spec_Version 1.3.0](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.3.0-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.3.0)
[![Consensus_Spec_Version 1.4.0](https://img.shields.io/badge/Consensus%20Spec%20Version-v1.4.0-blue.svg)](https://github.com/ethereum/consensus-specs/tree/v1.4.0)
[![Execution_API_Version 1.0.0-beta.2](https://img.shields.io/badge/Execution%20API%20Version-v1.0.0.beta.2-blue.svg)](https://github.com/ethereum/execution-apis/tree/v1.0.0-beta.2/src/engine)
[![Discord](https://user-images.githubusercontent.com/7288322/34471967-1df7808a-efbb-11e7-9088-ed0b04151291.png)](https://discord.gg/prysmaticlabs)
[![GitPOAP Badge](https://public-api.gitpoap.io/v1/repo/prysmaticlabs/prysm/badge)](https://www.gitpoap.io/gh/prysmaticlabs/prysm)
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/eth2/) specification, developed by [Prysmatic Labs](https://prysmaticlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
This is the core repository for Prysm, a [Golang](https://golang.org/) implementation of the [Ethereum Consensus](https://ethereum.org/en/developers/docs/consensus-mechanisms/#proof-of-stake) [specification](https://github.com/ethereum/consensus-specs), developed by [Offchain Labs](https://www.offchainlabs.com). See the [Changelog](https://github.com/prysmaticlabs/prysm/releases) for details of the latest releases and upcoming breaking changes.
### Getting Started

View File

@@ -1,45 +1,53 @@
## Terms of Use
# Terms of Use
Effective as of November 2, 2023
Effective as of Oct 14, 2020
By downloading, accessing or using the Prysm implementation (“Prysm”), you (referenced herein as “you” or the “user”) certify that you have read and agreed to the terms and conditions below (the “Terms”) which form a binding contract between you and Offchain Labs, Inc. (as successor in interest to Prysmatic Labs LLC) (referenced herein as “Offchain Labs”, “we” or “us”). If you do not agree to the Terms, do not download or use Prysm. Additionally, the Terms of Use available at https://arbitrum.io/tos (or any successor site, the “OCL Terms of Use”) are hereby incorporated by reference into these Terms. In the event of any conflict between provisions set forth herein and those set forth in the OCL Terms of Use, the provisions set forth herein shall control.
By downloading, accessing or using the Prysm implementation (“Prysm”), you (referenced herein as “you” or the “user”) certify that you have read and agreed to the terms and conditions below (the “Terms”) which form a binding contract between you and Prysmatic Labs (referenced herein as “we” or “us”). If you do not agree to the Terms, do not download or use Prysm.
## About Prysm
### About Prysm
Prysm is a client implementation for Ethereum consensus protocol for a proof-of-stake blockchain. To participate in the network, a user must send ETH from the Eth1.0 chain into a validator deposit contract, which will queue in the user as a validator in the system. Validators participate in proposing and voting on blocks in the protocol, and the network applies rewards/penalties based on their behavior. A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the official documentation portal, however, we do not warrant the accuracy, completeness or usefulness of this documentation. Any reliance you place on such information is strictly at your own risk.
Prysm is a client implementation for the Ethereum blockchains consensus protocol. To participate in the network, a user must send ETH from the Ethereum mainnet blockchain to a validator deposit smart contract on Ethereum mainnet. Validators participate in proposing and voting on blocks in the protocol, and the network applies rewards/penalties based on their behavior. A detailed set of installation and usage instructions as well as breakdowns of each individual component are available in the official documentation portal, however, we do not warrant the accuracy, completeness or usefulness of this documentation. Any reliance you place on such information is strictly at your own risk.
### Licensing Terms
Prysm is a fully open-source software program licensed pursuant to the GNU General Public License v3.0.
## Licensing Terms
Prysm is an open-source software program licensed pursuant to the GNU General Public License v3.0.
The Offchain Labs name, the term “Prysm” and all related names, logos, product and service names, designs and slogans are trademarks of Offchain Labs or its affiliates and/or licensors. You must not use such marks without our prior written permission.
PLEASE READ THESE TERMS CAREFULLY, AS THE OCL TERMS OF USE INCORPORATED BY REFERENCE HEREIN CONTAIN AN AGREEMENT TO ARBITRATE AND OTHER IMPORTANT INFORMATION REGARDING YOUR LEGAL RIGHTS, REMEDIES, AND OBLIGATIONS. THE AGREEMENT TO ARBITRATE REQUIRES (WITH LIMITED EXCEPTION) THAT YOU SUBMIT CLAIMS YOU HAVE AGAINST US TO BINDING AND FINAL ARBITRATION, AND FURTHER (1) YOU WILL ONLY BE PERMITTED TO PURSUE CLAIMS AGAINST OFFCHAIN LABS ON AN INDIVIDUAL BASIS, NOT AS A PLAINTIFF OR CLASS MEMBER IN ANY CLASS OR REPRESENTATIVE ACTION OR PROCEEDING, (2) YOU WILL ONLY BE PERMITTED TO SEEK RELIEF (INCLUDING MONETARY, INJUNCTIVE, AND DECLARATORY RELIEF) ON AN INDIVIDUAL BASIS, AND (3) YOU MAY NOT BE ABLE TO HAVE ANY CLAIMS YOU HAVE AGAINST US RESOLVED BY A JURY OR IN A COURT OF LAW.
The Prysmatic Labs name, the term “Prysm” and all related names, logos, product and service names, designs and slogans are trademarks of Prysmatic Labs or its affiliates and/or licensors. You must not use such marks without our prior written permission.
## Risks of Operating Prysm
### Risks of Operating Prysm
The use of Prysm and acting as a validator on the Ethereum network can lead to loss of money. Ethereum is still an experimental system and ETH remains a risky investment. You alone are responsible for your actions on Prysm including the security of your ETH and meeting any applicable minimum system requirements.
The use of Prysm and acting as a validator on the Ethereum network can lead to loss of money, tokens and value. Ethereum is still an experimental system and ETH remains a risky investment. You alone are responsible for your actions on Prysm, including the security of your ETH and meeting any applicable minimum system requirements.
Use of Prysm and the ability to receive rewards or penalties may be affected at any time by mistakes made by the user or other users, software problems such as bugs, errors, incorrectly constructed transactions, unsafe cryptographic libraries or malware affecting the network, technical failures in the hardware of a user, security problems experienced by a user and/or actions or inactions of third parties and/or events experienced by third parties, among other risks. We cannot and do not guarantee that any user of Prysm will make money, that the Prysm network will operate in accordance with the documentation or that transactions will be effective or secure.
We make no claims that Prysm is appropriate or permitted for use in any specific jurisdiction. Access to Prysm may not be legal by certain persons or in certain jurisdictions or countries. If you access Prysm, you do so on your own initiative and are responsible for compliance with local laws.
YOU ACKNOWLEDGE THAT WE ARE NOT RESPONSIBLE FOR ANY RISKS ASSOCIATED WITH YOUR USE OF PRYSM, AND CANNOT BE HELD LIABLE FOR ANY RESULTING LOSSES THAT YOU EXPERIENCE WHILE ACCESSING OR USING PRYSM.
Some Internet plans will charge an additional amount for any excess upload bandwidth used that isnt included in the plan and may terminate your connection without warning because of overuse. We advise that you check whether your Internet connection is subjected to such limitations and monitor your bandwidth use so that you can stop Prysm before you reach your upload limit.
BY ACCESSING AND USING PRYSM, YOU REPRESENT AND WARRANT THAT YOU UNDERSTAND THE INHERENT RISKS ASSOCIATED WITH USING CRYPTOGRAPHIC AND BLOCKCHAIN-BASED SYSTEMS, AND THAT YOU HAVE A WORKING KNOWLEDGE OF THE USAGE AND INTRICACIES OF DIGITAL ASSETS, SUCH AS THOSE FOLLOWING THE ETHEREUM TOKEN STANDARD (ERC-20). YOU FURTHER UNDERSTAND THAT THE MARKETS FOR DIGITAL ASSETS ARE HIGHLY VOLATILE DUE TO VARIOUS FACTORS, INCLUDING ADOPTION, SPECULATION, TECHNOLOGY, SECURITY, AND REGULATION. YOU ACKNOWLEDGE AND ACCEPT THAT THE COST AND SPEED OF TRANSACTING WITH CRYPTOGRAPHIC AND BLOCKCHAIN-BASED SYSTEMS SUCH AS ETHEREUM ARE VARIABLE AND MAY INCREASE DRAMATICALLY AT ANY TIME. YOU UNDERSTAND THAT ANYONE CAN CREATE A TOKEN, INCLUDING FAKE VERSIONS OF EXISTING TOKENS AND TOKENS THAT FALSELY CLAIM TO REPRESENT PROJECTS, AND ACKNOWLEDGE AND ACCEPT THE RISK THAT YOU MAY MISTAKENLY INTERACT WITH THOSE OR OTHER TOKENS. YOU FURTHER ACKNOWLEDGE THAT WE ARE NOT RESPONSIBLE FOR ANY OF THE VARIABLES OR RISKS DESCRIBED IN THESE TERMS. YOU UNDERSTAND AND AGREE TO ASSUME FULL RESPONSIBILITY FOR ALL OF THE RISKS OF ACCESSING AND USING PRYSM. YOU ARE SOLELY RESPONSIBLE FOR YOUR WALLETS, FOR SAFEGUARDING THE ASSOCIATED PRIVATE KEY AND FOR ANY ACTIVITY THAT OCCURS USING YOUR WALLET. WITHOUT LIMITING THE FOREGOING, YOU ALSO UNDERSTAND THAT THERE MAY BE TAX AND REGULATORY RISKS RELATED TO USING PRYSM. IT IS YOUR SOLE RESPONSIBILITY TO DETERMINE WHETHER, AND TO WHAT EXTENT, ANY TAXES APPLY TO ANY TRANSACTIONS YOU CONDUCT IN CONNECTION WITH YOUR USE OF PRYSM, AND TO WITHHOLD, COLLECT, REPORT AND REMIT THE CORRECT AMOUNTS OF TAXES TO THE APPROPRIATE TAX AUTHORITIES. DIGITAL ASSETS, BLOCKCHAIN TECHNOLOGY, AND ANY RELATED SOFTWARE AND SERVICES ARE ALSO SUBJECT TO LEGAL AND REGULATORY UNCERTAINTY IN THE UNITED STATES AND OTHER JURISDICTIONS. YOU UNDERSTAND THAT LEGISLATIVE AND REGULATORY CHANGES OR ACTIONS MAY ADVERSELY AFFECT THE USAGE, TRANSFERABILITY, TRANSACTABILITY AND ACCESSIBILITY RELATED TO PRYSM.
### Warranty Disclaimer
PRYSM IS PROVIDED ON AN “AS-IS” BASIS AND MAY INCLUDE ERRORS, OMISSIONS, OR OTHER INACCURACIES. PRYSMATIC LABS AND ITS CONTRIBUTORS MAKE NO REPRESENTATIONS OR WARRANTIES ABOUT PRYSM FOR ANY PURPOSE, AND HEREBY EXPRESSLY DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT OR ANY OTHER IMPLIED WARRANTY UNDER THE UNIFORM COMPUTER INFORMATION TRANSACTIONS ACT AS ENACTED BY ANY STATE. WE ALSO MAKE NO REPRESENTATIONS OR WARRANTIES THAT PRYSM WILL OPERATE ERROR-FREE, UNINTERRUPTED, OR IN A MANNER THAT WILL MEET YOUR REQUIREMENTS AND/OR NEEDS. THEREFORE, YOU ASSUME THE ENTIRE RISK REGARDING THE QUALITY AND/OR PERFORMANCE OF PRYSM AND ANY TRANSACTIONS ENTERED INTO THEREON.
We make no claims that Prysm is appropriate or permitted for use in any specific jurisdiction. Access to Prysm may not be legal by certain persons or in certain jurisdictions or countries. If you access Prysm, you do so on your own initiative and are responsible for compliance with all Applicable Law (as defined below), including, without limitation, for the avoidance of doubt, local laws.
### Limitation of Liability
In no event will Prysmatic Labs or any of its contributors be liable, whether in contract, warranty, tort (including negligence, whether active, passive or imputed), product liability, strict liability or other theory, breach of statutory duty or otherwise arising out of, or in connection with, your use of Prysm, for any direct, indirect, incidental, special or consequential damages (including any loss of profits or data, business interruption or other pecuniary loss, or damage, loss or other compromise of data, in each case whether direct, indirect, incidental, special or consequential) arising out of use Prysm, even if we or other users have been advised of the possibility of such damages. The foregoing limitations and disclaimers shall apply to the maximum extent permitted by applicable law, even if any remedy fails of its essential purpose. You acknowledge and agree that the limitations of liability afforded us hereunder constitute a material and actual inducement and condition to entering into these Terms, and are reasonable, fair and equitable in scope to protect our legitimate interests in light of the fact that we are not receiving consideration from you for providing Prysm.
Some Internet plans will charge additional amounts for bandwidth or any excess upload bandwidth used that isnt included in the plan and may terminate your connection without warning because of overuse. We advise that you check whether your Internet connection is subjected to any such limitations and monitor your bandwidth use and upload volumes.
### Indemnification
To the maximum extent permitted by law, you will defend, indemnify and hold Prysmatic Labs and its contributors harmless from and against any and all claims, actions, suits, investigations, or proceedings by any third party (including any party or purported party to or beneficiary or purported beneficiary of any transaction on Prysm), as well as any and all losses, liabilities,
damages, costs, and expenses (including reasonable attorneys fees) arising out of, accruing from, or in any way related to (i) your breach of the terms of this Agreement, (ii) any transaction, or the failure to occur of any transaction on Prysm, and (iii) your negligence, fraud, or willful misconduct.
## Warranty Disclaimer
### Compliance with Laws and Tax Obligations
Your use of Prysm is subject to all applicable laws of any governmental authority, including, without limitation, federal, state and foreign securities laws, tax laws, tariff and trade laws, ordinances, judgments, decrees, injunctions, writs and orders or like actions of any governmental authority and rules, regulations, orders, interpretations, licenses, and permits of any federal,
regional, state, county, municipal or other governmental authority and you agree to comply with all such laws in your use of Prysm. The users of Prysm are solely responsible to determinate what, if any, taxes apply to their ETH transactions. The owners of, or contributors to, Prysm are not responsible for determining the taxes that apply to ETH transactions.
PRYSM IS PROVIDED ON AN “AS-IS” BASIS AND MAY INCLUDE ERRORS, OMISSIONS, OR OTHER INACCURACIES. WITHOUT LIMITING ANYTHING SET FORTH ELSEWHERE IN THESE TERMS, OFFCHAIN LABS AND ITS CONTRIBUTORS MAKE NO REPRESENTATIONS OR WARRANTIES ABOUT PRYSM FOR ANY PURPOSE, AND HEREBY EXPRESSLY DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT OR ANY OTHER IMPLIED WARRANTY UNDER THE UNIFORM COMPUTER INFORMATION TRANSACTIONS ACT AS ENACTED BY ANY STATE OR OTHER GOVERNMENTAL AUTHORITY. WE ALSO MAKE NO REPRESENTATIONS OR WARRANTIES THAT PRYSM WILL OPERATE ERROR-FREE, UNINTERRUPTED, OR IN A MANNER THAT WILL MEET YOUR REQUIREMENTS AND/OR NEEDS. THEREFORE, YOU ASSUME THE ENTIRE RISK REGARDING THE QUALITY AND/OR PERFORMANCE OF PRYSM AND ANY TRANSACTIONS ENTERED INTO THEREON.
### Miscellaneous
These Terms will be construed and enforced in accordance with the laws of the state of Illinois as applied to agreements entered into and completely performed in Illinois. You agree to the personal jurisdiction by and venue in Illinois and waive any objection to such jurisdiction or venue.
## Limitation of Liability
We reserve the right to revise these Terms, and your rights and obligations are at all times subject to the then-current Terms provided on Prysm. Your continued use of Prysm constitutes acceptance of such revised Terms.
IN NO EVENT WILL OFFCHAIN LABS OR ANY OF ITS AFFILIATES OR ITS OR ANY SUCH AFFILIATES DIRECTORS, OFFICERS, EMPLOYEES, AGENTS, OR REPRESENTATIVES OR ANY CONTRIBUTORS (COLLECTIVELY, THE “OCL PARTIES”) BE LIABLE, WHETHER IN CONTRACT, WARRANTY, TORT (INCLUDING NEGLIGENCE, WHETHER ACTIVE, PASSIVE OR IMPUTED), PRODUCT LIABILITY, STRICT LIABILITY OR OTHER THEORY, BREACH OF STATUTORY DUTY OR OTHERWISE ARISING OUT OF, OR IN CONNECTION WITH, YOUR USE OF PRYSM, FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES (INCLUDING ANY LOSS OF PROFITS OR DATA, BUSINESS INTERRUPTION OR OTHER PECUNIARY LOSS, OR DAMAGE, LOSS OR OTHER COMPROMISE OF DATA, IN EACH CASE WHETHER DIRECT, INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL) ARISING OUT OF USE PRYSM, EVEN IF WE OR OTHER USERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. The foregoing limitations and disclaimers shall apply to the maximum extent permitted by Applicable Law, even if any remedy fails of its essential purpose. You acknowledge and agree that the limitations of liability afforded us hereunder constitute a material and actual inducement and condition to entering into these Terms, and are reasonable, fair and equitable in scope to protect our legitimate interests in light of the fact that we are not receiving consideration from you for providing Prysm.
These Terms constitute the entire agreement between you and Prysmatic Labs regarding use of Prysm and will supersede all prior agreements whether, written or oral. No usage of trade or other regular practice or method of dealing between the parties will be used to modify, interpret, supplement, or alter the terms of these Terms.
## Indemnification
To the maximum extent permitted by Applicable Law, you will defend, indemnify and hold each OCL Party harmless from and against any and all claims, actions, suits, investigations, or proceedings by any third party (including any party or purported party to or beneficiary or purported beneficiary of any transaction or other activity on Prysm), as well as any and all losses, liabilities, damages, costs, and expenses (including reasonable attorneys fees and costs) arising out of, accruing from, or in any way related to (i) your breach of the terms of this Agreement, (ii) any transaction, or the failure to occur of any transaction on Prysm, and (iii) your negligence, fraud, or willful misconduct.
## Compliance with Laws
Your use of Prysm is subject to all applicable laws of any governmental authority, including, without limitation, federal, state and foreign securities laws, tax laws, tariff and trade laws, ordinances, judgments, decrees, injunctions, writs and orders or like actions of any governmental authority and rules, regulations, orders, interpretations, licenses, and permits of any federal, regional, state, county, municipal or other governmental authority (collectively, “Applicable Law”) and you agree to comply with all such Applicable Law in your use of Prysm. The users of Prysm are solely responsible to determinate what, if any, taxes apply to their ETH transactions. The owners of, or contributors to, Prysm are not responsible for determining the taxes that apply to ETH transactions.
## Miscellaneous
These Terms will be governed by the laws of the State of Delaware without regard to its conflict of law provisions. With respect to any disputes or claims not subject to arbitration, as set forth in the OCL Terms of Use, you and Offchain Labs submit to the personal and exclusive jurisdiction of the state and federal courts located within New York, New York and waive any objection to such jurisdiction and venue. The failure of Offchain Labs to exercise or enforce any right or provision of these Terms will not constitute a waiver of such right or provision.
We reserve the right to revise these Terms, and your rights and obligations are at all times subject to the then-current Terms provided on Prysm. Your use of Prysm following any such revision to these Terms constitutes acceptance of such revised Terms.
These Terms constitute the entire agreement between you and Offchain Labs regarding use of Prysm and will supersede all prior agreements whether, written or oral. No usage of trade or other regular practice or method of dealing between the parties will be used to modify, interpret, supplement, or alter the terms of these Terms.
If any portion of these Terms is held invalid or unenforceable, such invalidity or enforceability will not affect the other provisions of these Terms, which will remain in full force and effect, and the invalid or unenforceable portion will be given effect to the greatest extent possible. The failure of a party to require performance of any provision will not affect that partys right to require performance at any time thereafter, nor will a waiver of any breach or default of these Terms or any provision of these Terms constitute a waiver of any subsequent breach or default or a waiver of the provision itself.
If any portion of these Terms is held invalid or unenforceable, such invalidity or enforceability will not affect the other provisions of these Terms, which will remain in full force and effect, and the invalid or unenforceable portion will be given effect to the greatest extent possible. The failure of a party to require performance of any provision will not affect that partys right to require performance at any time thereafter, nor will a waiver of any breach or default of these Terms or any provision of these Terms constitute a waiver of any subsequent breach or default or a waiver of the provision itself.

233
WORKSPACE
View File

@@ -16,12 +16,14 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
rules_pkg_dependencies()
HERMETIC_CC_TOOLCHAIN_VERSION = "v3.0.1"
http_archive(
name = "hermetic_cc_toolchain",
sha256 = "973ab22945b921ef45b8e1d6ce01ca7ce1b8a462167449a36e297438c4ec2755",
strip_prefix = "hermetic_cc_toolchain-5098046bccc15d2962f3cc8e7e53d6a2a26072dc",
sha256 = "3bc6ec127622fdceb4129cb06b6f7ab098c4d539124dde96a6318e7c32a53f7a",
urls = [
"https://github.com/uber/hermetic_cc_toolchain/archive/5098046bccc15d2962f3cc8e7e53d6a2a26072dc.tar.gz", # 2023-06-28
"https://mirror.bazel.build/github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
"https://github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
],
)
@@ -50,8 +52,6 @@ load("@prysm//tools/cross-toolchain:prysm_toolchains.bzl", "configure_prysm_tool
configure_prysm_toolchains()
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
@@ -67,10 +67,10 @@ bazel_skylib_workspace()
http_archive(
name = "bazel_gazelle",
sha256 = "29d5dafc2a5582995488c6735115d1d366fcd6a0fc2e2a153f02988706349825",
integrity = "sha256-MpOL2hbmcABjA1R5Bj2dJMYO2o15/Uc5Vj9Q0zHLMgk=",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.35.0/bazel-gazelle-v0.35.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.35.0/bazel-gazelle-v0.35.0.tar.gz",
],
)
@@ -82,9 +82,52 @@ http_archive(
)
http_archive(
name = "io_bazel_rules_docker",
sha256 = "b1e80761a8a8243d03ebca8845e9cc1ba6c82ce7c5179ce2b295cd36f7e394bf",
urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.25.0/rules_docker-v0.25.0.tar.gz"],
name = "rules_distroless",
sha256 = "e64f06e452cd153aeab81f752ccf4642955b3af319e64f7bc7a7c9252f76b10e",
strip_prefix = "rules_distroless-f5e678217b57ce3ad2f1c0204bd4e9d416255773",
url = "https://github.com/GoogleContainerTools/rules_distroless/archive/f5e678217b57ce3ad2f1c0204bd4e9d416255773.tar.gz",
)
load("@rules_distroless//distroless:dependencies.bzl", "rules_distroless_dependencies")
rules_distroless_dependencies()
http_archive(
name = "distroless",
integrity = "sha256-Cf00kUp1NyXA3LzbdyYy4Kda27wbkB8+A9MliTxq4jE=",
strip_prefix = "distroless-9dc924b9fe812eec2fa0061824dcad39eb09d0d6",
url = "https://github.com/GoogleContainerTools/distroless/archive/9dc924b9fe812eec2fa0061824dcad39eb09d0d6.tar.gz", # 2024-01-24
)
http_archive(
name = "aspect_bazel_lib",
sha256 = "f5ea76682b209cc0bd90d0f5a3b26d2f7a6a2885f0c5f615e72913f4805dbb0d",
strip_prefix = "bazel-lib-2.5.0",
url = "https://github.com/aspect-build/bazel-lib/releases/download/v2.5.0/bazel-lib-v2.5.0.tar.gz",
)
load("@aspect_bazel_lib//lib:repositories.bzl", "aspect_bazel_lib_dependencies", "aspect_bazel_lib_register_toolchains")
aspect_bazel_lib_dependencies()
aspect_bazel_lib_register_toolchains()
http_archive(
name = "rules_oci",
sha256 = "4a276e9566c03491649eef63f27c2816cc222f41ccdebd97d2c5159e84917c3b",
strip_prefix = "rules_oci-1.7.4",
url = "https://github.com/bazel-contrib/rules_oci/releases/download/v1.7.4/rules_oci-v1.7.4.tar.gz",
)
load("@rules_oci//oci:dependencies.bzl", "rules_oci_dependencies")
rules_oci_dependencies()
load("@rules_oci//oci:repositories.bzl", "LATEST_CRANE_VERSION", "oci_register_toolchains")
oci_register_toolchains(
name = "oci",
crane_version = LATEST_CRANE_VERSION,
)
http_archive(
@@ -94,10 +137,10 @@ http_archive(
# Expose internals of go_test for custom build transitions.
"//third_party:io_bazel_rules_go_test.patch",
],
sha256 = "bfc5ce70b9d1634ae54f4e7b495657a18a04e0d596785f672d35d5f505ab491a",
sha256 = "80a98277ad1311dacd837f9b16db62887702e9f1d1c4c9f796d0121a46c8e184",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.40.0/rules_go-v0.40.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.40.0/rules_go-v0.40.0.zip",
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip",
],
)
@@ -116,66 +159,37 @@ git_repository(
# gazelle args: -go_prefix github.com/gogo/protobuf -proto legacy
)
load(
"@io_bazel_rules_docker//repositories:repositories.bzl",
container_repositories = "repositories",
load("@rules_oci//oci:pull.bzl", "oci_pull")
# A multi-arch base image
oci_pull(
name = "linux_debian11_multiarch_base", # Debian bullseye
digest = "sha256:b82f113425c5b5c714151aaacd8039bc141821cdcd3c65202d42bdf9c43ae60b", # 2023-12-12
image = "gcr.io/distroless/cc-debian11",
platforms = [
"linux/amd64",
"linux/arm64/v8",
],
reproducible = True,
)
container_repositories()
load("@prysm//tools:image_deps.bzl", "prysm_image_deps")
load(
"@io_bazel_rules_docker//container:container.bzl",
"container_pull",
)
# Pulled gcr.io/distroless/cc-debian11:latest on 2022-02-23
container_pull(
name = "cc_image_base_amd64",
digest = "sha256:2a0daf90a7deb78465bfca3ef2eee6e91ce0a5706059f05d79d799a51d339523",
registry = "gcr.io",
repository = "distroless/cc-debian11",
)
# Pulled gcr.io/distroless/cc-debian11:debug on 2022-02-23
container_pull(
name = "cc_debug_image_base_amd64",
digest = "sha256:7bd596f5f200588f13a69c268eea6ce428b222b67cd7428d6a7fef95e75c052a",
registry = "gcr.io",
repository = "distroless/cc-debian11",
)
# Pulled from gcr.io/distroless/base-debian11:latest on 2022-02-23
container_pull(
name = "go_image_base_amd64",
digest = "sha256:34e682800774ecbd0954b1663d90238505f1ba5543692dbc75feef7dd4839e90",
registry = "gcr.io",
repository = "distroless/base-debian11",
)
# Pulled from gcr.io/distroless/base-debian11:debug on 2022-02-23
container_pull(
name = "go_debug_image_base_amd64",
digest = "sha256:0f503c6bfd207793bc416f20a35bf6b75d769a903c48f180ad73f60f7b60d7bd",
registry = "gcr.io",
repository = "distroless/base-debian11",
)
container_pull(
name = "alpine_cc_linux_amd64",
digest = "sha256:752aa0c9a88461ffc50c5267bb7497ef03a303e38b2c8f7f2ded9bebe5f1f00e",
registry = "index.docker.io",
repository = "pinglamb/alpine-glibc",
)
prysm_image_deps()
load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
go_rules_dependencies()
go_register_toolchains(
go_version = "1.20.7",
go_version = "1.22.4",
nogo = "@//:nogo",
)
load("//:distroless_deps.bzl", "distroless_deps")
distroless_deps()
http_archive(
name = "io_kubernetes_build",
sha256 = "b84fbd1173acee9d02a7d3698ad269fdf4f7aa081e9cecd40e012ad0ad8cfa2a",
@@ -194,8 +208,8 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "91434d5fd5e1c6eb7b0174fed2afe25e09bddf00e1e4c431db931b2cee4e7773",
url = "https://github.com/eth-clients/slashing-protection-interchange-tests/archive/b8413ca42dc92308019d0d4db52c87e9e125c4e9.tar.gz",
sha256 = "516d551cfb3e50e4ac2f42db0992f4ceb573a7cb1616d727a725c8161485329f",
url = "https://github.com/eth-clients/slashing-protection-interchange-tests/archive/refs/tags/v5.3.0.tar.gz",
)
http_archive(
@@ -213,7 +227,7 @@ filegroup(
url = "https://github.com/ethereum/EIPs/archive/5480440fe51742ed23342b68cf106cefd427e39d.tar.gz",
)
consensus_spec_version = "v1.4.0-alpha.1"
consensus_spec_version = "v1.5.0-alpha.6"
bls_test_version = "v0.1.1"
@@ -229,7 +243,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "1118a663be4a00ba00f0635eb20287157f2b2f993aed64335bfbcd04af424c2b",
integrity = "sha256-M7u/Ot/Vzorww+dFbHp0cxLyM2mezJjijCzq+LY3uvs=",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/general.tar.gz" % consensus_spec_version,
)
@@ -245,7 +259,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "acde6e10940d14f22277eda5b55b65a24623ac88e4c7a2e34134a6069f5eea82",
integrity = "sha256-deOSeLRsmHXvkRp8n2bs3HXdkGUJWWqu8KFM/QABbZg=",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/minimal.tar.gz" % consensus_spec_version,
)
@@ -261,7 +275,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "49c022f3a3478cea849ba8f877a9f7e4c1ded549edddc09993550bbc5bb192e1",
integrity = "sha256-Zz7YCf6XVf57nzSEGq9ToflJFHM0lAGwhd18l9Rf3hA=",
url = "https://github.com/ethereum/consensus-spec-tests/releases/download/%s/mainnet.tar.gz" % consensus_spec_version,
)
@@ -276,7 +290,7 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "c3e246ff01f6b7b9e9e41939954a6ff89dfca7297415f88781809165fa83267c",
integrity = "sha256-BoXckDxXnDcEmAjg/dQgf/tLiJsb6CT0aZvmWHFijrY=",
strip_prefix = "consensus-specs-" + consensus_spec_version[1:],
url = "https://github.com/ethereum/consensus-specs/archive/refs/tags/%s.tar.gz" % consensus_spec_version,
)
@@ -307,24 +321,69 @@ filegroup(
visibility = ["//visibility:public"],
)
""",
sha256 = "2701e1e1a3ec10c673fe7dbdbbe6f02c8ae8c922aebbf6e720d8c72d5458aafe",
strip_prefix = "eth2-networks-7b4897888cebef23801540236f73123e21774954",
url = "https://github.com/eth-clients/eth2-networks/archive/7b4897888cebef23801540236f73123e21774954.tar.gz",
sha256 = "77e7e3ed65e33b7bb19d30131f4c2bb39e4dfeb188ab9ae84651c3cc7600131d",
strip_prefix = "eth2-networks-934c948e69205dcf2deb87e4ae6cc140c335f94d",
url = "https://github.com/eth-clients/eth2-networks/archive/934c948e69205dcf2deb87e4ae6cc140c335f94d.tar.gz",
)
http_archive(
name = "holesky_testnet",
build_file_content = """
filegroup(
name = "configs",
srcs = [
"metadata/config.yaml",
],
visibility = ["//visibility:public"],
)
""",
integrity = "sha256-b7ZTT+olF+VXEJYNTV5jggNtCkt9dOejm1i2VE+zy+0=",
strip_prefix = "holesky-874c199423ccd180607320c38cbaca05d9a1573a",
url = "https://github.com/eth-clients/holesky/archive/874c199423ccd180607320c38cbaca05d9a1573a.tar.gz", # 2024-06-18
)
http_archive(
name = "sepolia_testnet",
build_file_content = """
filegroup(
name = "configs",
srcs = [
"metadata/config.yaml",
],
visibility = ["//visibility:public"],
)
""",
integrity = "sha256-cY/UgpCcYEhQf7JefD65FI8tn/A+rAvKhcm2/qiVdqY=",
strip_prefix = "sepolia-f2c219a93c4491cee3d90c18f2f8e82aed850eab",
url = "https://github.com/eth-clients/sepolia/archive/f2c219a93c4491cee3d90c18f2f8e82aed850eab.tar.gz", # 2024-09-19
)
http_archive(
name = "com_google_protobuf",
sha256 = "4e176116949be52b0408dfd24f8925d1eb674a781ae242a75296b17a1c721395",
strip_prefix = "protobuf-23.3",
sha256 = "9bd87b8280ef720d3240514f884e56a712f2218f0d693b48050c836028940a42",
strip_prefix = "protobuf-25.1",
urls = [
"https://github.com/protocolbuffers/protobuf/archive/v23.3.tar.gz",
"https://github.com/protocolbuffers/protobuf/archive/v25.1.tar.gz",
],
)
# Group the sources of the library so that CMake rule have access to it
all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""
# External dependencies
http_archive(
name = "googleapis",
sha256 = "9d1a930e767c93c825398b8f8692eca3fe353b9aaadedfbcf1fca2282c85df88",
strip_prefix = "googleapis-64926d52febbf298cb82a8f472ade4a3969ba922",
urls = [
"https://github.com/googleapis/googleapis/archive/64926d52febbf298cb82a8f472ade4a3969ba922.zip",
],
)
load("@googleapis//:repository_rules.bzl", "switched_rules_by_language")
switched_rules_by_language(
name = "com_google_googleapis_imports",
go = True,
)
load("//:deps.bzl", "prysm_deps")
# gazelle:repository_macro deps.bzl%prysm_deps
@@ -338,24 +397,6 @@ load("@prysm//testing/endtoend:deps.bzl", "e2e_deps")
e2e_deps()
load(
"@io_bazel_rules_docker//go:image.bzl",
_go_image_repos = "repositories",
)
# Golang images
# This is using gcr.io/distroless/base
_go_image_repos()
# CC images
# This is using gcr.io/distroless/base
load(
"@io_bazel_rules_docker//cc:image.bzl",
_cc_image_repos = "repositories",
)
_cc_image_repos()
load("@com_github_atlassian_bazel_tools//gometalinter:deps.bzl", "gometalinter_dependencies")
gometalinter_dependencies()

View File

@@ -1,8 +1,24 @@
load("@prysm//tools/go:def.bzl", "go_library")
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["headers.go"],
importpath = "github.com/prysmaticlabs/prysm/v4/api",
srcs = [
"constants.go",
"headers.go",
"jwt.go",
],
importpath = "github.com/prysmaticlabs/prysm/v5/api",
visibility = ["//visibility:public"],
deps = [
"//crypto/rand:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["jwt_test.go"],
embed = [":go_default_library"],
deps = ["//testing/require:go_default_library"],
)

View File

@@ -7,7 +7,7 @@ go_library(
"errors.go",
"options.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/client",
importpath = "github.com/prysmaticlabs/prysm/v5/api/client",
visibility = ["//visibility:public"],
deps = ["@com_github_pkg_errors//:go_default_library"],
)

View File

@@ -6,13 +6,17 @@ go_library(
"checkpoint.go",
"client.go",
"doc.go",
"health.go",
"log.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/client/beacon",
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/beacon",
visibility = ["//visibility:public"],
deps = [
"//api/client:go_default_library",
"//api/client/beacon/iface:go_default_library",
"//api/server:go_default_library",
"//api/server/structs:go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/rpc/apimiddleware:go_default_library",
"//beacon-chain/state:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
@@ -20,7 +24,6 @@ go_library(
"//encoding/ssz/detect:go_default_library",
"//io/file:go_default_library",
"//network/forks:go_default_library",
"//proto/eth/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"//time/slots:go_default_library",
@@ -36,10 +39,12 @@ go_test(
srcs = [
"checkpoint_test.go",
"client_test.go",
"health_test.go",
],
embed = [":go_default_library"],
deps = [
"//api/client:go_default_library",
"//api/client/beacon/testing:go_default_library",
"//beacon-chain/state:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
@@ -53,5 +58,6 @@ go_test(
"//testing/util:go_default_library",
"//time/slots:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@org_uber_go_mock//gomock:go_default_library",
],
)

View File

@@ -7,17 +7,17 @@ import (
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
base "github.com/prysmaticlabs/prysm/v4/api/client"
"github.com/prysmaticlabs/prysm/v4/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/v4/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v4/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v4/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v4/io/file"
"github.com/prysmaticlabs/prysm/v4/runtime/version"
"github.com/prysmaticlabs/prysm/v4/time/slots"
log "github.com/sirupsen/logrus"
base "github.com/prysmaticlabs/prysm/v5/api/client"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v5/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v5/io/file"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
"github.com/prysmaticlabs/prysm/v5/time/slots"
"github.com/sirupsen/logrus"
"golang.org/x/mod/semver"
)
@@ -74,7 +74,12 @@ func DownloadFinalizedData(ctx context.Context, client *Client) (*OriginData, er
if err != nil {
return nil, errors.Wrap(err, "error detecting chain config for finalized state")
}
log.Printf("detected supported config in remote finalized state, name=%s, fork=%s", vu.Config.ConfigName, version.String(vu.Fork))
log.WithFields(logrus.Fields{
"name": vu.Config.ConfigName,
"fork": version.String(vu.Fork),
}).Info("Detected supported config in remote finalized state")
s, err := vu.UnmarshalBeaconState(sb)
if err != nil {
return nil, errors.Wrap(err, "error unmarshaling finalized state to correct version")
@@ -108,10 +113,10 @@ func DownloadFinalizedData(ctx context.Context, client *Client) (*OriginData, er
}
log.
WithField("block_slot", b.Block().Slot()).
WithField("state_slot", s.Slot()).
WithField("state_root", hexutil.Encode(sr[:])).
WithField("block_root", hexutil.Encode(br[:])).
WithField("blockSlot", b.Block().Slot()).
WithField("stateSlot", s.Slot()).
WithField("stateRoot", hexutil.Encode(sr[:])).
WithField("blockRoot", hexutil.Encode(br[:])).
Info("Downloaded checkpoint sync state and block.")
return &OriginData{
st: s,

View File

@@ -9,22 +9,20 @@ import (
"net/http"
"testing"
"github.com/prysmaticlabs/prysm/v4/api/client"
"github.com/prysmaticlabs/prysm/v4/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v4/consensus-types/blocks"
blocktest "github.com/prysmaticlabs/prysm/v4/consensus-types/blocks/testing"
"github.com/prysmaticlabs/prysm/v4/network/forks"
ethpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v4/testing/util"
"github.com/prysmaticlabs/prysm/v4/time/slots"
"github.com/prysmaticlabs/prysm/v4/config/params"
"github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v4/runtime/version"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/api/client"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/state"
"github.com/prysmaticlabs/prysm/v5/config/params"
"github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
blocktest "github.com/prysmaticlabs/prysm/v5/consensus-types/blocks/testing"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/ssz/detect"
"github.com/prysmaticlabs/prysm/v5/network/forks"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
"github.com/prysmaticlabs/prysm/v5/testing/require"
"github.com/prysmaticlabs/prysm/v5/testing/util"
"github.com/prysmaticlabs/prysm/v5/time/slots"
)
type testRT struct {

View File

@@ -13,24 +13,23 @@ import (
"strconv"
"text/template"
"github.com/prysmaticlabs/prysm/v4/api/client"
"github.com/prysmaticlabs/prysm/v4/network/forks"
v1 "github.com/prysmaticlabs/prysm/v4/proto/eth/v1"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/beacon-chain/rpc/apimiddleware"
"github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
log "github.com/sirupsen/logrus"
"github.com/prysmaticlabs/prysm/v5/api/client"
"github.com/prysmaticlabs/prysm/v5/api/server"
"github.com/prysmaticlabs/prysm/v5/api/server/structs"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v5/network/forks"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/sirupsen/logrus"
)
const (
getSignedBlockPath = "/eth/v2/beacon/blocks"
getBlockRootPath = "/eth/v1/beacon/blocks/{{.Id}}/root"
getForkForStatePath = "/eth/v1/beacon/states/{{.Id}}/fork"
getWeakSubjectivityPath = "/eth/v1/beacon/weak_subjectivity"
getWeakSubjectivityPath = "/prysm/v1/beacon/weak_subjectivity"
getForkSchedulePath = "/eth/v1/config/fork_schedule"
getConfigSpecPath = "/eth/v1/config/spec"
getStatePath = "/eth/v2/debug/beacon/states"
@@ -148,14 +147,14 @@ func (c *Client) GetFork(ctx context.Context, stateId StateOrBlockId) (*ethpb.Fo
if err != nil {
return nil, errors.Wrapf(err, "error requesting fork by state id = %s", stateId)
}
fr := &forkResponse{}
dataWrapper := &struct{ Data *forkResponse }{Data: fr}
fr := &structs.Fork{}
dataWrapper := &struct{ Data *structs.Fork }{Data: fr}
err = json.Unmarshal(body, dataWrapper)
if err != nil {
return nil, errors.Wrap(err, "error decoding json response in GetFork")
}
return fr.Fork()
return fr.ToConsensus()
}
// GetForkSchedule retrieve all forks, past present and future, of which this node is aware.
@@ -177,12 +176,12 @@ func (c *Client) GetForkSchedule(ctx context.Context) (forks.OrderedSchedule, er
}
// GetConfigSpec retrieve the current configs of the network used by the beacon node.
func (c *Client) GetConfigSpec(ctx context.Context) (*v1.SpecResponse, error) {
func (c *Client) GetConfigSpec(ctx context.Context) (*structs.GetSpecResponse, error) {
body, err := c.Get(ctx, getConfigSpecPath)
if err != nil {
return nil, errors.Wrap(err, "error requesting configSpecPath")
}
fsr := &v1.SpecResponse{}
fsr := &structs.GetSpecResponse{}
err = json.Unmarshal(body, fsr)
if err != nil {
return nil, err
@@ -257,16 +256,16 @@ func (c *Client) GetWeakSubjectivity(ctx context.Context) (*WeakSubjectivityData
if err != nil {
return nil, err
}
v := &apimiddleware.WeakSubjectivityResponse{}
v := &structs.GetWeakSubjectivityResponse{}
err = json.Unmarshal(body, v)
if err != nil {
return nil, err
}
epoch, err := strconv.ParseUint(v.Data.Checkpoint.Epoch, 10, 64)
epoch, err := strconv.ParseUint(v.Data.WsCheckpoint.Epoch, 10, 64)
if err != nil {
return nil, err
}
blockRoot, err := hexutil.Decode(v.Data.Checkpoint.Root)
blockRoot, err := hexutil.Decode(v.Data.WsCheckpoint.Root)
if err != nil {
return nil, err
}
@@ -283,7 +282,7 @@ func (c *Client) GetWeakSubjectivity(ctx context.Context) (*WeakSubjectivityData
// SubmitChangeBLStoExecution calls a beacon API endpoint to set the withdrawal addresses based on the given signed messages.
// If the API responds with something other than OK there will be failure messages associated to the corresponding request message.
func (c *Client) SubmitChangeBLStoExecution(ctx context.Context, request []*apimiddleware.SignedBLSToExecutionChangeJson) error {
func (c *Client) SubmitChangeBLStoExecution(ctx context.Context, request []*structs.SignedBLSToExecutionChange) error {
u := c.BaseURL().ResolveReference(&url.URL{Path: changeBLStoExecutionPath})
body, err := json.Marshal(request)
if err != nil {
@@ -304,15 +303,15 @@ func (c *Client) SubmitChangeBLStoExecution(ctx context.Context, request []*apim
if resp.StatusCode != http.StatusOK {
decoder := json.NewDecoder(resp.Body)
decoder.DisallowUnknownFields()
errorJson := &apimiddleware.IndexedVerificationFailureErrorJson{}
errorJson := &server.IndexedVerificationFailureError{}
if err := decoder.Decode(errorJson); err != nil {
return errors.Wrapf(err, "failed to decode error JSON for %s", resp.Request.URL)
}
for _, failure := range errorJson.Failures {
w := request[failure.Index].Message
log.WithFields(log.Fields{
"validator_index": w.ValidatorIndex,
"withdrawal_address": w.ToExecutionAddress,
log.WithFields(logrus.Fields{
"validatorIndex": w.ValidatorIndex,
"withdrawalAddress": w.ToExecutionAddress,
}).Error(failure.Message)
}
return errors.Errorf("POST error %d: %s", errorJson.Code, errorJson.Message)
@@ -322,12 +321,12 @@ func (c *Client) SubmitChangeBLStoExecution(ctx context.Context, request []*apim
// GetBLStoExecutionChanges gets all the set withdrawal messages in the node's operation pool.
// Returns a struct representation of json response.
func (c *Client) GetBLStoExecutionChanges(ctx context.Context) (*apimiddleware.BLSToExecutionChangesPoolResponseJson, error) {
func (c *Client) GetBLStoExecutionChanges(ctx context.Context) (*structs.BLSToExecutionChangesPoolResponse, error) {
body, err := c.Get(ctx, changeBLStoExecutionPath)
if err != nil {
return nil, err
}
poolResponse := &apimiddleware.BLSToExecutionChangesPoolResponseJson{}
poolResponse := &structs.BLSToExecutionChangesPoolResponse{}
err = json.Unmarshal(body, poolResponse)
if err != nil {
return nil, err
@@ -335,48 +334,16 @@ func (c *Client) GetBLStoExecutionChanges(ctx context.Context) (*apimiddleware.B
return poolResponse, nil
}
type forkResponse struct {
PreviousVersion string `json:"previous_version"`
CurrentVersion string `json:"current_version"`
Epoch string `json:"epoch"`
}
func (f *forkResponse) Fork() (*ethpb.Fork, error) {
epoch, err := strconv.ParseUint(f.Epoch, 10, 64)
if err != nil {
return nil, err
}
cSlice, err := hexutil.Decode(f.CurrentVersion)
if err != nil {
return nil, err
}
if len(cSlice) != 4 {
return nil, fmt.Errorf("got %d byte version for CurrentVersion, expected 4 bytes. hex=%s", len(cSlice), f.CurrentVersion)
}
pSlice, err := hexutil.Decode(f.PreviousVersion)
if err != nil {
return nil, err
}
if len(pSlice) != 4 {
return nil, fmt.Errorf("got %d byte version, expected 4 bytes. version hex=%s", len(pSlice), f.PreviousVersion)
}
return &ethpb.Fork{
CurrentVersion: cSlice,
PreviousVersion: pSlice,
Epoch: primitives.Epoch(epoch),
}, nil
}
type forkScheduleResponse struct {
Data []forkResponse
Data []structs.Fork
}
func (fsr *forkScheduleResponse) OrderedForkSchedule() (forks.OrderedSchedule, error) {
ofs := make(forks.OrderedSchedule, 0)
for _, d := range fsr.Data {
epoch, err := strconv.Atoi(d.Epoch)
epoch, err := strconv.ParseUint(d.Epoch, 10, 64)
if err != nil {
return nil, err
return nil, errors.Wrapf(err, "error parsing epoch %s", d.Epoch)
}
vSlice, err := hexutil.Decode(d.CurrentVersion)
if err != nil {
@@ -388,7 +355,7 @@ func (fsr *forkScheduleResponse) OrderedForkSchedule() (forks.OrderedSchedule, e
version := bytesutil.ToBytes4(vSlice)
ofs = append(ofs, forks.ForkScheduleEntry{
Version: version,
Epoch: primitives.Epoch(uint64(epoch)),
Epoch: primitives.Epoch(epoch),
})
}
sort.Sort(ofs)

View File

@@ -4,8 +4,8 @@ import (
"net/url"
"testing"
"github.com/prysmaticlabs/prysm/v4/api/client"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/api/client"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestParseNodeVersion(t *testing.T) {

View File

@@ -0,0 +1,60 @@
package beacon
import (
"context"
"sync"
"github.com/prysmaticlabs/prysm/v5/api/client/beacon/iface"
)
type NodeHealthTracker struct {
isHealthy *bool
healthChan chan bool
node iface.HealthNode
sync.RWMutex
}
func NewNodeHealthTracker(node iface.HealthNode) *NodeHealthTracker {
return &NodeHealthTracker{
node: node,
healthChan: make(chan bool, 1),
}
}
// HealthUpdates provides a read-only channel for health updates.
func (n *NodeHealthTracker) HealthUpdates() <-chan bool {
return n.healthChan
}
func (n *NodeHealthTracker) IsHealthy() bool {
n.RLock()
defer n.RUnlock()
if n.isHealthy == nil {
return false
}
return *n.isHealthy
}
func (n *NodeHealthTracker) CheckHealth(ctx context.Context) bool {
n.Lock()
defer n.Unlock()
newStatus := n.node.IsHealthy(ctx)
if n.isHealthy == nil {
n.isHealthy = &newStatus
}
isStatusChanged := newStatus != *n.isHealthy
if isStatusChanged {
// Update the health status
n.isHealthy = &newStatus
// Send the new status to the health channel, potentially overwriting the existing value
select {
case <-n.healthChan:
n.healthChan <- newStatus
default:
n.healthChan <- newStatus
}
}
return newStatus
}

View File

@@ -0,0 +1,112 @@
package beacon
import (
"context"
"sync"
"testing"
healthTesting "github.com/prysmaticlabs/prysm/v5/api/client/beacon/testing"
"go.uber.org/mock/gomock"
)
func TestNodeHealth_IsHealthy(t *testing.T) {
tests := []struct {
name string
isHealthy bool
want bool
}{
{"initially healthy", true, true},
{"initially unhealthy", false, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
n := &NodeHealthTracker{
isHealthy: &tt.isHealthy,
healthChan: make(chan bool, 1),
}
if got := n.IsHealthy(); got != tt.want {
t.Errorf("IsHealthy() = %v, want %v", got, tt.want)
}
})
}
}
func TestNodeHealth_UpdateNodeHealth(t *testing.T) {
tests := []struct {
name string
initial bool // Initial health status
newStatus bool // Status to update to
shouldSend bool // Should a message be sent through the channel
}{
{"healthy to unhealthy", true, false, true},
{"unhealthy to healthy", false, true, true},
{"remain healthy", true, true, false},
{"remain unhealthy", false, false, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
client := healthTesting.NewMockHealthClient(ctrl)
client.EXPECT().IsHealthy(gomock.Any()).Return(tt.newStatus)
n := &NodeHealthTracker{
isHealthy: &tt.initial,
node: client,
healthChan: make(chan bool, 1),
}
s := n.CheckHealth(context.Background())
// Check if health status was updated
if s != tt.newStatus {
t.Errorf("UpdateNodeHealth() failed to update isHealthy from %v to %v", tt.initial, tt.newStatus)
}
select {
case status := <-n.HealthUpdates():
if !tt.shouldSend {
t.Errorf("UpdateNodeHealth() unexpectedly sent status %v to HealthCh", status)
} else if status != tt.newStatus {
t.Errorf("UpdateNodeHealth() sent wrong status %v, want %v", status, tt.newStatus)
}
default:
if tt.shouldSend {
t.Error("UpdateNodeHealth() did not send any status to HealthCh when expected")
}
}
})
}
}
func TestNodeHealth_Concurrency(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
client := healthTesting.NewMockHealthClient(ctrl)
n := NewNodeHealthTracker(client)
var wg sync.WaitGroup
// Number of goroutines to spawn for both reading and writing
numGoroutines := 6
wg.Add(numGoroutines * 2) // for readers and writers
// Concurrently update health status
for i := 0; i < numGoroutines; i++ {
go func() {
defer wg.Done()
client.EXPECT().IsHealthy(gomock.Any()).Return(false).Times(1)
n.CheckHealth(context.Background())
client.EXPECT().IsHealthy(gomock.Any()).Return(true).Times(1)
n.CheckHealth(context.Background())
}()
}
// Concurrently read health status
for i := 0; i < numGoroutines; i++ {
go func() {
defer wg.Done()
_ = n.IsHealthy() // Just read the value
}()
}
wg.Wait() // Wait for all goroutines to finish
}

View File

@@ -0,0 +1,8 @@
load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["health.go"],
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/beacon/iface",
visibility = ["//visibility:public"],
)

View File

@@ -0,0 +1,13 @@
package iface
import "context"
type HealthTracker interface {
HealthUpdates() <-chan bool
IsHealthy() bool
CheckHealth(ctx context.Context) bool
}
type HealthNode interface {
IsHealthy(ctx context.Context) bool
}

5
api/client/beacon/log.go Normal file
View File

@@ -0,0 +1,5 @@
package beacon
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "beacon")

View File

@@ -0,0 +1,12 @@
load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["mock.go"],
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/beacon/testing",
visibility = ["//visibility:public"],
deps = [
"//api/client/beacon/iface:go_default_library",
"@org_uber_go_mock//gomock:go_default_library",
],
)

View File

@@ -0,0 +1,59 @@
package testing
import (
"context"
"reflect"
"sync"
"github.com/prysmaticlabs/prysm/v5/api/client/beacon/iface"
"go.uber.org/mock/gomock"
)
var (
_ = iface.HealthNode(&MockHealthClient{})
)
// MockHealthClient is a mock of HealthClient interface.
type MockHealthClient struct {
ctrl *gomock.Controller
recorder *MockHealthClientMockRecorder
sync.Mutex
}
// MockHealthClientMockRecorder is the mock recorder for MockHealthClient.
type MockHealthClientMockRecorder struct {
mock *MockHealthClient
}
// IsHealthy mocks base method.
func (m *MockHealthClient) IsHealthy(arg0 context.Context) bool {
m.Lock()
defer m.Unlock()
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "IsHealthy", arg0)
ret0, ok := ret[0].(bool)
if !ok {
return false
}
return ret0
}
// EXPECT returns an object that allows the caller to indicate expected use.
func (m *MockHealthClient) EXPECT() *MockHealthClientMockRecorder {
return m.recorder
}
// IsHealthy indicates an expected call of IsHealthy.
func (mr *MockHealthClientMockRecorder) IsHealthy(arg0 any) *gomock.Call {
mr.mock.Lock()
defer mr.mock.Unlock()
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "IsHealthy", reflect.TypeOf((*MockHealthClient)(nil).IsHealthy), arg0)
}
// NewMockHealthClient creates a new mock instance.
func NewMockHealthClient(ctrl *gomock.Controller) *MockHealthClient {
mock := &MockHealthClient{ctrl: ctrl}
mock.recorder = &MockHealthClientMockRecorder{mock}
return mock
}

View File

@@ -8,9 +8,12 @@ go_library(
"errors.go",
"types.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/client/builder",
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/builder",
visibility = ["//visibility:public"],
deps = [
"//api:go_default_library",
"//api/server/structs:go_default_library",
"//config/fieldparams:go_default_library",
"//consensus-types:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/interfaces:go_default_library",
@@ -18,8 +21,7 @@ go_library(
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//monitoring/tracing:go_default_library",
"//network:go_default_library",
"//network/authorization:go_default_library",
"//monitoring/tracing/trace:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
@@ -27,7 +29,7 @@ go_library(
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_fastssz//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@io_opencensus_go//trace:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
],
)
@@ -40,16 +42,21 @@ go_test(
data = glob(["testdata/**"]),
embed = [":go_default_library"],
deps = [
"//api/server/structs:go_default_library",
"//config/fieldparams:go_default_library",
"//config/params:go_default_library",
"//consensus-types/blocks:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//runtime/version:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_golang_protobuf//proto:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_prysmaticlabs_go_bitfield//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)

View File

@@ -1,16 +1,14 @@
package builder
import (
"math/big"
"github.com/pkg/errors"
ssz "github.com/prysmaticlabs/fastssz"
consensus_types "github.com/prysmaticlabs/prysm/v4/consensus-types"
"github.com/prysmaticlabs/prysm/v4/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v4/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v4/math"
ethpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v4/runtime/version"
consensus_types "github.com/prysmaticlabs/prysm/v5/consensus-types"
"github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
)
// SignedBid is an interface describing the method set of a signed builder bid.
@@ -24,7 +22,8 @@ type SignedBid interface {
// Bid is an interface describing the method set of a builder bid.
type Bid interface {
Header() (interfaces.ExecutionData, error)
Value() []byte
BlobKzgCommitments() ([][]byte, error)
Value() primitives.Wei
Pubkey() []byte
Version() int
IsNil() bool
@@ -116,14 +115,19 @@ func (b builderBid) Header() (interfaces.ExecutionData, error) {
return blocks.WrappedExecutionPayloadHeader(b.p.Header)
}
// BlobKzgCommitments --
func (b builderBid) BlobKzgCommitments() ([][]byte, error) {
return [][]byte{}, errors.New("blob kzg commitments not available before Deneb")
}
// Version --
func (b builderBid) Version() int {
return version.Bellatrix
}
// Value --
func (b builderBid) Value() []byte {
return b.p.Value
func (b builderBid) Value() primitives.Wei {
return primitives.LittleEndianBytesToWei(b.p.Value)
}
// Pubkey --
@@ -162,8 +166,12 @@ func WrappedBuilderBidCapella(p *ethpb.BuilderBidCapella) (Bid, error) {
// Header returns the execution data interface.
func (b builderBidCapella) Header() (interfaces.ExecutionData, error) {
// We have to convert big endian to little endian because the value is coming from the execution layer.
v := big.NewInt(0).SetBytes(bytesutil.ReverseByteOrder(b.p.Value))
return blocks.WrappedExecutionPayloadHeaderCapella(b.p.Header, math.WeiToGwei(v))
return blocks.WrappedExecutionPayloadHeaderCapella(b.p.Header)
}
// BlobKzgCommitments --
func (b builderBidCapella) BlobKzgCommitments() ([][]byte, error) {
return [][]byte{}, errors.New("blob kzg commitments not available before Deneb")
}
// Version --
@@ -172,8 +180,8 @@ func (b builderBidCapella) Version() int {
}
// Value --
func (b builderBidCapella) Value() []byte {
return b.p.Value
func (b builderBidCapella) Value() primitives.Wei {
return primitives.LittleEndianBytesToWei(b.p.Value)
}
// Pubkey --
@@ -195,3 +203,90 @@ func (b builderBidCapella) HashTreeRoot() ([32]byte, error) {
func (b builderBidCapella) HashTreeRootWith(hh *ssz.Hasher) error {
return b.p.HashTreeRootWith(hh)
}
type builderBidDeneb struct {
p *ethpb.BuilderBidDeneb
}
// WrappedBuilderBidDeneb is a constructor which wraps a protobuf bid into an interface.
func WrappedBuilderBidDeneb(p *ethpb.BuilderBidDeneb) (Bid, error) {
w := builderBidDeneb{p: p}
if w.IsNil() {
return nil, consensus_types.ErrNilObjectWrapped
}
return w, nil
}
// Version --
func (b builderBidDeneb) Version() int {
return version.Deneb
}
// Value --
func (b builderBidDeneb) Value() primitives.Wei {
return primitives.LittleEndianBytesToWei(b.p.Value)
}
// Pubkey --
func (b builderBidDeneb) Pubkey() []byte {
return b.p.Pubkey
}
// IsNil --
func (b builderBidDeneb) IsNil() bool {
return b.p == nil
}
// HashTreeRoot --
func (b builderBidDeneb) HashTreeRoot() ([32]byte, error) {
return b.p.HashTreeRoot()
}
// HashTreeRootWith --
func (b builderBidDeneb) HashTreeRootWith(hh *ssz.Hasher) error {
return b.p.HashTreeRootWith(hh)
}
// Header --
func (b builderBidDeneb) Header() (interfaces.ExecutionData, error) {
// We have to convert big endian to little endian because the value is coming from the execution layer.
return blocks.WrappedExecutionPayloadHeaderDeneb(b.p.Header)
}
// BlobKzgCommitments --
func (b builderBidDeneb) BlobKzgCommitments() ([][]byte, error) {
return b.p.BlobKzgCommitments, nil
}
type signedBuilderBidDeneb struct {
p *ethpb.SignedBuilderBidDeneb
}
// WrappedSignedBuilderBidDeneb is a constructor which wraps a protobuf signed bit into an interface.
func WrappedSignedBuilderBidDeneb(p *ethpb.SignedBuilderBidDeneb) (SignedBid, error) {
w := signedBuilderBidDeneb{p: p}
if w.IsNil() {
return nil, consensus_types.ErrNilObjectWrapped
}
return w, nil
}
// Message --
func (b signedBuilderBidDeneb) Message() (Bid, error) {
return WrappedBuilderBidDeneb(b.p.Message)
}
// Signature --
func (b signedBuilderBidDeneb) Signature() []byte {
return b.p.Signature
}
// Version --
func (b signedBuilderBidDeneb) Version() int {
return version.Deneb
}
// IsNil --
func (b signedBuilderBidDeneb) IsNil() bool {
return b.p == nil
}

View File

@@ -13,16 +13,17 @@ import (
"text/template"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v4/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/monitoring/tracing"
"github.com/prysmaticlabs/prysm/v4/network"
"github.com/prysmaticlabs/prysm/v4/network/authorization"
ethpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v4/runtime/version"
"github.com/prysmaticlabs/prysm/v5/api"
"github.com/prysmaticlabs/prysm/v5/api/server/structs"
"github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/monitoring/tracing"
"github.com/prysmaticlabs/prysm/v5/monitoring/tracing/trace"
v1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
log "github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
const (
@@ -55,8 +56,8 @@ func (*requestLogger) observe(r *http.Request) (e error) {
b := bytes.NewBuffer(nil)
if r.Body == nil {
log.WithFields(log.Fields{
"body-base64": "(nil value)",
"url": r.URL.String(),
"bodyBase64": "(nil value)",
"url": r.URL.String(),
}).Info("builder http request")
return nil
}
@@ -72,8 +73,8 @@ func (*requestLogger) observe(r *http.Request) (e error) {
}
r.Body = io.NopCloser(b)
log.WithFields(log.Fields{
"body-base64": string(body),
"url": r.URL.String(),
"bodyBase64": string(body),
"url": r.URL.String(),
}).Info("builder http request")
return nil
@@ -86,7 +87,7 @@ type BuilderClient interface {
NodeURL() string
GetHeader(ctx context.Context, slot primitives.Slot, parentHash [32]byte, pubkey [48]byte) (SignedBid, error)
RegisterValidator(ctx context.Context, svr []*ethpb.SignedValidatorRegistrationV1) error
SubmitBlindedBlock(ctx context.Context, sb interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, error)
SubmitBlindedBlock(ctx context.Context, sb interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, *v1.BlobsBundle, error)
Status(ctx context.Context) error
}
@@ -101,8 +102,7 @@ type Client struct {
// `host` is the base host + port used to construct request urls. This value can be
// a URL string, or NewClient will assume an http endpoint if just `host:port` is used.
func NewClient(host string, opts ...ClientOpt) (*Client, error) {
endpoint := covertEndPoint(host)
u, err := urlForHost(endpoint.Url)
u, err := urlForHost(host)
if err != nil {
return nil, err
}
@@ -118,8 +118,7 @@ func NewClient(host string, opts ...ClientOpt) (*Client, error) {
func urlForHost(h string) (*url.URL, error) {
// try to parse as url (being permissive)
u, err := url.Parse(h)
if err == nil && u.Host != "" {
if u, err := url.Parse(h); err == nil && u.Host != "" {
return u, nil
}
// try to parse as host:port
@@ -137,7 +136,7 @@ func (c *Client) NodeURL() string {
type reqOption func(*http.Request)
// do is a generic, opinionated request function to reduce boilerplate amongst the methods in this package api/client/builder/types.go.
// do is a generic, opinionated request function to reduce boilerplate amongst the methods in this package api/client/builder.
func (c *Client) do(ctx context.Context, method string, path string, body io.Reader, opts ...reqOption) (res []byte, err error) {
ctx, span := trace.StartSpan(ctx, "builder.client.do")
defer func() {
@@ -147,7 +146,7 @@ func (c *Client) do(ctx context.Context, method string, path string, body io.Rea
u := c.baseURL.ResolveReference(&url.URL{Path: path})
span.AddAttributes(trace.StringAttribute("url", u.String()),
span.SetAttributes(trace.StringAttribute("url", u.String()),
trace.StringAttribute("method", method))
req, err := http.NewRequestWithContext(ctx, method, u.String(), body)
@@ -220,6 +219,16 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
return nil, errors.Wrapf(err, "error unmarshaling the builder GetHeader response, using slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
switch strings.ToLower(v.Version) {
case strings.ToLower(version.String(version.Deneb)):
hr := &ExecHeaderResponseDeneb{}
if err := json.Unmarshal(hb, hr); err != nil {
return nil, errors.Wrapf(err, "error unmarshaling the builder GetHeader response, using slot=%d, parentHash=%#x, pubkey=%#x", slot, parentHash, pubkey)
}
p, err := hr.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from header")
}
return WrappedSignedBuilderBidDeneb(p)
case strings.ToLower(version.String(version.Capella)):
hr := &ExecHeaderResponseCapella{}
if err := json.Unmarshal(hb, hr); err != nil {
@@ -250,16 +259,16 @@ func (c *Client) GetHeader(ctx context.Context, slot primitives.Slot, parentHash
func (c *Client) RegisterValidator(ctx context.Context, svr []*ethpb.SignedValidatorRegistrationV1) error {
ctx, span := trace.StartSpan(ctx, "builder.client.RegisterValidator")
defer span.End()
span.AddAttributes(trace.Int64Attribute("num_reqs", int64(len(svr))))
span.SetAttributes(trace.Int64Attribute("num_reqs", int64(len(svr))))
if len(svr) == 0 {
err := errors.Wrap(errMalformedRequest, "empty validator registration list")
tracing.AnnotateError(span, err)
return err
}
vs := make([]*SignedValidatorRegistration, len(svr))
vs := make([]*structs.SignedValidatorRegistration, len(svr))
for i := 0; i < len(svr); i++ {
vs[i] = &SignedValidatorRegistration{SignedValidatorRegistrationV1: svr[i]}
vs[i] = structs.SignedValidatorRegistrationFromConsensus(svr[i])
}
body, err := json.Marshal(vs)
if err != nil {
@@ -269,81 +278,75 @@ func (c *Client) RegisterValidator(ctx context.Context, svr []*ethpb.SignedValid
}
_, err = c.do(ctx, http.MethodPost, postRegisterValidatorPath, bytes.NewBuffer(body))
return err
if err != nil {
return err
}
log.WithField("num_registrations", len(svr)).Info("successfully registered validator(s) on builder")
return nil
}
var errResponseVersionMismatch = errors.New("builder API response uses a different version than requested in " + api.VersionHeader + " header")
// SubmitBlindedBlock calls the builder API endpoint that binds the validator to the builder and submits the block.
// The response is the full execution payload used to create the blinded block.
func (c *Client) SubmitBlindedBlock(ctx context.Context, sb interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, error) {
func (c *Client) SubmitBlindedBlock(ctx context.Context, sb interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, *v1.BlobsBundle, error) {
if !sb.IsBlinded() {
return nil, errNotBlinded
return nil, nil, errNotBlinded
}
switch sb.Version() {
case version.Bellatrix:
psb, err := sb.PbBlindedBellatrixBlock()
if err != nil {
return nil, errors.Wrapf(err, "could not get protobuf block")
}
b := &SignedBlindedBeaconBlockBellatrix{SignedBlindedBeaconBlockBellatrix: psb}
body, err := json.Marshal(b)
if err != nil {
return nil, errors.Wrap(err, "error encoding the SignedBlindedBeaconBlockBellatrix value body in SubmitBlindedBlock")
}
versionOpt := func(r *http.Request) {
r.Header.Add("Eth-Consensus-Version", version.String(version.Bellatrix))
}
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body), versionOpt)
if err != nil {
return nil, errors.Wrap(err, "error posting the SignedBlindedBeaconBlockBellatrix to the builder api")
}
ep := &ExecPayloadResponse{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, errors.Wrap(err, "error unmarshaling the builder SubmitBlindedBlock response")
}
if strings.ToLower(ep.Version) != version.String(version.Bellatrix) {
return nil, errors.New("not a bellatrix payload")
}
p, err := ep.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from payload")
}
return blocks.WrappedExecutionPayload(p)
case version.Capella:
psb, err := sb.PbBlindedCapellaBlock()
if err != nil {
return nil, errors.Wrapf(err, "could not get protobuf block")
}
b := &SignedBlindedBeaconBlockCapella{SignedBlindedBeaconBlockCapella: psb}
body, err := json.Marshal(b)
if err != nil {
return nil, errors.Wrap(err, "error encoding the SignedBlindedBeaconBlockCapella value body in SubmitBlindedBlockCapella")
}
versionOpt := func(r *http.Request) {
r.Header.Add("Eth-Consensus-Version", version.String(version.Capella))
}
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body), versionOpt)
if err != nil {
return nil, errors.Wrap(err, "error posting the SignedBlindedBeaconBlockCapella to the builder api")
}
ep := &ExecPayloadResponseCapella{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, errors.Wrap(err, "error unmarshaling the builder SubmitBlindedBlockCapella response")
}
if strings.ToLower(ep.Version) != version.String(version.Capella) {
return nil, errors.New("not a capella payload")
}
p, err := ep.ToProto()
if err != nil {
return nil, errors.Wrapf(err, "could not extract proto message from payload")
}
return blocks.WrappedExecutionPayloadCapella(p, 0)
default:
return nil, fmt.Errorf("unsupported block version %s", version.String(sb.Version()))
// massage the proto struct type data into the api response type.
mj, err := structs.SignedBeaconBlockMessageJsoner(sb)
if err != nil {
return nil, nil, errors.Wrap(err, "error generating blinded beacon block post request")
}
body, err := json.Marshal(mj)
if err != nil {
return nil, nil, errors.Wrap(err, "error marshaling blinded block post request to json")
}
postOpts := func(r *http.Request) {
r.Header.Add("Eth-Consensus-Version", version.String(sb.Version()))
r.Header.Set("Content-Type", api.JsonMediaType)
r.Header.Set("Accept", api.JsonMediaType)
}
// post the blinded block - the execution payload response should contain the unblinded payload, along with the
// blobs bundle if it is post deneb.
rb, err := c.do(ctx, http.MethodPost, postBlindedBeaconBlockPath, bytes.NewBuffer(body), postOpts)
if err != nil {
return nil, nil, errors.Wrap(err, "error posting the blinded block to the builder api")
}
// ExecutionPayloadResponse parses just the outer container and the Value key, enabling it to use the .Value
// key to determine which underlying data type to use to finish the unmarshaling.
ep := &ExecutionPayloadResponse{}
if err := json.Unmarshal(rb, ep); err != nil {
return nil, nil, errors.Wrap(err, "error unmarshaling the builder ExecutionPayloadResponse")
}
if strings.ToLower(ep.Version) != version.String(sb.Version()) {
return nil, nil, errors.Wrapf(errResponseVersionMismatch, "req=%s, recv=%s", strings.ToLower(ep.Version), version.String(sb.Version()))
}
// This parses the rest of the response and returns the inner data field.
pp, err := ep.ParsePayload()
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to parse execution payload from builder with version=%s", ep.Version)
}
// Get the payload as a proto.Message so it can be wrapped as an execution payload interface.
pb, err := pp.PayloadProto()
if err != nil {
return nil, nil, err
}
ed, err := blocks.NewWrappedExecutionData(pb)
if err != nil {
return nil, nil, err
}
bb, ok := pp.(BlobBundler)
if ok {
bbpb, err := bb.BundleProto()
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to extract blobs bundle from builder response with version=%s", ep.Version)
}
return ed, bbpb, nil
}
return ed, nil, nil
}
// Status asks the remote builder server for a health check. A response of 200 with an empty body is the success/healthy
@@ -365,38 +368,29 @@ func non200Err(response *http.Response) error {
}
msg := fmt.Sprintf("code=%d, url=%s, body=%s", response.StatusCode, response.Request.URL, body)
switch response.StatusCode {
case 204:
case http.StatusNoContent:
log.WithError(ErrNoContent).Debug(msg)
return ErrNoContent
case 400:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
case http.StatusBadRequest:
log.WithError(ErrBadRequest).Debug(msg)
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
return errors.Wrap(ErrBadRequest, errMessage.Message)
case 404:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
case http.StatusNotFound:
log.WithError(ErrNotFound).Debug(msg)
return errors.Wrap(ErrNotFound, errMessage.Message)
case 500:
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
return errors.Wrap(ErrNotFound, errMessage.Message)
case http.StatusInternalServerError:
log.WithError(ErrNotOK).Debug(msg)
if jsonErr := json.Unmarshal(bodyBytes, &errMessage); jsonErr != nil {
return errors.Wrap(jsonErr, "unable to read response body")
}
return errors.Wrap(ErrNotOK, errMessage.Message)
default:
log.WithError(ErrNotOK).Debug(msg)
return errors.Wrap(ErrNotOK, fmt.Sprintf("unsupported error code: %d", response.StatusCode))
}
}
func covertEndPoint(ep string) network.Endpoint {
return network.Endpoint{
Url: ep,
Auth: network.AuthorizationData{ // Auth is not used for builder.
Method: authorization.None,
Value: "",
}}
}

View File

@@ -6,21 +6,23 @@ import (
"encoding/json"
"fmt"
"io"
"math/big"
"net/http"
"net/url"
"strconv"
"testing"
"github.com/prysmaticlabs/go-bitfield"
"github.com/prysmaticlabs/prysm/v4/config/params"
"github.com/prysmaticlabs/prysm/v4/consensus-types/blocks"
types "github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v4/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/api/server/structs"
"github.com/prysmaticlabs/prysm/v5/config/params"
"github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
types "github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
log "github.com/sirupsen/logrus"
)
type roundtrip func(*http.Request) (*http.Response, error)
@@ -125,7 +127,6 @@ func TestClient_GetHeader(t *testing.T) {
var slot types.Slot = 23
parentHash := ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
pubkey := ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a")
t.Run("server error", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
@@ -197,12 +198,12 @@ func TestClient_GetHeader(t *testing.T) {
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expectedTxRoot, withdrawalsRoot))
require.Equal(t, uint64(1), bidHeader.GasUsed())
value, err := stringToUint256("652312848583266388373324160190187140051835877600158453279131187530910662656")
// this matches the value in the testExampleHeaderResponse
bidStr := "652312848583266388373324160190187140051835877600158453279131187530910662656"
value, err := stringToUint256(bidStr)
require.NoError(t, err)
require.Equal(t, fmt.Sprintf("%#x", value.SSZBytes()), fmt.Sprintf("%#x", bid.Value()))
bidValue := bytesutil.ReverseByteOrder(bid.Value())
require.DeepEqual(t, bidValue, value.Bytes())
require.DeepEqual(t, big.NewInt(0).SetBytes(bidValue), value.Int)
require.Equal(t, 0, value.Int.Cmp(primitives.WeiToBigInt(bid.Value())))
require.Equal(t, bidStr, primitives.WeiToBigInt(bid.Value()).String())
})
t.Run("capella", func(t *testing.T) {
hc := &http.Client{
@@ -229,12 +230,68 @@ func TestClient_GetHeader(t *testing.T) {
withdrawalsRoot, err := bidHeader.WithdrawalsRoot()
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expectedWithdrawalsRoot, withdrawalsRoot))
value, err := stringToUint256("652312848583266388373324160190187140051835877600158453279131187530910662656")
bidStr := "652312848583266388373324160190187140051835877600158453279131187530910662656"
value, err := stringToUint256(bidStr)
require.NoError(t, err)
require.Equal(t, fmt.Sprintf("%#x", value.SSZBytes()), fmt.Sprintf("%#x", bid.Value()))
bidValue := bytesutil.ReverseByteOrder(bid.Value())
require.DeepEqual(t, bidValue, value.Bytes())
require.DeepEqual(t, big.NewInt(0).SetBytes(bidValue), value.Int)
require.Equal(t, 0, value.Int.Cmp(primitives.WeiToBigInt(bid.Value())))
require.Equal(t, bidStr, primitives.WeiToBigInt(bid.Value()).String())
})
t.Run("deneb", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleHeaderResponseDeneb)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
h, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.NoError(t, err)
expectedWithdrawalsRoot := ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
bid, err := h.Message()
require.NoError(t, err)
bidHeader, err := bid.Header()
require.NoError(t, err)
withdrawalsRoot, err := bidHeader.WithdrawalsRoot()
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(expectedWithdrawalsRoot, withdrawalsRoot))
bidStr := "652312848583266388373324160190187140051835877600158453279131187530910662656"
value, err := stringToUint256(bidStr)
require.NoError(t, err)
require.Equal(t, 0, value.Int.Cmp(primitives.WeiToBigInt(bid.Value())))
require.Equal(t, bidStr, primitives.WeiToBigInt(bid.Value()).String())
kcgCommitments, err := bid.BlobKzgCommitments()
require.NoError(t, err)
require.Equal(t, len(kcgCommitments) > 0, true)
for i := range kcgCommitments {
require.Equal(t, len(kcgCommitments[i]) == 48, true)
}
})
t.Run("deneb, too many kzg commitments", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, expectedPath, r.URL.Path)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleHeaderResponseDenebTooManyBlobs)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
_, err := c.GetHeader(ctx, slot, bytesutil.ToBytes32(parentHash), bytesutil.ToBytes48(pubkey))
require.ErrorContains(t, "could not extract proto message from header: too many blob commitments: 7", err)
})
t.Run("unsupported version", func(t *testing.T) {
hc := &http.Client{
@@ -264,6 +321,8 @@ func TestSubmitBlindedBlock(t *testing.T) {
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, postBlindedBeaconBlockPath, r.URL.Path)
require.Equal(t, "bellatrix", r.Header.Get("Eth-Consensus-Version"))
require.Equal(t, "application/json", r.Header.Get("Content-Type"))
require.Equal(t, "application/json", r.Header.Get("Accept"))
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayload)),
@@ -277,7 +336,7 @@ func TestSubmitBlindedBlock(t *testing.T) {
}
sbbb, err := blocks.NewSignedBeaconBlock(testSignedBlindedBeaconBlockBellatrix(t))
require.NoError(t, err)
ep, err := c.SubmitBlindedBlock(ctx, sbbb)
ep, _, err := c.SubmitBlindedBlock(ctx, sbbb)
require.NoError(t, err)
require.Equal(t, true, bytes.Equal(ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"), ep.ParentHash()))
bfpg, err := stringToUint256("452312848583266388373324160190187140051835877600158453279131187530910662656")
@@ -290,6 +349,8 @@ func TestSubmitBlindedBlock(t *testing.T) {
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, postBlindedBeaconBlockPath, r.URL.Path)
require.Equal(t, "capella", r.Header.Get("Eth-Consensus-Version"))
require.Equal(t, "application/json", r.Header.Get("Content-Type"))
require.Equal(t, "application/json", r.Header.Get("Accept"))
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayloadCapella)),
@@ -303,7 +364,7 @@ func TestSubmitBlindedBlock(t *testing.T) {
}
sbb, err := blocks.NewSignedBeaconBlock(testSignedBlindedBeaconBlockCapella(t))
require.NoError(t, err)
ep, err := c.SubmitBlindedBlock(ctx, sbb)
ep, _, err := c.SubmitBlindedBlock(ctx, sbb)
require.NoError(t, err)
withdrawals, err := ep.Withdrawals()
require.NoError(t, err)
@@ -313,6 +374,46 @@ func TestSubmitBlindedBlock(t *testing.T) {
assert.DeepEqual(t, ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943"), withdrawals[0].Address)
assert.Equal(t, uint64(1), withdrawals[0].Amount)
})
t.Run("deneb", func(t *testing.T) {
test := testSignedBlindedBeaconBlockDeneb(t)
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
require.Equal(t, postBlindedBeaconBlockPath, r.URL.Path)
require.Equal(t, "deneb", r.Header.Get("Eth-Consensus-Version"))
require.Equal(t, "application/json", r.Header.Get("Content-Type"))
require.Equal(t, "application/json", r.Header.Get("Accept"))
var req structs.SignedBlindedBeaconBlockDeneb
err := json.NewDecoder(r.Body).Decode(&req)
require.NoError(t, err)
block, err := req.ToConsensus()
require.NoError(t, err)
require.DeepEqual(t, block, test)
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(bytes.NewBufferString(testExampleExecutionPayloadDeneb)),
Request: r.Clone(ctx),
}, nil
}),
}
c := &Client{
hc: hc,
baseURL: &url.URL{Host: "localhost:3500", Scheme: "http"},
}
sbb, err := blocks.NewSignedBeaconBlock(test)
require.NoError(t, err)
ep, blobBundle, err := c.SubmitBlindedBlock(ctx, sbb)
require.NoError(t, err)
withdrawals, err := ep.Withdrawals()
require.NoError(t, err)
require.Equal(t, 1, len(withdrawals))
assert.Equal(t, uint64(1), withdrawals[0].Index)
assert.Equal(t, types.ValidatorIndex(1), withdrawals[0].ValidatorIndex)
assert.DeepEqual(t, ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943"), withdrawals[0].Address)
assert.Equal(t, uint64(1), withdrawals[0].Amount)
require.NotNil(t, blobBundle)
})
t.Run("mismatched versions, expected bellatrix got capella", func(t *testing.T) {
hc := &http.Client{
Transport: roundtrip(func(r *http.Request) (*http.Response, error) {
@@ -330,13 +431,13 @@ func TestSubmitBlindedBlock(t *testing.T) {
}
sbbb, err := blocks.NewSignedBeaconBlock(testSignedBlindedBeaconBlockBellatrix(t))
require.NoError(t, err)
_, err = c.SubmitBlindedBlock(ctx, sbbb)
require.ErrorContains(t, "not a bellatrix payload", err)
_, _, err = c.SubmitBlindedBlock(ctx, sbbb)
require.ErrorIs(t, err, errResponseVersionMismatch)
})
t.Run("not blinded", func(t *testing.T) {
sbb, err := blocks.NewSignedBeaconBlock(&eth.SignedBeaconBlockBellatrix{Block: &eth.BeaconBlockBellatrix{Body: &eth.BeaconBlockBodyBellatrix{}}})
sbb, err := blocks.NewSignedBeaconBlock(&eth.SignedBeaconBlockBellatrix{Block: &eth.BeaconBlockBellatrix{Body: &eth.BeaconBlockBodyBellatrix{ExecutionPayload: &v1.ExecutionPayload{}}}})
require.NoError(t, err)
_, err = (&Client{}).SubmitBlindedBlock(ctx, sbb)
_, _, err = (&Client{}).SubmitBlindedBlock(ctx, sbb)
require.ErrorIs(t, err, errNotBlinded)
})
}
@@ -626,6 +727,155 @@ func testSignedBlindedBeaconBlockCapella(t *testing.T) *eth.SignedBlindedBeaconB
}
}
func testSignedBlindedBeaconBlockDeneb(t *testing.T) *eth.SignedBlindedBeaconBlockDeneb {
basebytes, err := bytesutil.Uint256ToSSZBytes("14074904626401341155369551180448584754667373453244490859944217516317499064576")
if err != nil {
log.Error(err)
}
return &eth.SignedBlindedBeaconBlockDeneb{
Message: &eth.BlindedBeaconBlockDeneb{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Body: &eth.BlindedBeaconBlockBodyDeneb{
RandaoReveal: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
Eth1Data: &eth.Eth1Data{
DepositRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
DepositCount: 1,
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Graffiti: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
ProposerSlashings: []*eth.ProposerSlashing{
{
Header_1: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Header_2: &eth.SignedBeaconBlockHeader{
Header: &eth.BeaconBlockHeader{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BodyRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
AttesterSlashings: []*eth.AttesterSlashing{
{
Attestation_1: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
Attestation_2: &eth.IndexedAttestation{
AttestingIndices: []uint64{1},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
Attestations: []*eth.Attestation{
{
AggregationBits: bitfield.Bitlist{0x01},
Data: &eth.AttestationData{
Slot: 1,
CommitteeIndex: 1,
BeaconBlockRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Source: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
Target: &eth.Checkpoint{
Epoch: 1,
Root: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
Deposits: []*eth.Deposit{
{
Proof: [][]byte{ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")},
Data: &eth.Deposit_Data{
PublicKey: ezDecode(t, "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"),
WithdrawalCredentials: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
Amount: 1,
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
},
VoluntaryExits: []*eth.SignedVoluntaryExit{
{
Exit: &eth.VoluntaryExit{
Epoch: 1,
ValidatorIndex: 1,
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
},
},
SyncAggregate: &eth.SyncAggregate{
SyncCommitteeSignature: make([]byte, 96),
SyncCommitteeBits: ezDecode(t, "0x6451e9f951ebf05edc01de67e593484b672877054f055903ff0df1a1a945cf30ca26bb4d4b154f94a1bc776bcf5d0efb3603e1f9b8ee2499ccdcfe2a18cef458"),
},
ExecutionPayloadHeader: &v1.ExecutionPayloadHeaderDeneb{
ParentHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
FeeRecipient: ezDecode(t, "0xabcf8e0d4e9587369b2301d0790347320302cc09"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
ReceiptsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
LogsBloom: ezDecode(t, "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"),
PrevRandao: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlockNumber: 1,
GasLimit: 1,
GasUsed: 1,
Timestamp: 1,
ExtraData: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BaseFeePerGas: basebytes,
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
TransactionsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
WithdrawalsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlobGasUsed: 1,
ExcessBlobGas: 2,
},
},
},
Signature: ezDecode(t, "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"),
}
}
func TestRequestLogger(t *testing.T) {
wo := WithObserver(&requestLogger{})
c, err := NewClient("localhost:3500", wo)

File diff suppressed because one or more lines are too long

View File

@@ -3,13 +3,14 @@ load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["mock.go"],
importpath = "github.com/prysmaticlabs/prysm/v4/api/client/builder/testing",
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/builder/testing",
visibility = ["//visibility:public"],
deps = [
"//api/client/builder:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
],
)

View File

@@ -3,11 +3,12 @@ package testing
import (
"context"
"github.com/prysmaticlabs/prysm/v4/api/client/builder"
"github.com/prysmaticlabs/prysm/v4/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
ethpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/api/client/builder"
"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
)
// MockClient is a mock implementation of BuilderClient.
@@ -40,8 +41,8 @@ func (m MockClient) RegisterValidator(_ context.Context, svr []*ethpb.SignedVali
}
// SubmitBlindedBlock --
func (MockClient) SubmitBlindedBlock(_ context.Context, _ interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, error) {
return nil, nil
func (MockClient) SubmitBlindedBlock(_ context.Context, _ interfaces.ReadOnlySignedBeaconBlock) (interfaces.ExecutionData, *v1.BlobsBundle, error) {
return nil, nil, nil
}
// Status --

View File

@@ -8,93 +8,18 @@ import (
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
types "github.com/prysmaticlabs/prysm/v4/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
v1 "github.com/prysmaticlabs/prysm/v4/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
fieldparams "github.com/prysmaticlabs/prysm/v5/config/fieldparams"
consensusblocks "github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
types "github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v5/math"
v1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
"google.golang.org/protobuf/proto"
)
// SignedValidatorRegistration a struct for signed validator registrations.
type SignedValidatorRegistration struct {
*eth.SignedValidatorRegistrationV1
}
// ValidatorRegistration a struct for validator registrations.
type ValidatorRegistration struct {
*eth.ValidatorRegistrationV1
}
// MarshalJSON returns a json representation copy of signed validator registration.
func (r *SignedValidatorRegistration) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *ValidatorRegistration `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &ValidatorRegistration{r.Message},
Signature: r.SignedValidatorRegistrationV1.Signature,
})
}
// UnmarshalJSON returns a byte representation of signed validator registration from json.
func (r *SignedValidatorRegistration) UnmarshalJSON(b []byte) error {
if r.SignedValidatorRegistrationV1 == nil {
r.SignedValidatorRegistrationV1 = &eth.SignedValidatorRegistrationV1{}
}
o := struct {
Message *ValidatorRegistration `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{}
if err := json.Unmarshal(b, &o); err != nil {
return err
}
r.Message = o.Message.ValidatorRegistrationV1
r.Signature = o.Signature
return nil
}
// MarshalJSON returns a json representation copy of validator registration.
func (r *ValidatorRegistration) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
GasLimit string `json:"gas_limit"`
Timestamp string `json:"timestamp"`
Pubkey hexutil.Bytes `json:"pubkey"`
}{
FeeRecipient: r.FeeRecipient,
GasLimit: fmt.Sprintf("%d", r.GasLimit),
Timestamp: fmt.Sprintf("%d", r.Timestamp),
Pubkey: r.Pubkey,
})
}
// UnmarshalJSON returns a byte representation of validator registration from json.
func (r *ValidatorRegistration) UnmarshalJSON(b []byte) error {
if r.ValidatorRegistrationV1 == nil {
r.ValidatorRegistrationV1 = &eth.ValidatorRegistrationV1{}
}
o := struct {
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
GasLimit string `json:"gas_limit"`
Timestamp string `json:"timestamp"`
Pubkey hexutil.Bytes `json:"pubkey"`
}{}
if err := json.Unmarshal(b, &o); err != nil {
return err
}
r.FeeRecipient = o.FeeRecipient
r.Pubkey = o.Pubkey
var err error
if r.GasLimit, err = strconv.ParseUint(o.GasLimit, 10, 64); err != nil {
return errors.Wrap(err, "failed to parse gas limit")
}
if r.Timestamp, err = strconv.ParseUint(o.Timestamp, 10, 64); err != nil {
return errors.Wrap(err, "failed to parse timestamp")
}
return nil
}
var errInvalidUint256 = errors.New("invalid Uint256")
var errDecodeUint256 = errors.New("unable to decode into Uint256")
@@ -103,14 +28,10 @@ type Uint256 struct {
*big.Int
}
func isValidUint256(bi *big.Int) bool {
return bi.Cmp(big.NewInt(0)) >= 0 && bi.BitLen() <= 256
}
func stringToUint256(s string) (Uint256, error) {
bi := new(big.Int)
_, ok := bi.SetString(s, 10)
if !ok || !isValidUint256(bi) {
if !ok || !math.IsValidUint256(bi) {
return Uint256{}, errors.Wrapf(errDecodeUint256, "value=%s", s)
}
return Uint256{Int: bi}, nil
@@ -119,7 +40,7 @@ func stringToUint256(s string) (Uint256, error) {
// sszBytesToUint256 creates a Uint256 from a ssz-style (little-endian byte slice) representation.
func sszBytesToUint256(b []byte) (Uint256, error) {
bi := bytesutil.LittleEndianBytesToBigInt(b)
if !isValidUint256(bi) {
if !math.IsValidUint256(bi) {
return Uint256{}, errors.Wrapf(errDecodeUint256, "value=%s", b)
}
return Uint256{Int: bi}, nil
@@ -127,7 +48,10 @@ func sszBytesToUint256(b []byte) (Uint256, error) {
// SSZBytes creates an ssz-style (little-endian byte slice) representation of the Uint256.
func (s Uint256) SSZBytes() []byte {
if !isValidUint256(s.Int) {
if s.Int == nil {
s.Int = big.NewInt(0)
}
if !math.IsValidUint256(s.Int) {
return []byte{}
}
return bytesutil.PadTo(bytesutil.ReverseByteOrder(s.Int.Bytes()), 32)
@@ -154,7 +78,7 @@ func (s *Uint256) UnmarshalText(t []byte) error {
if !ok {
return errors.Wrapf(errDecodeUint256, "value=%s", t)
}
if !isValidUint256(z) {
if !math.IsValidUint256(z) {
return errors.Wrapf(errDecodeUint256, "value=%s", t)
}
s.Int = z
@@ -174,7 +98,10 @@ func (s Uint256) MarshalJSON() ([]byte, error) {
// MarshalText returns a text byte representation of Uint256.
func (s Uint256) MarshalText() ([]byte, error) {
if !isValidUint256(s.Int) {
if s.Int == nil {
s.Int = big.NewInt(0)
}
if !math.IsValidUint256(s.Int) {
return nil, errors.Wrapf(errInvalidUint256, "value=%s", s.Int)
}
return []byte(s.String()), nil
@@ -200,7 +127,7 @@ type VersionResponse struct {
Version string `json:"version"`
}
// ExecHeaderResponse is a JSON representation of the builder API header response for Bellatrix.
// ExecHeaderResponse is a JSON representation of the builder API header response for Bellatrix.
type ExecHeaderResponse struct {
Version string `json:"version"`
Data struct {
@@ -229,6 +156,8 @@ func (bb *BuilderBid) ToProto() (*eth.BuilderBid, error) {
}
return &eth.BuilderBid{
Header: header,
// Note that SSZBytes() reverses byte order for the little-endian representation.
// Uint256.Bytes() is big-endian, SSZBytes takes this value and reverses it.
Value: bb.Value.SSZBytes(),
Pubkey: bb.Pubkey,
}, nil
@@ -348,6 +277,11 @@ func (r *ExecPayloadResponse) ToProto() (*v1.ExecutionPayload, error) {
return r.Data.ToProto()
}
func (r *ExecutionPayload) PayloadProto() (proto.Message, error) {
pb, err := r.ToProto()
return pb, err
}
// ToProto returns a ExecutionPayload Proto
func (p *ExecutionPayload) ToProto() (*v1.ExecutionPayload, error) {
txs := make([][]byte, len(p.Transactions))
@@ -440,6 +374,90 @@ func FromProtoCapella(payload *v1.ExecutionPayloadCapella) (ExecutionPayloadCape
}, nil
}
func FromProtoDeneb(payload *v1.ExecutionPayloadDeneb) (ExecutionPayloadDeneb, error) {
bFee, err := sszBytesToUint256(payload.BaseFeePerGas)
if err != nil {
return ExecutionPayloadDeneb{}, err
}
txs := make([]hexutil.Bytes, len(payload.Transactions))
for i := range payload.Transactions {
txs[i] = bytesutil.SafeCopyBytes(payload.Transactions[i])
}
withdrawals := make([]Withdrawal, len(payload.Withdrawals))
for i, w := range payload.Withdrawals {
withdrawals[i] = Withdrawal{
Index: Uint256{Int: big.NewInt(0).SetUint64(w.Index)},
ValidatorIndex: Uint256{Int: big.NewInt(0).SetUint64(uint64(w.ValidatorIndex))},
Address: bytesutil.SafeCopyBytes(w.Address),
Amount: Uint256{Int: big.NewInt(0).SetUint64(w.Amount)},
}
}
return ExecutionPayloadDeneb{
ParentHash: bytesutil.SafeCopyBytes(payload.ParentHash),
FeeRecipient: bytesutil.SafeCopyBytes(payload.FeeRecipient),
StateRoot: bytesutil.SafeCopyBytes(payload.StateRoot),
ReceiptsRoot: bytesutil.SafeCopyBytes(payload.ReceiptsRoot),
LogsBloom: bytesutil.SafeCopyBytes(payload.LogsBloom),
PrevRandao: bytesutil.SafeCopyBytes(payload.PrevRandao),
BlockNumber: Uint64String(payload.BlockNumber),
GasLimit: Uint64String(payload.GasLimit),
GasUsed: Uint64String(payload.GasUsed),
Timestamp: Uint64String(payload.Timestamp),
ExtraData: bytesutil.SafeCopyBytes(payload.ExtraData),
BaseFeePerGas: bFee,
BlockHash: bytesutil.SafeCopyBytes(payload.BlockHash),
Transactions: txs,
Withdrawals: withdrawals,
BlobGasUsed: Uint64String(payload.BlobGasUsed),
ExcessBlobGas: Uint64String(payload.ExcessBlobGas),
}, nil
}
var errInvalidTypeConversion = errors.New("unable to translate between api and foreign type")
// ExecutionPayloadResponseFromData converts an ExecutionData interface value to a payload response.
// This involves serializing the execution payload value so that the abstract payload envelope can be used.
func ExecutionPayloadResponseFromData(ed interfaces.ExecutionData, bundle *v1.BlobsBundle) (*ExecutionPayloadResponse, error) {
pb := ed.Proto()
var data interface{}
var err error
var ver string
switch pbStruct := pb.(type) {
case *v1.ExecutionPayload:
ver = version.String(version.Bellatrix)
data, err = FromProto(pbStruct)
if err != nil {
return nil, errors.Wrap(err, "failed to convert a Bellatrix ExecutionPayload to an API response")
}
case *v1.ExecutionPayloadCapella:
ver = version.String(version.Capella)
data, err = FromProtoCapella(pbStruct)
if err != nil {
return nil, errors.Wrap(err, "failed to convert a Capella ExecutionPayload to an API response")
}
case *v1.ExecutionPayloadDeneb:
ver = version.String(version.Deneb)
payloadStruct, err := FromProtoDeneb(pbStruct)
if err != nil {
return nil, errors.Wrap(err, "failed to convert a Deneb ExecutionPayload to an API response")
}
data = &ExecutionPayloadDenebAndBlobsBundle{
ExecutionPayload: &payloadStruct,
BlobsBundle: FromBundleProto(bundle),
}
default:
return nil, errInvalidTypeConversion
}
encoded, err := json.Marshal(data)
if err != nil {
return nil, errors.Wrapf(err, "failed to marshal execution payload version=%s", ver)
}
return &ExecutionPayloadResponse{
Version: ver,
Data: encoded,
}, nil
}
// ExecHeaderResponseCapella is the response of builder API /eth/v1/builder/header/{slot}/{parent_hash}/{pubkey} for Capella.
type ExecHeaderResponseCapella struct {
Data struct {
@@ -468,6 +486,8 @@ func (bb *BuilderBidCapella) ToProto() (*eth.BuilderBidCapella, error) {
}
return &eth.BuilderBidCapella{
Header: header,
// Note that SSZBytes() reverses byte order for the little-endian representation.
// Uint256.Bytes() is big-endian, SSZBytes takes this value and reverses it.
Value: bytesutil.SafeCopyBytes(bb.Value.SSZBytes()),
Pubkey: bytesutil.SafeCopyBytes(bb.Pubkey),
}, nil
@@ -567,6 +587,42 @@ type ExecPayloadResponseCapella struct {
Data ExecutionPayloadCapella `json:"data"`
}
// ExecutionPayloadResponse allows for unmarshaling just the Version field of the payload.
// This allows it to return different ExecutionPayload types based on the version field.
type ExecutionPayloadResponse struct {
Version string `json:"version"`
Data json.RawMessage `json:"data"`
}
// ParsedPayload can retrieve the underlying protobuf message for the given execution payload response.
type ParsedPayload interface {
PayloadProto() (proto.Message, error)
}
// BlobBundler can retrieve the underlying blob bundle protobuf message for the given execution payload response.
type BlobBundler interface {
BundleProto() (*v1.BlobsBundle, error)
}
func (r *ExecutionPayloadResponse) ParsePayload() (ParsedPayload, error) {
var toProto ParsedPayload
switch r.Version {
case version.String(version.Bellatrix):
toProto = &ExecutionPayload{}
case version.String(version.Capella):
toProto = &ExecutionPayloadCapella{}
case version.String(version.Deneb):
toProto = &ExecutionPayloadDenebAndBlobsBundle{}
default:
return nil, consensusblocks.ErrUnsupportedVersion
}
if err := json.Unmarshal(r.Data, toProto); err != nil {
return nil, errors.Wrap(err, "failed to unmarshal the response .Data field with the stated version schema")
}
return toProto, nil
}
// ExecutionPayloadCapella is a field of ExecPayloadResponseCapella.
type ExecutionPayloadCapella struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
@@ -591,6 +647,11 @@ func (r *ExecPayloadResponseCapella) ToProto() (*v1.ExecutionPayloadCapella, err
return r.Data.ToProto()
}
func (p *ExecutionPayloadCapella) PayloadProto() (proto.Message, error) {
pb, err := p.ToProto()
return pb, err
}
// ToProto returns a ExecutionPayloadCapella Proto.
func (p *ExecutionPayloadCapella) ToProto() (*v1.ExecutionPayloadCapella, error) {
txs := make([][]byte, len(p.Transactions))
@@ -638,44 +699,6 @@ type SignedBlindedBeaconBlockBellatrix struct {
*eth.SignedBlindedBeaconBlockBellatrix
}
// BlindedBeaconBlockBellatrix is a field in SignedBlindedBeaconBlockBellatrix.
type BlindedBeaconBlockBellatrix struct {
*eth.BlindedBeaconBlockBellatrix
}
// BlindedBeaconBlockBodyBellatrix is a field in BlindedBeaconBlockBellatrix.
type BlindedBeaconBlockBodyBellatrix struct {
*eth.BlindedBeaconBlockBodyBellatrix
}
// MarshalJSON returns a JSON byte array representation of SignedBlindedBeaconBlockBellatrix.
func (r *SignedBlindedBeaconBlockBellatrix) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *BlindedBeaconBlockBellatrix `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &BlindedBeaconBlockBellatrix{r.SignedBlindedBeaconBlockBellatrix.Block},
Signature: r.SignedBlindedBeaconBlockBellatrix.Signature,
})
}
// MarshalJSON returns a JSON byte array representation of BlindedBeaconBlockBellatrix.
func (b *BlindedBeaconBlockBellatrix) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot hexutil.Bytes `json:"parent_root"`
StateRoot hexutil.Bytes `json:"state_root"`
Body *BlindedBeaconBlockBodyBellatrix `json:"body"`
}{
Slot: fmt.Sprintf("%d", b.Slot),
ProposerIndex: fmt.Sprintf("%d", b.ProposerIndex),
ParentRoot: b.ParentRoot,
StateRoot: b.StateRoot,
Body: &BlindedBeaconBlockBodyBellatrix{b.BlindedBeaconBlockBellatrix.Body},
})
}
// ProposerSlashing is a field in BlindedBeaconBlockBodyCapella.
type ProposerSlashing struct {
*eth.ProposerSlashing
@@ -930,53 +953,6 @@ func (e *Eth1Data) MarshalJSON() ([]byte, error) {
})
}
// MarshalJSON returns a JSON byte array representation of BlindedBeaconBlockBodyBellatrix.
func (b *BlindedBeaconBlockBodyBellatrix) MarshalJSON() ([]byte, error) {
sve := make([]*SignedVoluntaryExit, len(b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits))
for i := range b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits {
sve[i] = &SignedVoluntaryExit{SignedVoluntaryExit: b.BlindedBeaconBlockBodyBellatrix.VoluntaryExits[i]}
}
deps := make([]*Deposit, len(b.BlindedBeaconBlockBodyBellatrix.Deposits))
for i := range b.BlindedBeaconBlockBodyBellatrix.Deposits {
deps[i] = &Deposit{Deposit: b.BlindedBeaconBlockBodyBellatrix.Deposits[i]}
}
atts := make([]*Attestation, len(b.BlindedBeaconBlockBodyBellatrix.Attestations))
for i := range b.BlindedBeaconBlockBodyBellatrix.Attestations {
atts[i] = &Attestation{Attestation: b.BlindedBeaconBlockBodyBellatrix.Attestations[i]}
}
atsl := make([]*AttesterSlashing, len(b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings))
for i := range b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings {
atsl[i] = &AttesterSlashing{AttesterSlashing: b.BlindedBeaconBlockBodyBellatrix.AttesterSlashings[i]}
}
pros := make([]*ProposerSlashing, len(b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings))
for i := range b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings {
pros[i] = &ProposerSlashing{ProposerSlashing: b.BlindedBeaconBlockBodyBellatrix.ProposerSlashings[i]}
}
return json.Marshal(struct {
RandaoReveal hexutil.Bytes `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti hexutil.Bytes `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeader `json:"execution_payload_header"`
}{
RandaoReveal: b.RandaoReveal,
Eth1Data: &Eth1Data{b.BlindedBeaconBlockBodyBellatrix.Eth1Data},
Graffiti: b.BlindedBeaconBlockBodyBellatrix.Graffiti,
ProposerSlashings: pros,
AttesterSlashings: atsl,
Attestations: atts,
Deposits: deps,
VoluntaryExits: sve,
SyncAggregate: &SyncAggregate{b.BlindedBeaconBlockBodyBellatrix.SyncAggregate},
ExecutionPayloadHeader: &ExecutionPayloadHeader{ExecutionPayloadHeader: b.BlindedBeaconBlockBodyBellatrix.ExecutionPayloadHeader},
})
}
// SignedBLSToExecutionChange is a field in Beacon Block Body for capella and above.
type SignedBLSToExecutionChange struct {
*eth.SignedBLSToExecutionChange
@@ -1011,100 +987,323 @@ func (ch *BLSToExecutionChange) MarshalJSON() ([]byte, error) {
})
}
// SignedBlindedBeaconBlockCapella is part of the request object sent to builder API /eth/v1/builder/blinded_blocks for Capella.
type SignedBlindedBeaconBlockCapella struct {
*eth.SignedBlindedBeaconBlockCapella
// ExecHeaderResponseDeneb is the header response for builder API /eth/v1/builder/header/{slot}/{parent_hash}/{pubkey}.
type ExecHeaderResponseDeneb struct {
Data struct {
Signature hexutil.Bytes `json:"signature"`
Message *BuilderBidDeneb `json:"message"`
} `json:"data"`
}
// BlindedBeaconBlockCapella is a field in SignedBlindedBeaconBlockCapella.
type BlindedBeaconBlockCapella struct {
*eth.BlindedBeaconBlockCapella
// ToProto creates a SignedBuilderBidDeneb Proto from ExecHeaderResponseDeneb.
func (ehr *ExecHeaderResponseDeneb) ToProto() (*eth.SignedBuilderBidDeneb, error) {
bb, err := ehr.Data.Message.ToProto()
if err != nil {
return nil, err
}
return &eth.SignedBuilderBidDeneb{
Message: bb,
Signature: bytesutil.SafeCopyBytes(ehr.Data.Signature),
}, nil
}
// BlindedBeaconBlockBodyCapella is a field in BlindedBeaconBlockCapella.
type BlindedBeaconBlockBodyCapella struct {
*eth.BlindedBeaconBlockBodyCapella
// ToProto creates a BuilderBidDeneb Proto from BuilderBidDeneb.
func (bb *BuilderBidDeneb) ToProto() (*eth.BuilderBidDeneb, error) {
header, err := bb.Header.ToProto()
if err != nil {
return nil, err
}
if len(bb.BlobKzgCommitments) > fieldparams.MaxBlobsPerBlock {
return nil, fmt.Errorf("too many blob commitments: %d", len(bb.BlobKzgCommitments))
}
kzgCommitments := make([][]byte, len(bb.BlobKzgCommitments))
for i, commit := range bb.BlobKzgCommitments {
if len(commit) != fieldparams.BLSPubkeyLength {
return nil, fmt.Errorf("commitment length %d is not %d", len(commit), fieldparams.BLSPubkeyLength)
}
kzgCommitments[i] = bytesutil.SafeCopyBytes(commit)
}
return &eth.BuilderBidDeneb{
Header: header,
BlobKzgCommitments: kzgCommitments,
// Note that SSZBytes() reverses byte order for the little-endian representation.
// Uint256.Bytes() is big-endian, SSZBytes takes this value and reverses it.
Value: bytesutil.SafeCopyBytes(bb.Value.SSZBytes()),
Pubkey: bytesutil.SafeCopyBytes(bb.Pubkey),
}, nil
}
// MarshalJSON returns a JSON byte array representation of SignedBlindedBeaconBlockCapella.
func (b *SignedBlindedBeaconBlockCapella) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Message *BlindedBeaconBlockCapella `json:"message"`
Signature hexutil.Bytes `json:"signature"`
}{
Message: &BlindedBeaconBlockCapella{b.Block},
Signature: b.Signature,
// BuilderBidDeneb is a field of ExecHeaderResponseDeneb.
type BuilderBidDeneb struct {
Header *ExecutionPayloadHeaderDeneb `json:"header"`
BlobKzgCommitments []hexutil.Bytes `json:"blob_kzg_commitments"`
Value Uint256 `json:"value"`
Pubkey hexutil.Bytes `json:"pubkey"`
}
// ExecutionPayloadHeaderDeneb a field part of the BuilderBidDeneb.
type ExecutionPayloadHeaderDeneb struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
TransactionsRoot hexutil.Bytes `json:"transactions_root"`
WithdrawalsRoot hexutil.Bytes `json:"withdrawals_root"`
BlobGasUsed Uint64String `json:"blob_gas_used"` // new in deneb
ExcessBlobGas Uint64String `json:"excess_blob_gas"` // new in deneb
*v1.ExecutionPayloadHeaderDeneb
}
// MarshalJSON returns a JSON byte array representing the ExecutionPayloadHeaderDeneb struct.
func (h *ExecutionPayloadHeaderDeneb) MarshalJSON() ([]byte, error) {
type MarshalCaller ExecutionPayloadHeaderDeneb
baseFeePerGas, err := sszBytesToUint256(h.ExecutionPayloadHeaderDeneb.BaseFeePerGas)
if err != nil {
return []byte{}, errors.Wrapf(err, "invalid BaseFeePerGas")
}
return json.Marshal(&MarshalCaller{
ParentHash: h.ExecutionPayloadHeaderDeneb.ParentHash,
FeeRecipient: h.ExecutionPayloadHeaderDeneb.FeeRecipient,
StateRoot: h.ExecutionPayloadHeaderDeneb.StateRoot,
ReceiptsRoot: h.ExecutionPayloadHeaderDeneb.ReceiptsRoot,
LogsBloom: h.ExecutionPayloadHeaderDeneb.LogsBloom,
PrevRandao: h.ExecutionPayloadHeaderDeneb.PrevRandao,
BlockNumber: Uint64String(h.ExecutionPayloadHeaderDeneb.BlockNumber),
GasLimit: Uint64String(h.ExecutionPayloadHeaderDeneb.GasLimit),
GasUsed: Uint64String(h.ExecutionPayloadHeaderDeneb.GasUsed),
Timestamp: Uint64String(h.ExecutionPayloadHeaderDeneb.Timestamp),
ExtraData: h.ExecutionPayloadHeaderDeneb.ExtraData,
BaseFeePerGas: baseFeePerGas,
BlockHash: h.ExecutionPayloadHeaderDeneb.BlockHash,
TransactionsRoot: h.ExecutionPayloadHeaderDeneb.TransactionsRoot,
WithdrawalsRoot: h.ExecutionPayloadHeaderDeneb.WithdrawalsRoot,
BlobGasUsed: Uint64String(h.ExecutionPayloadHeaderDeneb.BlobGasUsed),
ExcessBlobGas: Uint64String(h.ExecutionPayloadHeaderDeneb.ExcessBlobGas),
})
}
// MarshalJSON returns a JSON byte array representation of BlindedBeaconBlockCapella
func (b *BlindedBeaconBlockCapella) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot hexutil.Bytes `json:"parent_root"`
StateRoot hexutil.Bytes `json:"state_root"`
Body *BlindedBeaconBlockBodyCapella `json:"body"`
}{
Slot: fmt.Sprintf("%d", b.Slot),
ProposerIndex: fmt.Sprintf("%d", b.ProposerIndex),
ParentRoot: b.ParentRoot,
StateRoot: b.StateRoot,
Body: &BlindedBeaconBlockBodyCapella{b.Body},
})
// UnmarshalJSON takes in a byte array and unmarshals the value into ExecutionPayloadHeaderDeneb.
func (h *ExecutionPayloadHeaderDeneb) UnmarshalJSON(b []byte) error {
type UnmarshalCaller ExecutionPayloadHeaderDeneb
uc := &UnmarshalCaller{}
if err := json.Unmarshal(b, uc); err != nil {
return err
}
ep := ExecutionPayloadHeaderDeneb(*uc)
*h = ep
var err error
h.ExecutionPayloadHeaderDeneb, err = h.ToProto()
return err
}
// MarshalJSON returns a JSON byte array representation of BlindedBeaconBlockBodyCapella
func (b *BlindedBeaconBlockBodyCapella) MarshalJSON() ([]byte, error) {
sve := make([]*SignedVoluntaryExit, len(b.VoluntaryExits))
for i := range b.VoluntaryExits {
sve[i] = &SignedVoluntaryExit{SignedVoluntaryExit: b.VoluntaryExits[i]}
// ToProto returns a ExecutionPayloadHeaderDeneb Proto object.
func (h *ExecutionPayloadHeaderDeneb) ToProto() (*v1.ExecutionPayloadHeaderDeneb, error) {
return &v1.ExecutionPayloadHeaderDeneb{
ParentHash: bytesutil.SafeCopyBytes(h.ParentHash),
FeeRecipient: bytesutil.SafeCopyBytes(h.FeeRecipient),
StateRoot: bytesutil.SafeCopyBytes(h.StateRoot),
ReceiptsRoot: bytesutil.SafeCopyBytes(h.ReceiptsRoot),
LogsBloom: bytesutil.SafeCopyBytes(h.LogsBloom),
PrevRandao: bytesutil.SafeCopyBytes(h.PrevRandao),
BlockNumber: uint64(h.BlockNumber),
GasLimit: uint64(h.GasLimit),
GasUsed: uint64(h.GasUsed),
Timestamp: uint64(h.Timestamp),
ExtraData: bytesutil.SafeCopyBytes(h.ExtraData),
BaseFeePerGas: bytesutil.SafeCopyBytes(h.BaseFeePerGas.SSZBytes()),
BlockHash: bytesutil.SafeCopyBytes(h.BlockHash),
TransactionsRoot: bytesutil.SafeCopyBytes(h.TransactionsRoot),
WithdrawalsRoot: bytesutil.SafeCopyBytes(h.WithdrawalsRoot),
BlobGasUsed: uint64(h.BlobGasUsed),
ExcessBlobGas: uint64(h.ExcessBlobGas),
}, nil
}
// ExecPayloadResponseDeneb the response to the build API /eth/v1/builder/blinded_blocks that includes the version, execution payload object , and blobs bundle object.
type ExecPayloadResponseDeneb struct {
Version string `json:"version"`
Data *ExecutionPayloadDenebAndBlobsBundle `json:"data"`
}
// ExecutionPayloadDenebAndBlobsBundle the main field used in ExecPayloadResponseDeneb.
type ExecutionPayloadDenebAndBlobsBundle struct {
ExecutionPayload *ExecutionPayloadDeneb `json:"execution_payload"`
BlobsBundle *BlobsBundle `json:"blobs_bundle"`
}
// ExecutionPayloadDeneb is a field used in ExecutionPayloadDenebAndBlobsBundle.
type ExecutionPayloadDeneb struct {
ParentHash hexutil.Bytes `json:"parent_hash"`
FeeRecipient hexutil.Bytes `json:"fee_recipient"`
StateRoot hexutil.Bytes `json:"state_root"`
ReceiptsRoot hexutil.Bytes `json:"receipts_root"`
LogsBloom hexutil.Bytes `json:"logs_bloom"`
PrevRandao hexutil.Bytes `json:"prev_randao"`
BlockNumber Uint64String `json:"block_number"`
GasLimit Uint64String `json:"gas_limit"`
GasUsed Uint64String `json:"gas_used"`
Timestamp Uint64String `json:"timestamp"`
ExtraData hexutil.Bytes `json:"extra_data"`
BaseFeePerGas Uint256 `json:"base_fee_per_gas"`
BlockHash hexutil.Bytes `json:"block_hash"`
Transactions []hexutil.Bytes `json:"transactions"`
Withdrawals []Withdrawal `json:"withdrawals"`
BlobGasUsed Uint64String `json:"blob_gas_used"` // new in deneb
ExcessBlobGas Uint64String `json:"excess_blob_gas"` // new in deneb
}
// BlobsBundle is a field in ExecutionPayloadDenebAndBlobsBundle.
type BlobsBundle struct {
Commitments []hexutil.Bytes `json:"commitments"`
Proofs []hexutil.Bytes `json:"proofs"`
Blobs []hexutil.Bytes `json:"blobs"`
}
// ToProto returns a BlobsBundle Proto.
func (b BlobsBundle) ToProto() (*v1.BlobsBundle, error) {
if len(b.Blobs) > fieldparams.MaxBlobCommitmentsPerBlock {
return nil, fmt.Errorf("blobs length %d is more than max %d", len(b.Blobs), fieldparams.MaxBlobCommitmentsPerBlock)
}
deps := make([]*Deposit, len(b.Deposits))
for i := range b.Deposits {
deps[i] = &Deposit{Deposit: b.Deposits[i]}
if len(b.Commitments) != len(b.Blobs) {
return nil, fmt.Errorf("commitments length %d does not equal blobs length %d", len(b.Commitments), len(b.Blobs))
}
atts := make([]*Attestation, len(b.Attestations))
for i := range b.Attestations {
atts[i] = &Attestation{Attestation: b.Attestations[i]}
if len(b.Proofs) != len(b.Blobs) {
return nil, fmt.Errorf("proofs length %d does not equal blobs length %d", len(b.Proofs), len(b.Blobs))
}
atsl := make([]*AttesterSlashing, len(b.AttesterSlashings))
for i := range b.AttesterSlashings {
atsl[i] = &AttesterSlashing{AttesterSlashing: b.AttesterSlashings[i]}
commitments := make([][]byte, len(b.Commitments))
for i := range b.Commitments {
if len(b.Commitments[i]) != fieldparams.BLSPubkeyLength {
return nil, fmt.Errorf("commitment length %d is not %d", len(b.Commitments[i]), fieldparams.BLSPubkeyLength)
}
commitments[i] = bytesutil.SafeCopyBytes(b.Commitments[i])
}
pros := make([]*ProposerSlashing, len(b.ProposerSlashings))
for i := range b.ProposerSlashings {
pros[i] = &ProposerSlashing{ProposerSlashing: b.ProposerSlashings[i]}
proofs := make([][]byte, len(b.Proofs))
for i := range b.Proofs {
if len(b.Proofs[i]) != fieldparams.BLSPubkeyLength {
return nil, fmt.Errorf("proof length %d is not %d", len(b.Proofs[i]), fieldparams.BLSPubkeyLength)
}
proofs[i] = bytesutil.SafeCopyBytes(b.Proofs[i])
}
chs := make([]*SignedBLSToExecutionChange, len(b.BlsToExecutionChanges))
for i := range b.BlsToExecutionChanges {
chs[i] = &SignedBLSToExecutionChange{SignedBLSToExecutionChange: b.BlsToExecutionChanges[i]}
blobs := make([][]byte, len(b.Blobs))
for i := range b.Blobs {
if len(b.Blobs[i]) != fieldparams.BlobLength {
return nil, fmt.Errorf("blob length %d is not %d", len(b.Blobs[i]), fieldparams.BlobLength)
}
blobs[i] = bytesutil.SafeCopyBytes(b.Blobs[i])
}
return json.Marshal(struct {
RandaoReveal hexutil.Bytes `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti hexutil.Bytes `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeaderCapella `json:"execution_payload_header"`
}{
RandaoReveal: b.RandaoReveal,
Eth1Data: &Eth1Data{b.Eth1Data},
Graffiti: b.Graffiti,
ProposerSlashings: pros,
AttesterSlashings: atsl,
Attestations: atts,
Deposits: deps,
VoluntaryExits: sve,
BLSToExecutionChanges: chs,
SyncAggregate: &SyncAggregate{b.SyncAggregate},
ExecutionPayloadHeader: &ExecutionPayloadHeaderCapella{ExecutionPayloadHeaderCapella: b.ExecutionPayloadHeader},
})
return &v1.BlobsBundle{
KzgCommitments: commitments,
Proofs: proofs,
Blobs: blobs,
}, nil
}
// FromBundleProto converts the proto bundle type to the builder
// type.
func FromBundleProto(bundle *v1.BlobsBundle) *BlobsBundle {
commitments := make([]hexutil.Bytes, len(bundle.KzgCommitments))
for i := range bundle.KzgCommitments {
commitments[i] = bytesutil.SafeCopyBytes(bundle.KzgCommitments[i])
}
proofs := make([]hexutil.Bytes, len(bundle.Proofs))
for i := range bundle.Proofs {
proofs[i] = bytesutil.SafeCopyBytes(bundle.Proofs[i])
}
blobs := make([]hexutil.Bytes, len(bundle.Blobs))
for i := range bundle.Blobs {
blobs[i] = bytesutil.SafeCopyBytes(bundle.Blobs[i])
}
return &BlobsBundle{
Commitments: commitments,
Proofs: proofs,
Blobs: blobs,
}
}
// ToProto returns ExecutionPayloadDeneb Proto and BlobsBundle Proto separately.
func (r *ExecPayloadResponseDeneb) ToProto() (*v1.ExecutionPayloadDeneb, *v1.BlobsBundle, error) {
if r.Data == nil {
return nil, nil, errors.New("data field in response is empty")
}
if r.Data.ExecutionPayload == nil {
return nil, nil, errors.Wrap(consensusblocks.ErrNilObject, "nil execution payload")
}
if r.Data.BlobsBundle == nil {
return nil, nil, errors.Wrap(consensusblocks.ErrNilObject, "nil blobs bundle")
}
payload, err := r.Data.ExecutionPayload.ToProto()
if err != nil {
return nil, nil, err
}
bundle, err := r.Data.BlobsBundle.ToProto()
if err != nil {
return nil, nil, err
}
return payload, bundle, nil
}
func (r *ExecutionPayloadDenebAndBlobsBundle) PayloadProto() (proto.Message, error) {
if r.ExecutionPayload == nil {
return nil, errors.Wrap(consensusblocks.ErrNilObject, "nil execution payload in combined deneb payload")
}
pb, err := r.ExecutionPayload.ToProto()
return pb, err
}
func (r *ExecutionPayloadDenebAndBlobsBundle) BundleProto() (*v1.BlobsBundle, error) {
if r.BlobsBundle == nil {
return nil, errors.Wrap(consensusblocks.ErrNilObject, "nil blobs bundle")
}
return r.BlobsBundle.ToProto()
}
// ToProto returns the ExecutionPayloadDeneb Proto.
func (p *ExecutionPayloadDeneb) ToProto() (*v1.ExecutionPayloadDeneb, error) {
if p == nil {
return nil, errors.Wrap(consensusblocks.ErrNilObject, "nil execution payload")
}
txs := make([][]byte, len(p.Transactions))
for i := range p.Transactions {
txs[i] = bytesutil.SafeCopyBytes(p.Transactions[i])
}
withdrawals := make([]*v1.Withdrawal, len(p.Withdrawals))
for i, w := range p.Withdrawals {
withdrawals[i] = &v1.Withdrawal{
Index: w.Index.Uint64(),
ValidatorIndex: types.ValidatorIndex(w.ValidatorIndex.Uint64()),
Address: bytesutil.SafeCopyBytes(w.Address),
Amount: w.Amount.Uint64(),
}
}
return &v1.ExecutionPayloadDeneb{
ParentHash: bytesutil.SafeCopyBytes(p.ParentHash),
FeeRecipient: bytesutil.SafeCopyBytes(p.FeeRecipient),
StateRoot: bytesutil.SafeCopyBytes(p.StateRoot),
ReceiptsRoot: bytesutil.SafeCopyBytes(p.ReceiptsRoot),
LogsBloom: bytesutil.SafeCopyBytes(p.LogsBloom),
PrevRandao: bytesutil.SafeCopyBytes(p.PrevRandao),
BlockNumber: uint64(p.BlockNumber),
GasLimit: uint64(p.GasLimit),
GasUsed: uint64(p.GasUsed),
Timestamp: uint64(p.Timestamp),
ExtraData: bytesutil.SafeCopyBytes(p.ExtraData),
BaseFeePerGas: bytesutil.SafeCopyBytes(p.BaseFeePerGas.SSZBytes()),
BlockHash: bytesutil.SafeCopyBytes(p.BlockHash),
Transactions: txs,
Withdrawals: withdrawals,
BlobGasUsed: uint64(p.BlobGasUsed),
ExcessBlobGas: uint64(p.ExcessBlobGas),
}, nil
}
// ErrorMessage is a JSON representation of the builder API's returned error message.

View File

@@ -12,12 +12,17 @@ import (
"testing"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/golang/protobuf/proto"
"github.com/pkg/errors"
"github.com/prysmaticlabs/go-bitfield"
v1 "github.com/prysmaticlabs/prysm/v4/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/api/server/structs"
fieldparams "github.com/prysmaticlabs/prysm/v5/config/fieldparams"
consensusblocks "github.com/prysmaticlabs/prysm/v5/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v5/math"
v1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
eth "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/runtime/version"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func ezDecode(t *testing.T, s string) []byte {
@@ -36,7 +41,7 @@ func TestSignedValidatorRegistration_MarshalJSON(t *testing.T) {
},
Signature: make([]byte, 96),
}
a := &SignedValidatorRegistration{SignedValidatorRegistrationV1: svr}
a := structs.SignedValidatorRegistrationFromConsensus(svr)
je, err := json.Marshal(a)
require.NoError(t, err)
// decode with a struct w/ plain strings so we can check the string encoding of the hex fields
@@ -53,11 +58,11 @@ func TestSignedValidatorRegistration_MarshalJSON(t *testing.T) {
require.Equal(t, "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", un.Message.Pubkey)
t.Run("roundtrip", func(t *testing.T) {
b := &SignedValidatorRegistration{}
b := &structs.SignedValidatorRegistration{}
if err := json.Unmarshal(je, b); err != nil {
require.NoError(t, err)
}
require.Equal(t, proto.Equal(a.SignedValidatorRegistrationV1, b.SignedValidatorRegistrationV1), true)
require.DeepEqual(t, a, b)
})
}
@@ -116,6 +121,69 @@ var testExampleHeaderResponseCapella = `{
}
}`
var testExampleHeaderResponseDeneb = `{
"version": "deneb",
"data": {
"message": {
"header": {
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"withdrawals_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"blob_gas_used": "1",
"excess_blob_gas": "2"
},
"blob_kzg_commitments": [
"0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f"
],
"value": "652312848583266388373324160190187140051835877600158453279131187530910662656",
"pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
},
"signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
}
}`
var testExampleHeaderResponseDenebNoBundle = `{
"version": "deneb",
"data": {
"message": {
"header": {
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"withdrawals_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"blob_gas_used": "1",
"excess_blob_gas": "2"
},
"value": "652312848583266388373324160190187140051835877600158453279131187530910662656",
"pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
},
"signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
}
}`
var testExampleHeaderResponseUnknownVersion = `{
"version": "bad",
"data": {
@@ -144,6 +212,39 @@ var testExampleHeaderResponseUnknownVersion = `{
}
}`
var testExampleHeaderResponseDenebTooManyBlobs = `{
"version": "deneb",
"data": {
"message": {
"header": {
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"withdrawals_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"blob_gas_used": "1",
"excess_blob_gas": "2"
},
"blob_kzg_commitments": [
"","","","","","",""
],
"value": "652312848583266388373324160190187140051835877600158453279131187530910662656",
"pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
},
"signature": "0x1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505cc411d61252fb6cb3fa0017b679f8bb2305b26a285fa2737f175668d0dff91cc1b66ac1fb663c9bc59509846d6ec05345bd908eda73e670af888da41af171505"
}
}`
func TestExecutionHeaderResponseUnmarshal(t *testing.T) {
hr := &ExecHeaderResponse{}
require.NoError(t, json.Unmarshal([]byte(testExampleHeaderResponse), hr))
@@ -518,6 +619,196 @@ var testExampleExecutionPayloadCapella = `{
}
}`
var testExampleExecutionPayloadDeneb = fmt.Sprintf(`{
"version": "deneb",
"data": {
"execution_payload":{
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions": [
"0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
],
"withdrawals": [
{
"index": "1",
"validator_index": "1",
"address": "0xcf8e0d4e9587369b2301d0790347320302cc0943",
"amount": "1"
}
],
"blob_gas_used": "2",
"excess_blob_gas": "3"
},
"blobs_bundle": {
"commitments": [
"0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f"
],
"proofs": [
"0xb4021b0de10f743893d4f71e1bf830c019e832958efd6795baf2f83b8699a9eccc5dc99015d8d4d8ec370d0cc333c06a"
],
"blobs": [
"%s"
]
}
}
}`, hexutil.Encode(make([]byte, fieldparams.BlobLength)))
var testExampleExecutionPayloadDenebTooManyBlobs = fmt.Sprintf(`{
"version": "deneb",
"data": {
"execution_payload":{
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions": [
"0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
],
"withdrawals": [
{
"index": "1",
"validator_index": "1",
"address": "0xcf8e0d4e9587369b2301d0790347320302cc0943",
"amount": "1"
}
],
"blob_gas_used": "2",
"excess_blob_gas": "3"
},
"blobs_bundle": {
"commitments": [
"0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f"
],
"proofs": [
"0xb4021b0de10f743893d4f71e1bf830c019e832958efd6795baf2f83b8699a9eccc5dc99015d8d4d8ec370d0cc333c06a"
],
"blobs": %s
}
}
}`, beyondMaxEmptyBlobs())
func beyondMaxEmptyBlobs() string {
moreThanMax := fieldparams.MaxBlobCommitmentsPerBlock + 2
blobs := make([]string, moreThanMax)
b, err := json.Marshal(blobs)
if err != nil {
panic(err)
}
return string(b)
}
var testExampleExecutionPayloadDenebDifferentCommitmentCount = fmt.Sprintf(`{
"version": "deneb",
"data": {
"execution_payload":{
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions": [
"0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
],
"withdrawals": [
{
"index": "1",
"validator_index": "1",
"address": "0xcf8e0d4e9587369b2301d0790347320302cc0943",
"amount": "1"
}
],
"blob_gas_used": "2",
"excess_blob_gas": "3"
},
"blobs_bundle": {
"commitments": [
"0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f",
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
],
"proofs": [
"0xb4021b0de10f743893d4f71e1bf830c019e832958efd6795baf2f83b8699a9eccc5dc99015d8d4d8ec370d0cc333c06a"
],
"blobs": [
"%s"
]
}
}
}`, hexutil.Encode(make([]byte, fieldparams.BlobLength)))
var testExampleExecutionPayloadDenebDifferentProofCount = fmt.Sprintf(`{
"version": "deneb",
"data": {
"execution_payload":{
"parent_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"fee_recipient": "0xabcf8e0d4e9587369b2301d0790347320302cc09",
"state_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"receipts_root": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"prev_randao": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"block_number": "1",
"gas_limit": "1",
"gas_used": "1",
"timestamp": "1",
"extra_data": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"base_fee_per_gas": "452312848583266388373324160190187140051835877600158453279131187530910662656",
"block_hash": "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
"transactions": [
"0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
],
"withdrawals": [
{
"index": "1",
"validator_index": "1",
"address": "0xcf8e0d4e9587369b2301d0790347320302cc0943",
"amount": "1"
}
],
"blob_gas_used": "2",
"excess_blob_gas": "3"
},
"blobs_bundle": {
"commitments": [
"0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f"
],
"proofs": [
"0xb4021b0de10f743893d4f71e1bf830c019e832958efd6795baf2f83b8699a9eccc5dc99015d8d4d8ec370d0cc333c06a",
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
],
"blobs": [
"%s"
]
}
}
}`, hexutil.Encode(make([]byte, fieldparams.BlobLength)))
func TestExecutionPayloadResponseUnmarshal(t *testing.T) {
epr := &ExecPayloadResponse{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayload), epr))
@@ -689,6 +980,107 @@ func TestExecutionPayloadResponseCapellaUnmarshal(t *testing.T) {
assert.Equal(t, uint64(1), w.Amount.Uint64())
}
func TestExecutionPayloadResponseDenebUnmarshal(t *testing.T) {
epr := &ExecPayloadResponseDeneb{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadDeneb), epr))
cases := []struct {
expected string
actual string
name string
}{
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.ParentHash),
name: "ExecPayloadResponse.ExecutionPayload.ParentHash",
},
{
expected: "0xabcf8e0d4e9587369b2301d0790347320302cc09",
actual: hexutil.Encode(epr.Data.ExecutionPayload.FeeRecipient),
name: "ExecPayloadResponse.ExecutionPayload.FeeRecipient",
},
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.StateRoot),
name: "ExecPayloadResponse.ExecutionPayload.StateRoot",
},
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.ReceiptsRoot),
name: "ExecPayloadResponse.ExecutionPayload.ReceiptsRoot",
},
{
expected: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
actual: hexutil.Encode(epr.Data.ExecutionPayload.LogsBloom),
name: "ExecPayloadResponse.ExecutionPayload.LogsBloom",
},
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.PrevRandao),
name: "ExecPayloadResponse.ExecutionPayload.PrevRandao",
},
{
expected: "1",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.BlockNumber),
name: "ExecPayloadResponse.ExecutionPayload.BlockNumber",
},
{
expected: "1",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.GasLimit),
name: "ExecPayloadResponse.ExecutionPayload.GasLimit",
},
{
expected: "1",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.GasUsed),
name: "ExecPayloadResponse.ExecutionPayload.GasUsed",
},
{
expected: "1",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.Timestamp),
name: "ExecPayloadResponse.ExecutionPayload.Timestamp",
},
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.ExtraData),
name: "ExecPayloadResponse.ExecutionPayload.ExtraData",
},
{
expected: "452312848583266388373324160190187140051835877600158453279131187530910662656",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.BaseFeePerGas),
name: "ExecPayloadResponse.ExecutionPayload.BaseFeePerGas",
},
{
expected: "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2",
actual: hexutil.Encode(epr.Data.ExecutionPayload.BlockHash),
name: "ExecPayloadResponse.ExecutionPayload.BlockHash",
},
{
expected: "2",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.BlobGasUsed),
name: "ExecPayloadResponse.ExecutionPayload.BlobGasUsed",
},
{
expected: "3",
actual: fmt.Sprintf("%d", epr.Data.ExecutionPayload.ExcessBlobGas),
name: "ExecPayloadResponse.ExecutionPayload.ExcessBlobGas",
},
}
for _, c := range cases {
require.Equal(t, c.expected, c.actual, fmt.Sprintf("unexpected value for field %s", c.name))
}
require.Equal(t, 1, len(epr.Data.ExecutionPayload.Transactions))
txHash := "0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86"
require.Equal(t, txHash, hexutil.Encode(epr.Data.ExecutionPayload.Transactions[0]))
require.Equal(t, 1, len(epr.Data.ExecutionPayload.Withdrawals))
w := epr.Data.ExecutionPayload.Withdrawals[0]
assert.Equal(t, uint64(1), w.Index.Uint64())
assert.Equal(t, uint64(1), w.ValidatorIndex.Uint64())
assert.DeepEqual(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943", w.Address.String())
assert.Equal(t, uint64(1), w.Amount.Uint64())
assert.Equal(t, uint64(2), uint64(epr.Data.ExecutionPayload.BlobGasUsed))
assert.Equal(t, uint64(3), uint64(epr.Data.ExecutionPayload.ExcessBlobGas))
}
func TestExecutionPayloadResponseToProto(t *testing.T) {
hr := &ExecPayloadResponse{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayload), hr))
@@ -793,7 +1185,105 @@ func TestExecutionPayloadResponseCapellaToProto(t *testing.T) {
},
}
require.DeepEqual(t, expected, p)
}
func TestExecutionPayloadResponseDenebToProto(t *testing.T) {
hr := &ExecPayloadResponseDeneb{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadDeneb), hr))
p, blobsBundle, err := hr.ToProto()
require.NoError(t, err)
parentHash, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
feeRecipient, err := hexutil.Decode("0xabcf8e0d4e9587369b2301d0790347320302cc09")
require.NoError(t, err)
stateRoot, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
receiptsRoot, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
logsBloom, err := hexutil.Decode("0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000")
require.NoError(t, err)
prevRandao, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
extraData, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
blockHash, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2")
require.NoError(t, err)
tx, err := hexutil.Decode("0x02f878831469668303f51d843b9ac9f9843b9aca0082520894c93269b73096998db66be0441e836d873535cb9c8894a19041886f000080c001a031cc29234036afbf9a1fb9476b463367cb1f957ac0b919b69bbc798436e604aaa018c4e9c3914eb27aadd0b91e10b18655739fcf8c1fc398763a9f1beecb8ddc86")
require.NoError(t, err)
txList := [][]byte{tx}
address, err := hexutil.Decode("0xcf8e0d4e9587369b2301d0790347320302cc0943")
require.NoError(t, err)
bfpg, err := stringToUint256("452312848583266388373324160190187140051835877600158453279131187530910662656")
require.NoError(t, err)
expected := &v1.ExecutionPayloadDeneb{
ParentHash: parentHash,
FeeRecipient: feeRecipient,
StateRoot: stateRoot,
ReceiptsRoot: receiptsRoot,
LogsBloom: logsBloom,
PrevRandao: prevRandao,
BlockNumber: 1,
GasLimit: 1,
GasUsed: 1,
Timestamp: 1,
ExtraData: extraData,
BaseFeePerGas: bfpg.SSZBytes(),
BlockHash: blockHash,
Transactions: txList,
Withdrawals: []*v1.Withdrawal{
{
Index: 1,
ValidatorIndex: 1,
Address: address,
Amount: 1,
},
},
BlobGasUsed: 2,
ExcessBlobGas: 3,
}
require.DeepEqual(t, expected, p)
commitment, err := hexutil.Decode("0x8dab030c51e16e84be9caab84ee3d0b8bbec1db4a0e4de76439da8424d9b957370a10a78851f97e4b54d2ce1ab0d686f")
require.NoError(t, err)
proof, err := hexutil.Decode("0xb4021b0de10f743893d4f71e1bf830c019e832958efd6795baf2f83b8699a9eccc5dc99015d8d4d8ec370d0cc333c06a")
require.NoError(t, err)
expectedBlobs := &v1.BlobsBundle{
KzgCommitments: [][]byte{
commitment,
},
Proofs: [][]byte{
proof,
},
Blobs: [][]byte{
make([]byte, fieldparams.BlobLength),
},
}
require.DeepEqual(t, blobsBundle, expectedBlobs)
}
func TestExecutionPayloadResponseDenebToProtoInvalidBlobCount(t *testing.T) {
hr := &ExecPayloadResponseDeneb{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadDenebTooManyBlobs), hr))
_, _, err := hr.ToProto()
require.ErrorContains(t, fmt.Sprintf("blobs length %d is more than max %d", fieldparams.MaxBlobCommitmentsPerBlock+2, fieldparams.MaxBlobCommitmentsPerBlock), err)
}
func TestExecutionPayloadResponseDenebToProtoDifferentCommitmentCount(t *testing.T) {
hr := &ExecPayloadResponseDeneb{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadDenebDifferentCommitmentCount), hr))
_, _, err := hr.ToProto()
require.ErrorContains(t, "commitments length 2 does not equal blobs length 1", err)
}
func TestExecutionPayloadResponseDenebToProtoDifferentProofCount(t *testing.T) {
hr := &ExecPayloadResponseDeneb{}
require.NoError(t, json.Unmarshal([]byte(testExampleExecutionPayloadDenebDifferentProofCount), hr))
_, _, err := hr.ToProto()
require.ErrorContains(t, "proofs length 2 does not equal blobs length 1", err)
}
func pbEth1Data() *eth.Eth1Data {
@@ -1026,6 +1516,30 @@ func pbExecutionPayloadHeaderCapella(t *testing.T) *v1.ExecutionPayloadHeaderCap
}
}
func pbExecutionPayloadHeaderDeneb(t *testing.T) *v1.ExecutionPayloadHeaderDeneb {
bfpg, err := stringToUint256("452312848583266388373324160190187140051835877600158453279131187530910662656")
require.NoError(t, err)
return &v1.ExecutionPayloadHeaderDeneb{
ParentHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
FeeRecipient: ezDecode(t, "0xabcf8e0d4e9587369b2301d0790347320302cc09"),
StateRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
ReceiptsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
LogsBloom: ezDecode(t, "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"),
PrevRandao: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlockNumber: 1,
GasLimit: 1,
GasUsed: 1,
Timestamp: 1,
ExtraData: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BaseFeePerGas: bfpg.SSZBytes(),
BlockHash: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
TransactionsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
WithdrawalsRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
BlobGasUsed: 1,
ExcessBlobGas: 2,
}
}
func TestExecutionPayloadHeader_MarshalJSON(t *testing.T) {
h := &ExecutionPayloadHeader{
ExecutionPayloadHeader: pbExecutionPayloadHeader(t),
@@ -1046,6 +1560,16 @@ func TestExecutionPayloadHeaderCapella_MarshalJSON(t *testing.T) {
require.Equal(t, expected, string(b))
}
func TestExecutionPayloadHeaderDeneb_MarshalJSON(t *testing.T) {
h := &ExecutionPayloadHeaderDeneb{
ExecutionPayloadHeaderDeneb: pbExecutionPayloadHeaderDeneb(t),
}
b, err := json.Marshal(h)
require.NoError(t, err)
expected := `{"parent_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","fee_recipient":"0xabcf8e0d4e9587369b2301d0790347320302cc09","state_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","receipts_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","logs_bloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prev_randao":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","block_number":"1","gas_limit":"1","gas_used":"1","timestamp":"1","extra_data":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","base_fee_per_gas":"452312848583266388373324160190187140051835877600158453279131187530910662656","block_hash":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","transactions_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","withdrawals_root":"0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2","blob_gas_used":"1","excess_blob_gas":"2"}`
require.Equal(t, expected, string(b))
}
var testBuilderBid = `{
"version":"bellatrix",
"data":{
@@ -1079,7 +1603,6 @@ func TestBuilderBidUnmarshalUint256(t *testing.T) {
require.NoError(t, expectedValue.UnmarshalText([]byte(base10)))
r := &ExecHeaderResponse{}
require.NoError(t, json.Unmarshal([]byte(testBuilderBid), r))
//require.Equal(t, expectedValue, r.Data.Message.Value)
marshaled := r.Data.Message.Value.String()
require.Equal(t, base10, marshaled)
require.Equal(t, 0, expectedValue.Cmp(r.Data.Message.Value.Int))
@@ -1104,40 +1627,40 @@ func TestIsValidUint256(t *testing.T) {
_, ok = value.SetString("-10000000000000000000000000000000000000000000000000000000000000000", 16)
require.Equal(t, true, ok)
require.Equal(t, 257, value.BitLen())
require.Equal(t, false, isValidUint256(value))
require.Equal(t, false, math.IsValidUint256(value))
// negative uint256.max
_, ok = value.SetString("-ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", 16)
require.Equal(t, true, ok)
require.Equal(t, 256, value.BitLen())
require.Equal(t, false, isValidUint256(value))
require.Equal(t, false, math.IsValidUint256(value))
// negative number
_, ok = value.SetString("-1", 16)
require.Equal(t, true, ok)
require.Equal(t, false, isValidUint256(value))
require.Equal(t, false, math.IsValidUint256(value))
// uint256.min
_, ok = value.SetString("0", 16)
require.Equal(t, true, ok)
require.Equal(t, true, isValidUint256(value))
require.Equal(t, true, math.IsValidUint256(value))
// positive number
_, ok = value.SetString("1", 16)
require.Equal(t, true, ok)
require.Equal(t, true, isValidUint256(value))
require.Equal(t, true, math.IsValidUint256(value))
// uint256.max
_, ok = value.SetString("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", 16)
require.Equal(t, true, ok)
require.Equal(t, 256, value.BitLen())
require.Equal(t, true, isValidUint256(value))
require.Equal(t, true, math.IsValidUint256(value))
// uint256.max + 1
_, ok = value.SetString("10000000000000000000000000000000000000000000000000000000000000000", 16)
require.Equal(t, true, ok)
require.Equal(t, 257, value.BitLen())
require.Equal(t, false, isValidUint256(value))
require.Equal(t, false, math.IsValidUint256(value))
}
func TestUint256Unmarshal(t *testing.T) {
@@ -1197,7 +1720,7 @@ func TestUint256UnmarshalTooBig(t *testing.T) {
func TestMarshalBlindedBeaconBlockBodyBellatrix(t *testing.T) {
expected, err := os.ReadFile("testdata/blinded-block.json")
require.NoError(t, err)
b := &BlindedBeaconBlockBellatrix{BlindedBeaconBlockBellatrix: &eth.BlindedBeaconBlockBellatrix{
b, err := structs.BlindedBeaconBlockBellatrixFromConsensus(&eth.BlindedBeaconBlockBellatrix{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
@@ -1214,7 +1737,8 @@ func TestMarshalBlindedBeaconBlockBodyBellatrix(t *testing.T) {
SyncAggregate: pbSyncAggregate(),
ExecutionPayloadHeader: pbExecutionPayloadHeader(t),
},
}}
})
require.NoError(t, err)
m, err := json.Marshal(b)
require.NoError(t, err)
// string error output is easier to deal with
@@ -1226,7 +1750,7 @@ func TestMarshalBlindedBeaconBlockBodyBellatrix(t *testing.T) {
func TestMarshalBlindedBeaconBlockBodyCapella(t *testing.T) {
expected, err := os.ReadFile("testdata/blinded-block-capella.json")
require.NoError(t, err)
b := &BlindedBeaconBlockCapella{BlindedBeaconBlockCapella: &eth.BlindedBeaconBlockCapella{
b, err := structs.BlindedBeaconBlockCapellaFromConsensus(&eth.BlindedBeaconBlockCapella{
Slot: 1,
ProposerIndex: 1,
ParentRoot: ezDecode(t, "0xcf8e0d4e9587369b2301d0790347320302cc0943d5a1884560367e8208d920f2"),
@@ -1243,7 +1767,8 @@ func TestMarshalBlindedBeaconBlockBodyCapella(t *testing.T) {
SyncAggregate: pbSyncAggregate(),
ExecutionPayloadHeader: pbExecutionPayloadHeaderCapella(t),
},
}}
})
require.NoError(t, err)
m, err := json.Marshal(b)
require.NoError(t, err)
// string error output is easier to deal with
@@ -1384,3 +1909,41 @@ func TestErrorMessage_non200Err(t *testing.T) {
})
}
}
func TestEmptyResponseBody(t *testing.T) {
t.Run("empty buffer", func(t *testing.T) {
var b []byte
r := &ExecutionPayloadResponse{}
err := json.Unmarshal(b, r)
var syntaxError *json.SyntaxError
ok := errors.As(err, &syntaxError)
require.Equal(t, true, ok)
})
t.Run("empty object", func(t *testing.T) {
empty := []byte("{}")
emptyResponse := &ExecutionPayloadResponse{}
require.NoError(t, json.Unmarshal(empty, emptyResponse))
_, err := emptyResponse.ParsePayload()
require.ErrorIs(t, err, consensusblocks.ErrUnsupportedVersion)
})
versions := []int{version.Bellatrix, version.Capella, version.Deneb}
for i := range versions {
vstr := version.String(versions[i])
t.Run("populated version without payload"+vstr, func(t *testing.T) {
in := &ExecutionPayloadResponse{Version: vstr}
encoded, err := json.Marshal(in)
require.NoError(t, err)
epr := &ExecutionPayloadResponse{}
require.NoError(t, json.Unmarshal(encoded, epr))
pp, err := epr.ParsePayload()
require.NoError(t, err)
pb, err := pp.PayloadProto()
if err == nil {
require.NoError(t, err)
require.Equal(t, false, pb == nil)
} else {
require.ErrorIs(t, err, consensusblocks.ErrNilObject)
}
})
}
}

View File

@@ -4,7 +4,7 @@ import (
"net/url"
"testing"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestValidHostname(t *testing.T) {

View File

@@ -21,6 +21,9 @@ var ErrNotFound = errors.Wrap(ErrNotOK, "recv 404 NotFound response from API")
// ErrInvalidNodeVersion indicates that the /eth/v1/node/version API response format was not recognized.
var ErrInvalidNodeVersion = errors.New("invalid node version response")
// ErrConnectionIssue represents a connection problem.
var ErrConnectionIssue = errors.New("could not connect")
// Non200Err is a function that parses an HTTP response to handle responses that are not 200 with a formatted error.
func Non200Err(response *http.Response) error {
bodyBytes, err := io.ReadAll(response.Body)
@@ -32,7 +35,7 @@ func Non200Err(response *http.Response) error {
}
msg := fmt.Sprintf("code=%d, url=%s, body=%s", response.StatusCode, response.Request.URL, body)
switch response.StatusCode {
case 404:
case http.StatusNotFound:
return errors.Wrap(ErrNotFound, msg)
default:
return errors.Wrap(ErrNotOK, msg)

View File

@@ -0,0 +1,30 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"event_stream.go",
"utils.go",
],
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/event",
visibility = ["//visibility:public"],
deps = [
"//api:go_default_library",
"//api/client:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"event_stream_test.go",
"utils_test.go",
],
embed = [":go_default_library"],
deps = [
"//testing/require:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)

View File

@@ -0,0 +1,150 @@
package event
import (
"bufio"
"context"
"net/http"
"net/url"
"strings"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v5/api"
"github.com/prysmaticlabs/prysm/v5/api/client"
log "github.com/sirupsen/logrus"
)
const (
EventHead = "head"
EventBlock = "block"
EventAttestation = "attestation"
EventVoluntaryExit = "voluntary_exit"
EventBlsToExecutionChange = "bls_to_execution_change"
EventProposerSlashing = "proposer_slashing"
EventAttesterSlashing = "attester_slashing"
EventFinalizedCheckpoint = "finalized_checkpoint"
EventChainReorg = "chain_reorg"
EventContributionAndProof = "contribution_and_proof"
EventLightClientFinalityUpdate = "light_client_finality_update"
EventLightClientOptimisticUpdate = "light_client_optimistic_update"
EventPayloadAttributes = "payload_attributes"
EventBlobSidecar = "blob_sidecar"
EventError = "error"
EventConnectionError = "connection_error"
)
var (
_ = EventStreamClient(&EventStream{})
)
var DefaultEventTopics = []string{EventHead}
type EventStreamClient interface {
Subscribe(eventsChannel chan<- *Event)
}
type Event struct {
EventType string
Data []byte
}
// EventStream is responsible for subscribing to the Beacon API events endpoint
// and dispatching received events to subscribers.
type EventStream struct {
ctx context.Context
httpClient *http.Client
host string
topics []string
}
func NewEventStream(ctx context.Context, httpClient *http.Client, host string, topics []string) (*EventStream, error) {
// Check if the host is a valid URL
_, err := url.ParseRequestURI(host)
if err != nil {
return nil, err
}
if len(topics) == 0 {
return nil, errors.New("no topics provided")
}
return &EventStream{
ctx: ctx,
httpClient: httpClient,
host: host,
topics: topics,
}, nil
}
func (h *EventStream) Subscribe(eventsChannel chan<- *Event) {
allTopics := strings.Join(h.topics, ",")
log.WithField("topics", allTopics).Info("Listening to Beacon API events")
fullUrl := h.host + "/eth/v1/events?topics=" + allTopics
req, err := http.NewRequestWithContext(h.ctx, http.MethodGet, fullUrl, nil)
if err != nil {
eventsChannel <- &Event{
EventType: EventConnectionError,
Data: []byte(errors.Wrap(err, "failed to create HTTP request").Error()),
}
}
req.Header.Set("Accept", api.EventStreamMediaType)
req.Header.Set("Connection", api.KeepAlive)
resp, err := h.httpClient.Do(req)
if err != nil {
eventsChannel <- &Event{
EventType: EventConnectionError,
Data: []byte(errors.Wrap(err, client.ErrConnectionIssue.Error()).Error()),
}
}
defer func() {
if closeErr := resp.Body.Close(); closeErr != nil {
log.WithError(closeErr).Error("Failed to close events response body")
}
}()
// Create a new scanner to read lines from the response body
scanner := bufio.NewScanner(resp.Body)
// Set the split function for the scanning operation
scanner.Split(scanLinesWithCarriage)
var eventType, data string // Variables to store event type and data
// Iterate over lines of the event stream
for scanner.Scan() {
select {
case <-h.ctx.Done():
log.Info("Context canceled, stopping event stream")
close(eventsChannel)
return
default:
line := scanner.Text()
// Handle the event based on your specific format
if line == "" {
// Empty line indicates the end of an event
if eventType != "" && data != "" {
// Process the event when both eventType and data are set
eventsChannel <- &Event{EventType: eventType, Data: []byte(data)}
}
// Reset eventType and data for the next event
eventType, data = "", ""
continue
}
et, ok := strings.CutPrefix(line, "event: ")
if ok {
// Extract event type from the "event" field
eventType = et
}
d, ok := strings.CutPrefix(line, "data: ")
if ok {
// Extract data from the "data" field
data = d
}
}
}
if err := scanner.Err(); err != nil {
eventsChannel <- &Event{
EventType: EventConnectionError,
Data: []byte(errors.Wrap(err, errors.Wrap(client.ErrConnectionIssue, "scanner failed").Error()).Error()),
}
}
}

View File

@@ -0,0 +1,81 @@
package event
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/prysmaticlabs/prysm/v5/testing/require"
log "github.com/sirupsen/logrus"
)
func TestNewEventStream(t *testing.T) {
validURL := "http://localhost:8080"
invalidURL := "://invalid"
topics := []string{"topic1", "topic2"}
tests := []struct {
name string
host string
topics []string
wantErr bool
}{
{"Valid input", validURL, topics, false},
{"Invalid URL", invalidURL, topics, true},
{"No topics", validURL, []string{}, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := NewEventStream(context.Background(), &http.Client{}, tt.host, tt.topics)
if (err != nil) != tt.wantErr {
t.Errorf("NewEventStream() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestEventStream(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/eth/v1/events", func(w http.ResponseWriter, r *http.Request) {
flusher, ok := w.(http.Flusher)
require.Equal(t, true, ok)
for i := 1; i <= 3; i++ {
events := [3]string{"event: head\ndata: data%d\n\n", "event: head\rdata: data%d\r\r", "event: head\r\ndata: data%d\r\n\r\n"}
_, err := fmt.Fprintf(w, events[i-1], i)
require.NoError(t, err)
flusher.Flush() // Trigger flush to simulate streaming data
time.Sleep(100 * time.Millisecond) // Simulate delay between events
}
})
server := httptest.NewServer(mux)
defer server.Close()
topics := []string{"head"}
eventsChannel := make(chan *Event, 1)
stream, err := NewEventStream(context.Background(), http.DefaultClient, server.URL, topics)
require.NoError(t, err)
go stream.Subscribe(eventsChannel)
// Collect events
var events []*Event
for len(events) != 3 {
select {
case event := <-eventsChannel:
log.Info(event)
events = append(events, event)
}
}
// Assertions to verify the events content
expectedData := []string{"data1", "data2", "data3"}
for i, event := range events {
if string(event.Data) != expectedData[i] {
t.Errorf("Expected event data %q, got %q", expectedData[i], string(event.Data))
}
}
}

36
api/client/event/utils.go Normal file
View File

@@ -0,0 +1,36 @@
package event
import (
"bytes"
)
// adapted from ScanLines in scan.go to handle carriage return characters as separators
func scanLinesWithCarriage(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i, j := bytes.IndexByte(data, '\n'), bytes.IndexByte(data, '\r'); i >= 0 || j >= 0 {
in := i
// Select the first index of \n or \r or the second index of \r if it is followed by \n
if i < 0 || (i > j && i != j+1 && j >= 0) {
in = j
}
// We have a full newline-terminated line.
return in + 1, dropCR(data[0:in]), nil
}
// If we're at EOF, we have a final, non-terminated line. Return it.
if atEOF {
return len(data), dropCR(data), nil
}
// Request more data.
return 0, nil, nil
}
// dropCR drops a terminal \r from the data.
func dropCR(data []byte) []byte {
if len(data) > 0 && data[len(data)-1] == '\r' {
return data[0 : len(data)-1]
}
return data
}

View File

@@ -0,0 +1,97 @@
package event
import (
"bufio"
"bytes"
"testing"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestScanLinesWithCarriage(t *testing.T) {
testCases := []struct {
name string
input string
expected []string
}{
{
name: "LF line endings",
input: "line1\nline2\nline3",
expected: []string{"line1", "line2", "line3"},
},
{
name: "CR line endings",
input: "line1\rline2\rline3",
expected: []string{"line1", "line2", "line3"},
},
{
name: "CRLF line endings",
input: "line1\r\nline2\r\nline3",
expected: []string{"line1", "line2", "line3"},
},
{
name: "Mixed line endings",
input: "line1\nline2\rline3\r\nline4",
expected: []string{"line1", "line2", "line3", "line4"},
},
{
name: "Empty lines",
input: "line1\n\nline2\r\rline3",
expected: []string{"line1", "", "line2", "", "line3"},
},
{
name: "Empty lines 2",
input: "line1\n\rline2\n\rline3",
expected: []string{"line1", "", "line2", "", "line3"},
},
{
name: "No line endings",
input: "single line without ending",
expected: []string{"single line without ending"},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
scanner := bufio.NewScanner(bytes.NewReader([]byte(tc.input)))
scanner.Split(scanLinesWithCarriage)
var lines []string
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
require.NoError(t, scanner.Err())
require.Equal(t, len(tc.expected), len(lines), "Number of lines does not match")
for i, line := range lines {
require.Equal(t, tc.expected[i], line, "Line %d does not match", i)
}
})
}
}
// TestScanLinesWithCarriageEdgeCases tests edge cases and potential error scenarios
func TestScanLinesWithCarriageEdgeCases(t *testing.T) {
t.Run("Empty input", func(t *testing.T) {
scanner := bufio.NewScanner(bytes.NewReader([]byte("")))
scanner.Split(scanLinesWithCarriage)
require.Equal(t, scanner.Scan(), false)
require.NoError(t, scanner.Err())
})
t.Run("Very long line", func(t *testing.T) {
longLine := bytes.Repeat([]byte("a"), bufio.MaxScanTokenSize+1)
scanner := bufio.NewScanner(bytes.NewReader(longLine))
scanner.Split(scanLinesWithCarriage)
require.Equal(t, scanner.Scan(), false)
require.NotNil(t, scanner.Err())
})
t.Run("Line ending at max token size", func(t *testing.T) {
input := append(bytes.Repeat([]byte("a"), bufio.MaxScanTokenSize-1), '\n')
scanner := bufio.NewScanner(bytes.NewReader(input))
scanner.Split(scanLinesWithCarriage)
require.Equal(t, scanner.Scan(), true)
require.Equal(t, string(bytes.Repeat([]byte("a"), bufio.MaxScanTokenSize-1)), scanner.Text())
})
}

View File

@@ -3,11 +3,11 @@ load("@prysm//tools/go:def.bzl", "go_library")
go_library(
name = "go_default_library",
srcs = ["client.go"],
importpath = "github.com/prysmaticlabs/prysm/v4/api/client/validator",
importpath = "github.com/prysmaticlabs/prysm/v5/api/client/validator",
visibility = ["//visibility:public"],
deps = [
"//api/client:go_default_library",
"//validator/rpc/apimiddleware:go_default_library",
"//validator/rpc:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)

View File

@@ -7,8 +7,8 @@ import (
"strings"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/api/client"
"github.com/prysmaticlabs/prysm/v4/validator/rpc/apimiddleware"
"github.com/prysmaticlabs/prysm/v5/api/client"
"github.com/prysmaticlabs/prysm/v5/validator/rpc"
)
const (
@@ -41,17 +41,17 @@ func (c *Client) GetValidatorPubKeys(ctx context.Context) ([]string, error) {
if err != nil {
return nil, err
}
if len(jsonlocal.Keystores) == 0 && len(jsonremote.Keystores) == 0 {
if len(jsonlocal.Data) == 0 && len(jsonremote.Data) == 0 {
return nil, errors.New("there are no local keys or remote keys on the validator")
}
hexKeys := make(map[string]bool)
for index := range jsonlocal.Keystores {
hexKeys[jsonlocal.Keystores[index].ValidatingPubkey] = true
for index := range jsonlocal.Data {
hexKeys[jsonlocal.Data[index].ValidatingPubkey] = true
}
for index := range jsonremote.Keystores {
hexKeys[jsonremote.Keystores[index].Pubkey] = true
for index := range jsonremote.Data {
hexKeys[jsonremote.Data[index].Pubkey] = true
}
keys := make([]string, 0)
for k := range hexKeys {
@@ -61,12 +61,12 @@ func (c *Client) GetValidatorPubKeys(ctx context.Context) ([]string, error) {
}
// GetLocalValidatorKeys calls the keymanager APIs for local validator keys
func (c *Client) GetLocalValidatorKeys(ctx context.Context) (*apimiddleware.ListKeystoresResponseJson, error) {
func (c *Client) GetLocalValidatorKeys(ctx context.Context) (*rpc.ListKeystoresResponse, error) {
localBytes, err := c.Get(ctx, localKeysPath, client.WithAuthorizationToken(c.Token()))
if err != nil {
return nil, err
}
jsonlocal := &apimiddleware.ListKeystoresResponseJson{}
jsonlocal := &rpc.ListKeystoresResponse{}
if err := json.Unmarshal(localBytes, jsonlocal); err != nil {
return nil, errors.Wrap(err, "failed to parse local keystore list")
}
@@ -74,14 +74,14 @@ func (c *Client) GetLocalValidatorKeys(ctx context.Context) (*apimiddleware.List
}
// GetRemoteValidatorKeys calls the keymanager APIs for web3signer validator keys
func (c *Client) GetRemoteValidatorKeys(ctx context.Context) (*apimiddleware.ListRemoteKeysResponseJson, error) {
func (c *Client) GetRemoteValidatorKeys(ctx context.Context) (*rpc.ListRemoteKeysResponse, error) {
remoteBytes, err := c.Get(ctx, remoteKeysPath, client.WithAuthorizationToken(c.Token()))
if err != nil {
if !strings.Contains(err.Error(), "Prysm Wallet is not of type Web3Signer") {
return nil, err
}
}
jsonremote := &apimiddleware.ListRemoteKeysResponseJson{}
jsonremote := &rpc.ListRemoteKeysResponse{}
if len(remoteBytes) != 0 {
if err := json.Unmarshal(remoteBytes, jsonremote); err != nil {
return nil, errors.Wrap(err, "failed to parse remote keystore list")
@@ -107,13 +107,13 @@ func (c *Client) GetFeeRecipientAddresses(ctx context.Context, validators []stri
}
// GetFeeRecipientAddress takes a public key and calls the keymanager API to return its fee recipient.
func (c *Client) GetFeeRecipientAddress(ctx context.Context, pubkey string) (*apimiddleware.GetFeeRecipientByPubkeyResponseJson, error) {
func (c *Client) GetFeeRecipientAddress(ctx context.Context, pubkey string) (*rpc.GetFeeRecipientByPubkeyResponse, error) {
path := strings.Replace(feeRecipientPath, "{pubkey}", pubkey, 1)
b, err := c.Get(ctx, path, client.WithAuthorizationToken(c.Token()))
if err != nil {
return nil, err
}
feejson := &apimiddleware.GetFeeRecipientByPubkeyResponseJson{}
feejson := &rpc.GetFeeRecipientByPubkeyResponse{}
if err := json.Unmarshal(b, feejson); err != nil {
return nil, errors.Wrap(err, "failed to parse fee recipient")
}

9
api/constants.go Normal file
View File

@@ -0,0 +1,9 @@
package api
const (
WebUrlPrefix = "/v2/validator/"
WebApiUrlPrefix = "/api/v2/validator/"
KeymanagerApiPrefix = "/eth/v1"
SystemLogsPrefix = "health/logs"
AuthTokenFileName = "auth-token"
)

View File

@@ -1,44 +0,0 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"gateway.go",
"log.go",
"modifiers.go",
"options.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/gateway",
visibility = [
"//beacon-chain:__subpackages__",
"//validator:__subpackages__",
],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//runtime:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_rs_cors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//connectivity:go_default_library",
"@org_golang_google_grpc//credentials:go_default_library",
"@org_golang_google_protobuf//proto:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["gateway_test.go"],
embed = [":go_default_library"],
deps = [
"//api/gateway/apimiddleware:go_default_library",
"//cmd/beacon-chain/flags:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@com_github_urfave_cli_v2//:go_default_library",
],
)

View File

@@ -1,43 +0,0 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"api_middleware.go",
"log.go",
"param_handling.go",
"process_field.go",
"process_request.go",
"structs.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/gateway/apimiddleware",
visibility = ["//visibility:public"],
deps = [
"//api:go_default_library",
"//api/grpc:go_default_library",
"//encoding/bytesutil:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
"@com_github_wealdtech_go_bytesutil//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = [
"param_handling_test.go",
"process_request_test.go",
],
embed = [":go_default_library"],
deps = [
"//api:go_default_library",
"//api/grpc:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_gorilla_mux//:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
],
)

View File

@@ -1,265 +0,0 @@
package apimiddleware
import (
"net/http"
"reflect"
"time"
"github.com/gorilla/mux"
)
// ApiProxyMiddleware is a proxy between an Ethereum consensus API HTTP client and grpc-gateway.
// The purpose of the proxy is to handle HTTP requests and gRPC responses in such a way that:
// - Ethereum consensus API requests can be handled by grpc-gateway correctly
// - gRPC responses can be returned as spec-compliant Ethereum consensus API responses
type ApiProxyMiddleware struct {
GatewayAddress string
EndpointCreator EndpointFactory
Timeout time.Duration
router *mux.Router
}
// EndpointFactory is responsible for creating new instances of Endpoint values.
type EndpointFactory interface {
Create(path string) (*Endpoint, error)
Paths() []string
IsNil() bool
}
// Endpoint is a representation of an API HTTP endpoint that should be proxied by the middleware.
type Endpoint struct {
Path string // The path of the HTTP endpoint.
GetResponse interface{} // The struct corresponding to the JSON structure used in a GET response.
PostRequest interface{} // The struct corresponding to the JSON structure used in a POST request.
PostResponse interface{} // The struct corresponding to the JSON structure used in a POST response.
DeleteRequest interface{} // The struct corresponding to the JSON structure used in a DELETE request.
DeleteResponse interface{} // The struct corresponding to the JSON structure used in a DELETE response.
RequestURLLiterals []string // Names of URL parameters that should not be base64-encoded.
RequestQueryParams []QueryParam // Query parameters of the request.
Err ErrorJson // The struct corresponding to the error that should be returned in case of a request failure.
Hooks HookCollection // A collection of functions that can be invoked at various stages of the request/response cycle.
CustomHandlers []CustomHandler // Functions that will be executed instead of the default request/response behaviour.
}
// RunDefault expresses whether the default processing logic should be carried out after running a pre hook.
type RunDefault bool
// DefaultEndpoint returns an Endpoint with default configuration, e.g. DefaultErrorJson for error handling.
func DefaultEndpoint() Endpoint {
return Endpoint{
Err: &DefaultErrorJson{},
}
}
// QueryParam represents a single query parameter's metadata.
type QueryParam struct {
Name string
Hex bool
Enum bool
}
// CustomHandler is a function that can be invoked at the very beginning of the request,
// essentially replacing the whole default request/response logic with custom logic for a specific endpoint.
type CustomHandler = func(m *ApiProxyMiddleware, endpoint Endpoint, w http.ResponseWriter, req *http.Request) (handled bool)
// HookCollection contains hooks that can be used to amend the default request/response cycle with custom logic for a specific endpoint.
type HookCollection struct {
OnPreDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) (RunDefault, ErrorJson)
OnPostDeserializeRequestBodyIntoContainer func(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson
OnPreDeserializeGrpcResponseBodyIntoContainer func([]byte, interface{}) (RunDefault, ErrorJson)
OnPreSerializeMiddlewareResponseIntoJson func(interface{}) (RunDefault, []byte, ErrorJson)
}
// fieldProcessor applies the processing function f to a value when the tag is present on the field.
type fieldProcessor struct {
tag string
f func(value reflect.Value) error
}
// Run starts the proxy, registering all proxy endpoints.
func (m *ApiProxyMiddleware) Run(gatewayRouter *mux.Router) {
for _, path := range m.EndpointCreator.Paths() {
gatewayRouter.HandleFunc(path, m.WithMiddleware(path))
}
m.router = gatewayRouter
}
// ServeHTTP for the proxy middleware.
func (m *ApiProxyMiddleware) ServeHTTP(w http.ResponseWriter, req *http.Request) {
m.router.ServeHTTP(w, req)
}
// WithMiddleware wraps the given endpoint handler with the middleware logic.
func (m *ApiProxyMiddleware) WithMiddleware(path string) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
endpoint, err := m.EndpointCreator.Create(path)
if err != nil {
log.WithError(err).Errorf("Could not create endpoint for path: %s", path)
return
}
for _, handler := range endpoint.CustomHandlers {
if handler(m, *endpoint, w, req) {
return
}
}
if req.Method == "POST" {
if errJson := handlePostRequestForEndpoint(endpoint, w, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if req.Method == "DELETE" && req.Body != http.NoBody {
if errJson := handleDeleteRequestForEndpoint(endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := m.PrepareRequestForProxying(*endpoint, req); errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcResp, errJson := m.ProxyRequest(req)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
grpcRespBody, errJson := ReadGrpcResponseBody(grpcResp.Body)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
var respJson []byte
if !GrpcResponseIsEmpty(grpcRespBody) {
respHasError, errJson := HandleGrpcResponseError(endpoint.Err, grpcResp, grpcRespBody, w)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
if respHasError {
return
}
var resp interface{}
if req.Method == "GET" {
resp = endpoint.GetResponse
} else if req.Method == "DELETE" {
resp = endpoint.DeleteResponse
} else {
resp = endpoint.PostResponse
}
if errJson := deserializeGrpcResponseBodyIntoContainerWrapped(endpoint, grpcRespBody, resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := ProcessMiddlewareResponseFields(resp); errJson != nil {
WriteError(w, errJson, nil)
return
}
respJson, errJson = serializeMiddlewareResponseIntoJsonWrapped(endpoint, respJson, resp)
if errJson != nil {
WriteError(w, errJson, nil)
return
}
}
if errJson := WriteMiddlewareResponseHeadersAndBody(grpcResp, respJson, w); errJson != nil {
WriteError(w, errJson, nil)
return
}
if errJson := Cleanup(grpcResp.Body); errJson != nil {
WriteError(w, errJson, nil)
return
}
}
}
func handlePostRequestForEndpoint(endpoint *Endpoint, w http.ResponseWriter, req *http.Request) ErrorJson {
if errJson := deserializeRequestBodyIntoContainerWrapped(endpoint, req, w); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.PostRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.PostRequest, req)
}
func handleDeleteRequestForEndpoint(endpoint *Endpoint, req *http.Request) ErrorJson {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.DeleteRequest); errJson != nil {
return errJson
}
if errJson := ProcessRequestContainerFields(endpoint.DeleteRequest); errJson != nil {
return errJson
}
return SetRequestBodyToRequestContainer(endpoint.DeleteRequest, req)
}
func deserializeRequestBodyIntoContainerWrapped(endpoint *Endpoint, req *http.Request, w http.ResponseWriter) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeRequestBodyIntoContainer(endpoint, w, req)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeRequestBodyIntoContainer(req.Body, endpoint.PostRequest); errJson != nil {
return errJson
}
}
if endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer != nil {
if errJson := endpoint.Hooks.OnPostDeserializeRequestBodyIntoContainer(endpoint, w, req); errJson != nil {
return errJson
}
}
return nil
}
func deserializeGrpcResponseBodyIntoContainerWrapped(endpoint *Endpoint, grpcResponseBody []byte, resp interface{}) ErrorJson {
runDefault := true
if endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer != nil {
run, errJson := endpoint.Hooks.OnPreDeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp)
if errJson != nil {
return errJson
}
if !run {
runDefault = false
}
}
if runDefault {
if errJson := DeserializeGrpcResponseBodyIntoContainer(grpcResponseBody, resp); errJson != nil {
return errJson
}
}
return nil
}
func serializeMiddlewareResponseIntoJsonWrapped(endpoint *Endpoint, respJson []byte, resp interface{}) ([]byte, ErrorJson) {
runDefault := true
var errJson ErrorJson
if endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson != nil {
var run RunDefault
run, respJson, errJson = endpoint.Hooks.OnPreSerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
if !run {
runDefault = false
}
}
if runDefault {
respJson, errJson = SerializeMiddlewareResponseIntoJson(resp)
if errJson != nil {
return nil, errJson
}
}
return respJson, nil
}

View File

@@ -1,5 +0,0 @@
package apimiddleware
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "apimiddleware")

View File

@@ -1,103 +0,0 @@
package apimiddleware
import (
"encoding/base64"
"net/http"
"net/url"
"strings"
"github.com/gorilla/mux"
butil "github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
"github.com/wealdtech/go-bytesutil"
)
// HandleURLParameters processes URL parameters, allowing parameterized URLs to be safely and correctly proxied to grpc-gateway.
func HandleURLParameters(url string, req *http.Request, literals []string) ErrorJson {
segments := strings.Split(url, "/")
segmentsLoop:
for i, s := range segments {
// We only care about segments which are parameterized.
if isRequestParam(s) {
// Don't do anything with parameters which should be forwarded literally to gRPC.
for _, l := range literals {
if s == "{"+l+"}" {
continue segmentsLoop
}
}
routeVar := mux.Vars(req)[s[1:len(s)-1]]
bRouteVar := []byte(routeVar)
if butil.IsHex(bRouteVar) {
var err error
bRouteVar, err = bytesutil.FromHexString(string(bRouteVar))
if err != nil {
return InternalServerErrorWithMessage(err, "could not process URL parameter")
}
}
// Converting hex to base64 may result in a value which malforms the URL.
// We use URLEncoding to safely escape such values.
base64RouteVar := base64.URLEncoding.EncodeToString(bRouteVar)
// Merge segments back into the full URL.
splitPath := strings.Split(req.URL.Path, "/")
splitPath[i] = base64RouteVar
req.URL.Path = strings.Join(splitPath, "/")
}
}
return nil
}
// HandleQueryParameters processes query parameters, allowing them to be safely and correctly proxied to grpc-gateway.
func HandleQueryParameters(req *http.Request, params []QueryParam) ErrorJson {
queryParams := req.URL.Query()
normalizeQueryValues(queryParams)
for key, vals := range queryParams {
for _, p := range params {
if key == p.Name {
if p.Hex {
queryParams.Del(key)
for _, v := range vals {
b := []byte(v)
if butil.IsHex(b) {
var err error
b, err = bytesutil.FromHexString(v)
if err != nil {
return InternalServerErrorWithMessage(err, "could not process query parameter")
}
}
queryParams.Add(key, base64.URLEncoding.EncodeToString(b))
}
}
if p.Enum {
queryParams.Del(key)
for _, v := range vals {
// gRPC expects uppercase enum values.
queryParams.Add(key, strings.ToUpper(v))
}
}
}
}
}
req.URL.RawQuery = queryParams.Encode()
return nil
}
// isRequestParam verifies whether the passed string is a request parameter.
// Request parameters are enclosed in { and }.
func isRequestParam(s string) bool {
return len(s) > 2 && s[0] == '{' && s[len(s)-1] == '}'
}
func normalizeQueryValues(queryParams url.Values) {
// Replace comma-separated values with individual values.
for key, vals := range queryParams {
splitVals := make([]string, 0)
for _, v := range vals {
splitVals = append(splitVals, strings.Split(v, ",")...)
}
queryParams[key] = splitVals
}
}

View File

@@ -1,124 +0,0 @@
package apimiddleware
import (
"bytes"
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
)
func TestHandleURLParameters(t *testing.T) {
var body bytes.Buffer
t.Run("no_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example/bar", &body)
errJson := HandleURLParameters("/not_param", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar", request.URL.Path)
})
t.Run("with_params", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
muxVars["quux_param"] = "quux"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz/quux", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/{quux_param}", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz/cXV1eA==", request.URL.Path)
})
t.Run("with_literal", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["bar_param"] = "bar"
request := httptest.NewRequest("GET", "http://foo.example/bar/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{bar_param}/not_param/", request, []string{"bar_param"})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/bar/baz", request.URL.Path)
})
t.Run("with_hex", func(t *testing.T) {
muxVars := make(map[string]string)
muxVars["hex_param"] = "0x626172"
request := httptest.NewRequest("GET", "http://foo.example/0x626172/baz", &body)
request = mux.SetURLVars(request, muxVars)
errJson := HandleURLParameters("/{hex_param}/not_param/", request, []string{})
require.Equal(t, true, errJson == nil)
assert.Equal(t, "/YmFy/baz", request.URL.Path)
})
}
func TestHandleQueryParameters(t *testing.T) {
var body bytes.Buffer
t.Run("regular_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?bar=bar&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "bar"}, {Name: "baz"}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["bar"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "bar", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "baz", v[0])
})
t.Run("hex_and_enum_params", func(t *testing.T) {
request := httptest.NewRequest("GET", "http://foo.example?hex=0x626172&baz=baz", &body)
errJson := HandleQueryParameters(request, []QueryParam{{Name: "hex", Hex: true}, {Name: "baz", Enum: true}})
require.Equal(t, true, errJson == nil)
query := request.URL.Query()
v, ok := query["hex"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "YmFy", v[0])
v, ok = query["baz"]
require.Equal(t, true, ok, "query param not found")
require.Equal(t, 1, len(v), "wrong number of query param values")
assert.Equal(t, "BAZ", v[0])
})
}
func TestIsRequestParam(t *testing.T) {
tests := []struct {
s string
b bool
}{
{"", false},
{"{", false},
{"}", false},
{"{}", false},
{"{x}", true},
{"{very_long_parameter_name_with_underscores}", true},
}
for _, tt := range tests {
b := isRequestParam(tt.s)
assert.Equal(t, tt.b, b)
}
}
func TestNormalizeQueryValues(t *testing.T) {
input := make(map[string][]string)
input["key"] = []string{"value1", "value2,value3,value4", "value5"}
normalizeQueryValues(input)
require.Equal(t, 5, len(input["key"]))
assert.Equal(t, "value1", input["key"][0])
assert.Equal(t, "value2", input["key"][1])
assert.Equal(t, "value3", input["key"][2])
assert.Equal(t, "value4", input["key"][3])
assert.Equal(t, "value5", input["key"][4])
}

View File

@@ -1,179 +0,0 @@
package apimiddleware
import (
"encoding/base64"
"fmt"
"math/big"
"reflect"
"strconv"
"strings"
"time"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/wealdtech/go-bytesutil"
)
// processField calls each processor function on any field that has the matching tag set.
// It is a recursive function.
func processField(s interface{}, processors []fieldProcessor) error {
kind := reflect.TypeOf(s).Kind()
if kind != reflect.Ptr && kind != reflect.Slice && kind != reflect.Array {
return fmt.Errorf("processing fields of kind '%v' is unsupported", kind)
}
t := reflect.TypeOf(s).Elem()
v := reflect.Indirect(reflect.ValueOf(s))
for i := 0; i < t.NumField(); i++ {
switch v.Field(i).Kind() {
case reflect.Slice:
sliceElem := t.Field(i).Type.Elem()
kind := sliceElem.Kind()
// Recursively process slices to struct pointers.
switch {
case kind == reflect.Ptr && sliceElem.Elem().Kind() == reflect.Struct:
for j := 0; j < v.Field(i).Len(); j++ {
if err := processField(v.Field(i).Index(j).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
// Process each string in string slices.
case kind == reflect.String:
for _, proc := range processors {
_, hasTag := t.Field(i).Tag.Lookup(proc.tag)
if !hasTag {
continue
}
for j := 0; j < v.Field(i).Len(); j++ {
if err := proc.f(v.Field(i).Index(j)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
// Recursively process struct pointers.
case reflect.Ptr:
if v.Field(i).Elem().Kind() == reflect.Struct {
if err := processField(v.Field(i).Interface(), processors); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
default:
field := t.Field(i)
for _, proc := range processors {
if _, hasTag := field.Tag.Lookup(proc.tag); hasTag {
if err := proc.f(v.Field(i)); err != nil {
return errors.Wrapf(err, "could not process field '%s'", t.Field(i).Name)
}
}
}
}
}
return nil
}
func hexToBase64Processor(v reflect.Value) error {
if v.String() == "0x" {
v.SetString("")
return nil
}
b, err := bytesutil.FromHexString(v.String())
if err != nil {
return err
}
v.SetString(base64.StdEncoding.EncodeToString(b))
return nil
}
func base64ToHexProcessor(v reflect.Value) error {
if v.String() == "" {
// Empty hex values are represented as "0x".
v.SetString("0x")
return nil
}
b, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
v.SetString(hexutil.Encode(b))
return nil
}
func base64ToChecksumAddressProcessor(v reflect.Value) error {
if v.String() == "" {
// Empty hex values are represented as "0x".
v.SetString("0x")
return nil
}
b, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
v.SetString(common.BytesToAddress(b).Hex())
return nil
}
func base64ToUint256Processor(v reflect.Value) error {
if v.String() == "" {
return nil
}
littleEndian, err := base64.StdEncoding.DecodeString(v.String())
if err != nil {
return err
}
if len(littleEndian) != 32 {
return errors.New("invalid length for Uint256")
}
// Integers are stored as little-endian, but
// big.Int expects big-endian. So we need to reverse
// the byte order before decoding.
var bigEndian [32]byte
for i := 0; i < len(littleEndian); i++ {
bigEndian[i] = littleEndian[len(littleEndian)-1-i]
}
var uint256 big.Int
uint256.SetBytes(bigEndian[:])
v.SetString(uint256.String())
return nil
}
func uint256ToBase64Processor(v reflect.Value) error {
if v.String() == "" {
return nil
}
uint256, ok := new(big.Int).SetString(v.String(), 10)
if !ok {
return fmt.Errorf("could not parse Uint256")
}
bigEndian := uint256.Bytes()
if len(bigEndian) > 32 {
return fmt.Errorf("number too big for Uint256")
}
// Integers are stored as little-endian, but
// big.Int gives big-endian. So we need to reverse
// the byte order before encoding.
var littleEndian [32]byte
for i := 0; i < len(bigEndian); i++ {
littleEndian[i] = bigEndian[len(bigEndian)-1-i]
}
v.SetString(base64.StdEncoding.EncodeToString(littleEndian[:]))
return nil
}
func enumToLowercaseProcessor(v reflect.Value) error {
v.SetString(strings.ToLower(v.String()))
return nil
}
func timeToUnixProcessor(v reflect.Value) error {
t, err := time.Parse(time.RFC3339, v.String())
if err != nil {
return err
}
v.SetString(strconv.FormatUint(uint64(t.Unix()), 10))
return nil
}

View File

@@ -1,283 +0,0 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"io"
"net"
"net/http"
"strconv"
"strings"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/api"
"github.com/prysmaticlabs/prysm/v4/api/grpc"
)
// DeserializeRequestBodyIntoContainer deserializes the request's body into an endpoint-specific struct.
func DeserializeRequestBodyIntoContainer(body io.Reader, requestContainer interface{}) ErrorJson {
decoder := json.NewDecoder(body)
decoder.DisallowUnknownFields()
if err := decoder.Decode(&requestContainer); err != nil {
if strings.Contains(err.Error(), "json: unknown field") {
e := errors.Wrap(err, "could not decode request body")
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusBadRequest,
}
}
return InternalServerErrorWithMessage(err, "could not decode request body")
}
return nil
}
// ProcessRequestContainerFields processes fields of an endpoint-specific container according to field tags.
func ProcessRequestContainerFields(requestContainer interface{}) ErrorJson {
if err := processField(requestContainer, []fieldProcessor{
{
tag: "hex",
f: hexToBase64Processor,
},
{
tag: "uint256",
f: uint256ToBase64Processor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process request data")
}
return nil
}
// SetRequestBodyToRequestContainer makes the endpoint-specific container the new body of the request.
func SetRequestBodyToRequestContainer(requestContainer interface{}, req *http.Request) ErrorJson {
// Serialize the struct, which now includes a base64-encoded value, into JSON.
j, err := json.Marshal(requestContainer)
if err != nil {
return InternalServerErrorWithMessage(err, "could not marshal request")
}
// Set the body to the new JSON.
req.Body = io.NopCloser(bytes.NewReader(j))
req.Header.Set("Content-Length", strconv.Itoa(len(j)))
req.ContentLength = int64(len(j))
return nil
}
// PrepareRequestForProxying applies additional logic to the request so that it can be correctly proxied to grpc-gateway.
func (m *ApiProxyMiddleware) PrepareRequestForProxying(endpoint Endpoint, req *http.Request) ErrorJson {
req.URL.Scheme = "http"
req.URL.Host = m.GatewayAddress
req.RequestURI = ""
if errJson := HandleURLParameters(endpoint.Path, req, endpoint.RequestURLLiterals); errJson != nil {
return errJson
}
if errJson := HandleQueryParameters(req, endpoint.RequestQueryParams); errJson != nil {
return errJson
}
// We have to add the prefix after handling parameters because adding the prefix changes URL segment indexing.
req.URL.Path = "/internal" + req.URL.Path
return nil
}
// ProxyRequest proxies the request to grpc-gateway.
func (m *ApiProxyMiddleware) ProxyRequest(req *http.Request) (*http.Response, ErrorJson) {
// We do not use http.DefaultClient because it does not have any timeout.
netClient := &http.Client{Timeout: m.Timeout}
grpcResp, err := netClient.Do(req)
if err != nil {
if err, ok := err.(net.Error); ok && err.Timeout() {
return nil, TimeoutError()
}
return nil, InternalServerErrorWithMessage(err, "could not proxy request")
}
if grpcResp == nil {
return nil, &DefaultErrorJson{Message: "nil response from gRPC-gateway", Code: http.StatusInternalServerError}
}
return grpcResp, nil
}
// ReadGrpcResponseBody reads the body from the grpc-gateway's response.
func ReadGrpcResponseBody(r io.Reader) ([]byte, ErrorJson) {
body, err := io.ReadAll(r)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not read response body")
}
return body, nil
}
// HandleGrpcResponseError acts on an error that resulted from a grpc-gateway's response.
// Whether there was an error is indicated by the bool return value. In case of an error,
// there is no need to write to the response because it's taken care of by the function.
func HandleGrpcResponseError(errJson ErrorJson, resp *http.Response, respBody []byte, w http.ResponseWriter) (bool, ErrorJson) {
responseHasError := false
if err := json.Unmarshal(respBody, errJson); err != nil {
return false, InternalServerErrorWithMessage(err, "could not unmarshal error")
}
if errJson.Msg() != "" {
responseHasError = true
// Something went wrong, but the request completed, meaning we can write headers and the error message.
for h, vs := range resp.Header {
for _, v := range vs {
if strings.HasSuffix(h, api.VersionHeader) {
w.Header().Set(api.VersionHeader, v)
} else {
w.Header().Set(h, v)
}
}
}
// Handle gRPC timeout.
if resp.StatusCode == http.StatusGatewayTimeout {
WriteError(w, TimeoutError(), resp.Header)
} else {
// Set code to HTTP code because unmarshalled body contained gRPC code.
errJson.SetCode(resp.StatusCode)
WriteError(w, errJson, resp.Header)
}
}
return responseHasError, nil
}
// GrpcResponseIsEmpty determines whether the grpc-gateway's response body contains no data.
func GrpcResponseIsEmpty(grpcResponseBody []byte) bool {
return len(grpcResponseBody) == 0 || string(grpcResponseBody) == "{}"
}
// DeserializeGrpcResponseBodyIntoContainer deserializes the grpc-gateway's response body into an endpoint-specific struct.
func DeserializeGrpcResponseBodyIntoContainer(body []byte, responseContainer interface{}) ErrorJson {
if err := json.Unmarshal(body, &responseContainer); err != nil {
return InternalServerErrorWithMessage(err, "could not unmarshal response")
}
return nil
}
// ProcessMiddlewareResponseFields processes fields of an endpoint-specific container according to field tags.
func ProcessMiddlewareResponseFields(responseContainer interface{}) ErrorJson {
if err := processField(responseContainer, []fieldProcessor{
{
tag: "hex",
f: base64ToHexProcessor,
},
{
tag: "address",
f: base64ToChecksumAddressProcessor,
},
{
tag: "enum",
f: enumToLowercaseProcessor,
},
{
tag: "time",
f: timeToUnixProcessor,
},
{
tag: "uint256",
f: base64ToUint256Processor,
},
}); err != nil {
return InternalServerErrorWithMessage(err, "could not process response data")
}
return nil
}
// SerializeMiddlewareResponseIntoJson serializes the endpoint-specific response struct into a JSON representation.
func SerializeMiddlewareResponseIntoJson(responseContainer interface{}) (jsonResponse []byte, errJson ErrorJson) {
j, err := json.Marshal(responseContainer)
if err != nil {
return nil, InternalServerErrorWithMessage(err, "could not marshal response")
}
return j, nil
}
// WriteMiddlewareResponseHeadersAndBody populates headers and the body of the final response.
func WriteMiddlewareResponseHeadersAndBody(grpcResp *http.Response, responseJson []byte, w http.ResponseWriter) ErrorJson {
var statusCodeHeader string
for h, vs := range grpcResp.Header {
// We don't want to expose any gRPC metadata in the HTTP response, so we skip forwarding metadata headers.
if strings.HasPrefix(h, grpc.MetadataPrefix) {
if h == grpc.WithPrefix(grpc.HttpCodeMetadataKey) {
statusCodeHeader = vs[0]
} else if strings.HasSuffix(h, api.VersionHeader) {
w.Header().Set(api.VersionHeader, vs[0])
}
} else {
for _, v := range vs {
w.Header().Set(h, v)
}
}
}
if !GrpcResponseIsEmpty(responseJson) {
w.Header().Set("Content-Length", strconv.Itoa(len(responseJson)))
if statusCodeHeader != "" {
code, err := strconv.Atoi(statusCodeHeader)
if err != nil {
return InternalServerErrorWithMessage(err, "could not parse status code")
}
w.WriteHeader(code)
} else {
w.WriteHeader(grpcResp.StatusCode)
}
if _, err := io.Copy(w, io.NopCloser(bytes.NewReader(responseJson))); err != nil {
return InternalServerErrorWithMessage(err, "could not write response message")
}
} else {
w.Header().Set("Content-Length", "0")
w.WriteHeader(grpcResp.StatusCode)
}
return nil
}
// WriteError writes the error by manipulating headers and the body of the final response.
func WriteError(w http.ResponseWriter, errJson ErrorJson, responseHeader http.Header) {
// Include custom error in the error JSON.
hasCustomError := false
if responseHeader != nil {
customError, ok := responseHeader[grpc.WithPrefix(grpc.CustomErrorMetadataKey)]
if ok {
hasCustomError = true
// Assume header has only one value and read the 0 index.
if err := json.Unmarshal([]byte(customError[0]), errJson); err != nil {
log.WithError(err).Error("Could not unmarshal custom error message")
return
}
}
}
var j []byte
if hasCustomError {
var err error
j, err = json.Marshal(errJson)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
} else {
var err error
// We marshal the response body into a DefaultErrorJson if the custom error is not present.
// This is because the ErrorJson argument is the endpoint's error definition, which may contain custom fields.
// In such a scenario marhaling the endpoint's error would populate the resulting JSON
// with these fields even if they are not present in the gRPC header.
d := &DefaultErrorJson{
Message: errJson.Msg(),
Code: errJson.StatusCode(),
}
j, err = json.Marshal(d)
if err != nil {
log.WithError(err).Error("Could not marshal error message")
return
}
}
w.Header().Set("Content-Length", strconv.Itoa(len(j)))
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(errJson.StatusCode())
if _, err := io.Copy(w, io.NopCloser(bytes.NewReader(j))); err != nil {
log.WithError(err).Error("Could not write error message")
}
}
// Cleanup performs final cleanup on the initial response from grpc-gateway.
func Cleanup(grpcResponseBody io.ReadCloser) ErrorJson {
if err := grpcResponseBody.Close(); err != nil {
return InternalServerErrorWithMessage(err, "could not close response body")
}
return nil
}

View File

@@ -1,435 +0,0 @@
package apimiddleware
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/prysmaticlabs/prysm/v4/api"
"github.com/prysmaticlabs/prysm/v4/api/grpc"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/sirupsen/logrus/hooks/test"
)
type testRequestContainer struct {
TestString string
TestHexString string `hex:"true"`
TestEmptyHexString string `hex:"true"`
TestUint256String string `uint256:"true"`
}
func defaultRequestContainer() *testRequestContainer {
return &testRequestContainer{
TestString: "test string",
TestHexString: "0x666F6F", // hex encoding of "foo"
TestEmptyHexString: "0x",
TestUint256String: "4196",
}
}
type testResponseContainer struct {
TestString string
TestHex string `hex:"true"`
TestEmptyHex string `hex:"true"`
TestAddress string `address:"true"`
TestEmptyAddress string `address:"true"`
TestUint256 string `uint256:"true"`
TestEnum string `enum:"true"`
TestTime string `time:"true"`
}
func defaultResponseContainer() *testResponseContainer {
return &testResponseContainer{
TestString: "test string",
TestHex: "Zm9v", // base64 encoding of "foo"
TestEmptyHex: "",
TestAddress: "Zm9v",
TestEmptyAddress: "",
TestEnum: "Test Enum",
TestTime: "2006-01-02T15:04:05Z",
// base64 encoding of 4196 in little-endian
TestUint256: "ZBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
}
}
type testErrorJson struct {
Message string
Code int
CustomField string
}
// StatusCode returns the error's underlying error code.
func (e *testErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *testErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *testErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *testErrorJson) SetMsg(msg string) {
e.Message = msg
}
func TestDeserializeRequestBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
var bodyJson bytes.Buffer
err := json.NewEncoder(&bodyJson).Encode(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("unknown field", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("{\"foo\":\"foo\"}"))
errJson := DeserializeRequestBodyIntoContainer(&bodyJson, &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not decode request body"))
assert.Equal(t, http.StatusBadRequest, errJson.StatusCode())
})
}
func TestProcessRequestContainerFields(t *testing.T) {
t.Run("ok", func(t *testing.T) {
container := defaultRequestContainer()
errJson := ProcessRequestContainerFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "Zm9v", container.TestHexString)
assert.Equal(t, "", container.TestEmptyHexString)
assert.Equal(t, "ZBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=", container.TestUint256String)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessRequestContainerFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process request data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSetRequestBodyToRequestContainer(t *testing.T) {
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example", &body)
errJson := SetRequestBodyToRequestContainer(defaultRequestContainer(), request)
require.Equal(t, true, errJson == nil)
container := &testRequestContainer{}
require.NoError(t, json.NewDecoder(request.Body).Decode(container))
assert.Equal(t, "test string", container.TestString)
contentLengthHeader, ok := request.Header["Content-Length"]
require.Equal(t, true, ok)
require.Equal(t, 1, len(contentLengthHeader), "wrong number of header values")
assert.Equal(t, "108", contentLengthHeader[0])
assert.Equal(t, int64(108), request.ContentLength)
}
func TestPrepareRequestForProxying(t *testing.T) {
middleware := &ApiProxyMiddleware{
GatewayAddress: "http://gateway.example",
}
// We will set some params to make the request more interesting.
endpoint := Endpoint{
Path: "/{url_param}",
RequestURLLiterals: []string{"url_param"},
RequestQueryParams: []QueryParam{{Name: "query_param"}},
}
var body bytes.Buffer
request := httptest.NewRequest("GET", "http://foo.example?query_param=bar", &body)
errJson := middleware.PrepareRequestForProxying(endpoint, request)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "http", request.URL.Scheme)
assert.Equal(t, middleware.GatewayAddress, request.URL.Host)
assert.Equal(t, "", request.RequestURI)
}
func TestReadGrpcResponseBody(t *testing.T) {
var b bytes.Buffer
b.Write([]byte("foo"))
body, jsonErr := ReadGrpcResponseBody(&b)
require.Equal(t, true, jsonErr == nil)
assert.Equal(t, "foo", string(body))
}
func TestHandleGrpcResponseError(t *testing.T) {
response := &http.Response{
StatusCode: 400,
Header: http.Header{
"Foo": []string{"foo"},
"Bar": []string{"bar"},
},
}
writer := httptest.NewRecorder()
errJson := &testErrorJson{
Message: "foo",
Code: 400,
}
b, err := json.Marshal(errJson)
require.NoError(t, err)
hasError, e := HandleGrpcResponseError(errJson, response, b, writer)
require.Equal(t, true, e == nil)
assert.Equal(t, true, hasError)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Bar"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "bar", v[0])
assert.Equal(t, 400, errJson.StatusCode())
}
func TestGrpcResponseIsEmpty(t *testing.T) {
t.Run("nil", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(nil))
})
t.Run("empty_slice", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty(make([]byte, 0)))
})
t.Run("empty_brackets", func(t *testing.T) {
assert.Equal(t, true, GrpcResponseIsEmpty([]byte("{}")))
})
t.Run("non_empty", func(t *testing.T) {
assert.Equal(t, false, GrpcResponseIsEmpty([]byte("{\"foo\":\"bar\"})")))
})
}
func TestDeserializeGrpcResponseBodyIntoContainer(t *testing.T) {
t.Run("ok", func(t *testing.T) {
body, err := json.Marshal(defaultRequestContainer())
require.NoError(t, err)
container := &testRequestContainer{}
errJson := DeserializeGrpcResponseBodyIntoContainer(body, container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "test string", container.TestString)
})
t.Run("error", func(t *testing.T) {
var bodyJson bytes.Buffer
bodyJson.Write([]byte("foo"))
errJson := DeserializeGrpcResponseBodyIntoContainer(bodyJson.Bytes(), &testRequestContainer{})
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not unmarshal response"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestProcessMiddlewareResponseFields(t *testing.T) {
t.Run("Ok", func(t *testing.T) {
container := defaultResponseContainer()
errJson := ProcessMiddlewareResponseFields(container)
require.Equal(t, true, errJson == nil)
assert.Equal(t, "0x666f6f", container.TestHex)
assert.Equal(t, "0x", container.TestEmptyHex)
assert.Equal(t, "0x0000000000000000000000000000000000666F6f", container.TestAddress)
assert.Equal(t, "0x", container.TestEmptyAddress)
assert.Equal(t, "4196", container.TestUint256)
assert.Equal(t, "test enum", container.TestEnum)
assert.Equal(t, "1136214245", container.TestTime)
})
t.Run("error", func(t *testing.T) {
errJson := ProcessMiddlewareResponseFields("foo")
require.NotNil(t, errJson)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not process response data"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
}
func TestSerializeMiddlewareResponseIntoJson(t *testing.T) {
container := defaultResponseContainer()
j, errJson := SerializeMiddlewareResponseIntoJson(container)
assert.Equal(t, true, errJson == nil)
cToDeserialize := &testResponseContainer{}
require.NoError(t, json.Unmarshal(j, cToDeserialize))
assert.Equal(t, "test string", cToDeserialize.TestString)
}
func TestWriteMiddlewareResponseHeadersAndBody(t *testing.T) {
t.Run("GET", func(t *testing.T) {
response := &http.Response{
Header: http.Header{
"Foo": []string{"foo"},
grpc.WithPrefix(grpc.HttpCodeMetadataKey): []string{"204"},
grpc.WithPrefix(api.VersionHeader): []string{"capella"},
},
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
v, ok := writer.Header()["Foo"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "foo", v[0])
v, ok = writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "224", v[0])
v, ok = writer.Header()["Eth-Consensus-Version"]
require.Equal(t, true, ok, "header not found")
assert.Equal(t, "capella", v[0])
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("GET_no_grpc_status_code_header", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("GET_invalid_status_code", func(t *testing.T) {
response := &http.Response{
Header: http.Header{"Grpc-Metadata-Eth-Consensus-Version": []string{"capella"}},
}
// Set invalid status code.
response.Header[grpc.WithPrefix(grpc.HttpCodeMetadataKey)] = []string{"invalid"}
response.Header[grpc.WithPrefix(api.VersionHeader)] = []string{"capella"}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, false, errJson == nil)
assert.Equal(t, true, strings.Contains(errJson.Msg(), "could not parse status code"))
assert.Equal(t, http.StatusInternalServerError, errJson.StatusCode())
})
t.Run("POST", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
})
t.Run("POST_with_response_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
container := defaultResponseContainer()
responseJson, err := json.Marshal(container)
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, responseJson, writer.Body.Bytes())
})
t.Run("POST_with_empty_json_body", func(t *testing.T) {
response := &http.Response{
Header: http.Header{},
StatusCode: 204,
}
responseJson, err := json.Marshal(struct{}{})
require.NoError(t, err)
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
errJson := WriteMiddlewareResponseHeadersAndBody(response, responseJson, writer)
require.Equal(t, true, errJson == nil)
assert.Equal(t, 204, writer.Code)
assert.DeepEqual(t, []byte(nil), writer.Body.Bytes())
assert.Equal(t, "0", writer.Header()["Content-Length"][0])
})
}
func TestWriteError(t *testing.T) {
t.Run("ok", func(t *testing.T) {
responseHeader := http.Header{
grpc.WithPrefix(grpc.CustomErrorMetadataKey): []string{"{\"CustomField\":\"bar\"}"},
}
errJson := &testErrorJson{
Message: "foo",
Code: 500,
}
writer := httptest.NewRecorder()
writer.Body = &bytes.Buffer{}
WriteError(writer, errJson, responseHeader)
v, ok := writer.Header()["Content-Length"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "48", v[0])
v, ok = writer.Header()["Content-Type"]
require.Equal(t, true, ok, "header not found")
require.Equal(t, 1, len(v), "wrong number of header values")
assert.Equal(t, "application/json", v[0])
assert.Equal(t, 500, writer.Code)
eDeserialize := &testErrorJson{}
require.NoError(t, json.Unmarshal(writer.Body.Bytes(), eDeserialize))
assert.Equal(t, "foo", eDeserialize.Message)
assert.Equal(t, 500, eDeserialize.Code)
assert.Equal(t, "bar", eDeserialize.CustomField)
})
t.Run("invalid_custom_error_header", func(t *testing.T) {
logHook := test.NewGlobal()
responseHeader := http.Header{
grpc.WithPrefix(grpc.CustomErrorMetadataKey): []string{"invalid"},
}
WriteError(httptest.NewRecorder(), &testErrorJson{}, responseHeader)
assert.LogsContain(t, logHook, "Could not unmarshal custom error message")
})
}

View File

@@ -1,69 +0,0 @@
package apimiddleware
import (
"net/http"
"github.com/pkg/errors"
)
// ---------------
// Error handling.
// ---------------
// ErrorJson describes common functionality of all JSON error representations.
type ErrorJson interface {
StatusCode() int
SetCode(code int)
Msg() string
SetMsg(msg string)
}
// DefaultErrorJson is a JSON representation of a simple error value, containing only a message and an error code.
type DefaultErrorJson struct {
Message string `json:"message"`
Code int `json:"code"`
}
// InternalServerErrorWithMessage returns a DefaultErrorJson with 500 code and a custom message.
func InternalServerErrorWithMessage(err error, message string) *DefaultErrorJson {
e := errors.Wrapf(err, message)
return &DefaultErrorJson{
Message: e.Error(),
Code: http.StatusInternalServerError,
}
}
// InternalServerError returns a DefaultErrorJson with 500 code.
func InternalServerError(err error) *DefaultErrorJson {
return &DefaultErrorJson{
Message: err.Error(),
Code: http.StatusInternalServerError,
}
}
func TimeoutError() *DefaultErrorJson {
return &DefaultErrorJson{
Message: "Request timeout",
Code: http.StatusRequestTimeout,
}
}
// StatusCode returns the error's underlying error code.
func (e *DefaultErrorJson) StatusCode() int {
return e.Code
}
// Msg returns the error's underlying message.
func (e *DefaultErrorJson) Msg() string {
return e.Message
}
// SetCode sets the error's underlying error code.
func (e *DefaultErrorJson) SetCode(code int) {
e.Code = code
}
// SetMsg sets the error's underlying message.
func (e *DefaultErrorJson) SetMsg(msg string) {
e.Message = msg
}

View File

@@ -1,258 +0,0 @@
// Package gateway defines a grpc-gateway server that serves HTTP-JSON traffic and acts a proxy between HTTP and gRPC.
package gateway
import (
"context"
"fmt"
"net"
"net/http"
"path"
"strings"
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/v4/runtime"
"github.com/rs/cors"
"google.golang.org/grpc"
"google.golang.org/grpc/connectivity"
"google.golang.org/grpc/credentials"
)
var _ runtime.Service = (*Gateway)(nil)
// PbMux serves grpc-gateway requests for selected patterns using registered protobuf handlers.
type PbMux struct {
Registrations []PbHandlerRegistration // Protobuf registrations to be registered in Mux.
Patterns []string // URL patterns that will be handled by Mux.
Mux *gwruntime.ServeMux // The router that will be used for grpc-gateway requests.
}
// PbHandlerRegistration is a function that registers a protobuf handler.
type PbHandlerRegistration func(context.Context, *gwruntime.ServeMux, *grpc.ClientConn) error
// MuxHandler is a function that implements the mux handler functionality.
type MuxHandler func(
apiMiddlewareHandler *apimiddleware.ApiProxyMiddleware,
h http.HandlerFunc,
w http.ResponseWriter,
req *http.Request,
)
// Config parameters for setting up the gateway service.
type config struct {
maxCallRecvMsgSize uint64
remoteCert string
gatewayAddr string
remoteAddr string
allowedOrigins []string
apiMiddlewareEndpointFactory apimiddleware.EndpointFactory
muxHandler MuxHandler
pbHandlers []*PbMux
router *mux.Router
timeout time.Duration
}
// Gateway is the gRPC gateway to serve HTTP JSON traffic as a proxy and forward it to the gRPC server.
type Gateway struct {
cfg *config
conn *grpc.ClientConn
server *http.Server
cancel context.CancelFunc
proxy *apimiddleware.ApiProxyMiddleware
ctx context.Context
startFailure error
}
// New returns a new instance of the Gateway.
func New(ctx context.Context, opts ...Option) (*Gateway, error) {
g := &Gateway{
ctx: ctx,
cfg: &config{},
}
for _, opt := range opts {
if err := opt(g); err != nil {
return nil, err
}
}
if g.cfg.router == nil {
g.cfg.router = mux.NewRouter()
}
return g, nil
}
// Start the gateway service.
func (g *Gateway) Start() {
ctx, cancel := context.WithCancel(g.ctx)
g.cancel = cancel
conn, err := g.dial(ctx, "tcp", g.cfg.remoteAddr)
if err != nil {
log.WithError(err).Error("Failed to connect to gRPC server")
g.startFailure = err
return
}
g.conn = conn
for _, h := range g.cfg.pbHandlers {
for _, r := range h.Registrations {
if err := r(ctx, h.Mux, g.conn); err != nil {
log.WithError(err).Error("Failed to register handler")
g.startFailure = err
return
}
}
for _, p := range h.Patterns {
g.cfg.router.PathPrefix(p).Handler(h.Mux)
}
}
corsMux := g.corsMiddleware(g.cfg.router)
if g.cfg.apiMiddlewareEndpointFactory != nil && !g.cfg.apiMiddlewareEndpointFactory.IsNil() {
g.registerApiMiddleware()
}
if g.cfg.muxHandler != nil {
g.cfg.router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.cfg.muxHandler(g.proxy, corsMux.ServeHTTP, w, r)
})
}
g.server = &http.Server{
Addr: g.cfg.gatewayAddr,
Handler: corsMux,
ReadHeaderTimeout: time.Second,
}
go func() {
log.WithField("address", g.cfg.gatewayAddr).Info("Starting gRPC gateway")
if err := g.server.ListenAndServe(); err != http.ErrServerClosed {
log.WithError(err).Error("Failed to start gRPC gateway")
g.startFailure = err
return
}
}()
}
// Status of grpc gateway. Returns an error if this service is unhealthy.
func (g *Gateway) Status() error {
if g.startFailure != nil {
return g.startFailure
}
if s := g.conn.GetState(); s != connectivity.Ready {
return fmt.Errorf("grpc server is %s", s)
}
return nil
}
// Stop the gateway with a graceful shutdown.
func (g *Gateway) Stop() error {
if g.server != nil {
shutdownCtx, shutdownCancel := context.WithTimeout(g.ctx, 2*time.Second)
defer shutdownCancel()
if err := g.server.Shutdown(shutdownCtx); err != nil {
if errors.Is(err, context.DeadlineExceeded) {
log.Warn("Existing connections terminated")
} else {
log.WithError(err).Error("Failed to gracefully shut down server")
}
}
}
if g.cancel != nil {
g.cancel()
}
return nil
}
func (g *Gateway) corsMiddleware(h http.Handler) http.Handler {
c := cors.New(cors.Options{
AllowedOrigins: g.cfg.allowedOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodDelete, http.MethodOptions},
AllowCredentials: true,
MaxAge: 600,
AllowedHeaders: []string{"*"},
})
return c.Handler(h)
}
const swaggerDir = "proto/prysm/v1alpha1/"
// SwaggerServer returns swagger specification files located under "/swagger/"
func SwaggerServer() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !strings.HasSuffix(r.URL.Path, ".swagger.json") {
log.Debugf("Not found: %s", r.URL.Path)
http.NotFound(w, r)
return
}
log.Debugf("Serving %s\n", r.URL.Path)
p := strings.TrimPrefix(r.URL.Path, "/swagger/")
p = path.Join(swaggerDir, p)
http.ServeFile(w, r, p)
}
}
// dial the gRPC server.
func (g *Gateway) dial(ctx context.Context, network, addr string) (*grpc.ClientConn, error) {
switch network {
case "tcp":
return g.dialTCP(ctx, addr)
case "unix":
return g.dialUnix(ctx, addr)
default:
return nil, fmt.Errorf("unsupported network type %q", network)
}
}
// dialTCP creates a client connection via TCP.
// "addr" must be a valid TCP address with a port number.
func (g *Gateway) dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, error) {
security := grpc.WithInsecure()
if len(g.cfg.remoteCert) > 0 {
creds, err := credentials.NewClientTLSFromFile(g.cfg.remoteCert, "")
if err != nil {
return nil, err
}
security = grpc.WithTransportCredentials(creds)
}
opts := []grpc.DialOption{
security,
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
// dialUnix creates a client connection via a unix domain socket.
// "addr" must be a valid path to the socket.
func (g *Gateway) dialUnix(ctx context.Context, addr string) (*grpc.ClientConn, error) {
d := func(addr string, timeout time.Duration) (net.Conn, error) {
return net.DialTimeout("unix", addr, timeout)
}
f := func(ctx context.Context, addr string) (net.Conn, error) {
if deadline, ok := ctx.Deadline(); ok {
return d(addr, time.Until(deadline))
}
return d(addr, 0)
}
opts := []grpc.DialOption{
grpc.WithInsecure(),
grpc.WithContextDialer(f),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(int(g.cfg.maxCallRecvMsgSize))),
}
return grpc.DialContext(ctx, addr, opts...)
}
func (g *Gateway) registerApiMiddleware() {
g.proxy = &apimiddleware.ApiProxyMiddleware{
GatewayAddress: g.cfg.gatewayAddr,
EndpointCreator: g.cfg.apiMiddlewareEndpointFactory,
Timeout: g.cfg.timeout,
}
log.Info("Starting API middleware")
g.proxy.Run(g.cfg.router)
}

View File

@@ -1,128 +0,0 @@
package gateway
import (
"context"
"flag"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/gorilla/mux"
"github.com/prysmaticlabs/prysm/v4/api/gateway/apimiddleware"
"github.com/prysmaticlabs/prysm/v4/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"github.com/urfave/cli/v2"
)
type mockEndpointFactory struct {
}
func (*mockEndpointFactory) Paths() []string {
return []string{}
}
func (*mockEndpointFactory) Create(_ string) (*apimiddleware.Endpoint, error) {
return nil, nil
}
func (*mockEndpointFactory) IsNil() bool {
return false
}
func TestGateway_Customized(t *testing.T) {
r := mux.NewRouter()
cert := "cert"
origins := []string{"origin"}
size := uint64(100)
endpointFactory := &mockEndpointFactory{}
opts := []Option{
WithRouter(r),
WithRemoteCert(cert),
WithAllowedOrigins(origins),
WithMaxCallRecvMsgSize(size),
WithApiMiddleware(endpointFactory),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
assert.Equal(t, r, g.cfg.router)
assert.Equal(t, cert, g.cfg.remoteCert)
require.Equal(t, 1, len(g.cfg.allowedOrigins))
assert.Equal(t, origins[0], g.cfg.allowedOrigins[0])
assert.Equal(t, size, g.cfg.maxCallRecvMsgSize)
assert.Equal(t, endpointFactory, g.cfg.apiMiddlewareEndpointFactory)
}
func TestGateway_StartStop(t *testing.T) {
hook := logTest.NewGlobal()
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
WithMuxHandler(func(
_ *apimiddleware.ApiProxyMiddleware,
_ http.HandlerFunc,
_ http.ResponseWriter,
_ *http.Request,
) {
}),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
g.Start()
go func() {
require.LogsContain(t, hook, "Starting gRPC gateway")
require.LogsDoNotContain(t, hook, "Starting API middleware")
}()
err = g.Stop()
require.NoError(t, err)
}
func TestGateway_NilHandler_NotFoundHandlerRegistered(t *testing.T) {
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
gatewayPort := ctx.Int(flags.GRPCGatewayPort.Name)
gatewayHost := ctx.String(flags.GRPCGatewayHost.Name)
rpcHost := ctx.String(flags.RPCHost.Name)
selfAddress := fmt.Sprintf("%s:%d", rpcHost, ctx.Int(flags.RPCPort.Name))
gatewayAddress := fmt.Sprintf("%s:%d", gatewayHost, gatewayPort)
opts := []Option{
WithGatewayAddr(gatewayAddress),
WithRemoteAddr(selfAddress),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
writer := httptest.NewRecorder()
g.cfg.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
assert.Equal(t, http.StatusNotFound, writer.Code)
}

View File

@@ -1,5 +0,0 @@
package gateway
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "gateway")

View File

@@ -1,30 +0,0 @@
package gateway
import (
"context"
"net/http"
"strconv"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"google.golang.org/protobuf/proto"
)
func HttpResponseModifier(ctx context.Context, w http.ResponseWriter, _ proto.Message) error {
md, ok := gwruntime.ServerMetadataFromContext(ctx)
if !ok {
return nil
}
// set http status code
if vals := md.HeaderMD.Get("x-http-code"); len(vals) > 0 {
code, err := strconv.Atoi(vals[0])
if err != nil {
return err
}
// delete the headers to not expose any grpc-metadata in http response
delete(md.HeaderMD, "x-http-code")
delete(w.Header(), "Grpc-Metadata-X-Http-Code")
w.WriteHeader(code)
}
return nil
}

View File

@@ -1,88 +0,0 @@
package gateway
import (
"time"
"github.com/gorilla/mux"
gwruntime "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/v4/api/gateway/apimiddleware"
)
type Option func(g *Gateway) error
func WithPbHandlers(handlers []*PbMux) Option {
return func(g *Gateway) error {
g.cfg.pbHandlers = handlers
return nil
}
}
func WithMuxHandler(m MuxHandler) Option {
return func(g *Gateway) error {
g.cfg.muxHandler = m
return nil
}
}
func WithGatewayAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.gatewayAddr = addr
return nil
}
}
func WithRemoteAddr(addr string) Option {
return func(g *Gateway) error {
g.cfg.remoteAddr = addr
return nil
}
}
// WithRouter allows adding a custom mux router to the gateway.
func WithRouter(r *mux.Router) Option {
return func(g *Gateway) error {
g.cfg.router = r
return nil
}
}
// WithAllowedOrigins allows adding a set of allowed origins to the gateway.
func WithAllowedOrigins(origins []string) Option {
return func(g *Gateway) error {
g.cfg.allowedOrigins = origins
return nil
}
}
// WithRemoteCert allows adding a custom certificate to the gateway,
func WithRemoteCert(cert string) Option {
return func(g *Gateway) error {
g.cfg.remoteCert = cert
return nil
}
}
// WithMaxCallRecvMsgSize allows specifying the maximum allowed gRPC message size.
func WithMaxCallRecvMsgSize(size uint64) Option {
return func(g *Gateway) error {
g.cfg.maxCallRecvMsgSize = size
return nil
}
}
// WithApiMiddleware allows adding an API middleware proxy to the gateway.
func WithApiMiddleware(endpointFactory apimiddleware.EndpointFactory) Option {
return func(g *Gateway) error {
g.cfg.apiMiddlewareEndpointFactory = endpointFactory
return nil
}
}
// WithTimeout allows changing the timeout value for API calls.
func WithTimeout(seconds uint64) Option {
return func(g *Gateway) error {
g.cfg.timeout = time.Second * time.Duration(seconds)
gwruntime.DefaultContextTimeout = time.Second * time.Duration(seconds)
return nil
}
}

View File

@@ -6,7 +6,7 @@ go_library(
"grpcutils.go",
"parameters.go",
],
importpath = "github.com/prysmaticlabs/prysm/v4/api/grpc",
importpath = "github.com/prysmaticlabs/prysm/v5/api/grpc",
visibility = ["//visibility:public"],
deps = [
"@com_github_sirupsen_logrus//:go_default_library",
@@ -22,9 +22,7 @@ go_test(
deps = [
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_grpc_ecosystem_grpc_gateway_v2//runtime:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//metadata:go_default_library",
],
)

View File

@@ -2,8 +2,6 @@ package grpc
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
@@ -81,16 +79,3 @@ func AppendHeaders(parent context.Context, headers []string) context.Context {
}
return parent
}
// AppendCustomErrorHeader sets a CustomErrorMetadataKey gRPC header on the passed in context,
// using the passed in error data as the header's value. The data is serialized as JSON.
func AppendCustomErrorHeader(ctx context.Context, errorData interface{}) error {
j, err := json.Marshal(errorData)
if err != nil {
return fmt.Errorf("could not marshal error data into JSON: %w", err)
}
if err := grpc.SetHeader(ctx, metadata.Pairs(CustomErrorMetadataKey, string(j))); err != nil {
return fmt.Errorf("could not set custom error header: %w", err)
}
return nil
}

View File

@@ -2,15 +2,11 @@ package grpc
import (
"context"
"encoding/json"
"strings"
"testing"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
@@ -62,17 +58,3 @@ func TestAppendHeaders(t *testing.T) {
assert.Equal(t, "value=1", md.Get("first")[0])
})
}
func TestAppendCustomErrorHeader(t *testing.T) {
stream := &runtime.ServerTransportStream{}
ctx := grpc.NewContextWithServerTransportStream(context.Background(), stream)
data := &customErrorData{Message: "foo"}
require.NoError(t, AppendCustomErrorHeader(ctx, data))
// The stream used in test setup sets the metadata key in lowercase.
value, ok := stream.Header()[strings.ToLower(CustomErrorMetadataKey)]
require.Equal(t, true, ok, "Failed to retrieve custom error metadata value")
expected, err := json.Marshal(data)
require.NoError(t, err)
assert.Equal(t, string(expected), value[0])
}

View File

@@ -1,7 +1,20 @@
package api
import "net/http"
const (
VersionHeader = "Eth-Consensus-Version"
JsonMediaType = "application/json"
OctetStreamMediaType = "application/octet-stream"
VersionHeader = "Eth-Consensus-Version"
ExecutionPayloadBlindedHeader = "Eth-Execution-Payload-Blinded"
ExecutionPayloadValueHeader = "Eth-Execution-Payload-Value"
ConsensusBlockValueHeader = "Eth-Consensus-Block-Value"
JsonMediaType = "application/json"
OctetStreamMediaType = "application/octet-stream"
EventStreamMediaType = "text/event-stream"
KeepAlive = "keep-alive"
)
// SetSSEHeaders sets the headers needed for a server-sent event response.
func SetSSEHeaders(w http.ResponseWriter) {
w.Header().Set("Content-Type", EventStreamMediaType)
w.Header().Set("Connection", KeepAlive)
}

32
api/jwt.go Normal file
View File

@@ -0,0 +1,32 @@
package api
import (
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v5/crypto/rand"
)
// GenerateRandomHexString generates a random hex string that follows the standards for jwt token
// used for beacon node -> execution client
// used for web client -> validator client
func GenerateRandomHexString() (string, error) {
secret := make([]byte, 32)
randGen := rand.NewGenerator()
n, err := randGen.Read(secret)
if err != nil {
return "", err
} else if n != 32 {
return "", errors.New("rand: unexpected length")
}
return hexutil.Encode(secret), nil
}
// ValidateAuthToken validating auth token for web
func ValidateAuthToken(token string) error {
b, err := hexutil.Decode(token)
// token should be hex-encoded and at least 256 bits
if err != nil || len(b) < 32 {
return errors.New("invalid auth token: token should be hex-encoded and at least 256 bits")
}
return nil
}

13
api/jwt_test.go Normal file
View File

@@ -0,0 +1,13 @@
package api
import (
"testing"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestGenerateRandomHexString(t *testing.T) {
token, err := GenerateRandomHexString()
require.NoError(t, err)
require.NoError(t, ValidateAuthToken(token))
}

View File

@@ -3,7 +3,7 @@ load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["pagination.go"],
importpath = "github.com/prysmaticlabs/prysm/v4/api/pagination",
importpath = "github.com/prysmaticlabs/prysm/v5/api/pagination",
visibility = ["//visibility:public"],
deps = [
"//config/params:go_default_library",

View File

@@ -6,7 +6,7 @@ import (
"strconv"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v4/config/params"
"github.com/prysmaticlabs/prysm/v5/config/params"
)
// StartAndEndPage takes in the requested page token, wanted page size, total page size.
@@ -15,6 +15,9 @@ func StartAndEndPage(pageToken string, pageSize, totalSize int) (int, int, strin
if pageToken == "" {
pageToken = "0"
}
if pageSize < 0 || totalSize < 0 {
return 0, 0, "", errors.Errorf("invalid page and total sizes provided: page size %d , total size %d", pageSize, totalSize)
}
if pageSize == 0 {
pageSize = params.BeaconConfig().DefaultPageSize
}
@@ -23,6 +26,9 @@ func StartAndEndPage(pageToken string, pageSize, totalSize int) (int, int, strin
if err != nil {
return 0, 0, "", errors.Wrap(err, "could not convert page token")
}
if token < 0 {
return 0, 0, "", errors.Errorf("invalid token value provided: %d", token)
}
// Start page can not be greater than set size.
start := token * pageSize

View File

@@ -3,9 +3,9 @@ package pagination_test
import (
"testing"
"github.com/prysmaticlabs/prysm/v4/api/pagination"
"github.com/prysmaticlabs/prysm/v4/testing/assert"
"github.com/prysmaticlabs/prysm/v4/testing/require"
"github.com/prysmaticlabs/prysm/v5/api/pagination"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestStartAndEndPage(t *testing.T) {
@@ -85,3 +85,19 @@ func TestStartAndEndPage_ExceedsMaxPage(t *testing.T) {
_, _, _, err := pagination.StartAndEndPage("", 0, 0)
assert.ErrorContains(t, wanted, err)
}
func TestStartAndEndPage_InvalidPageValues(t *testing.T) {
_, _, _, err := pagination.StartAndEndPage("10", -1, 10)
assert.ErrorContains(t, "invalid page and total sizes provided", err)
_, _, _, err = pagination.StartAndEndPage("12", 10, -10)
assert.ErrorContains(t, "invalid page and total sizes provided", err)
_, _, _, err = pagination.StartAndEndPage("12", -50, -60)
assert.ErrorContains(t, "invalid page and total sizes provided", err)
}
func TestStartAndEndPage_InvalidTokenValue(t *testing.T) {
_, _, _, err := pagination.StartAndEndPage("-12", 50, 60)
assert.ErrorContains(t, "invalid token value provided", err)
}

15
api/server/BUILD.bazel Normal file
View File

@@ -0,0 +1,15 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = ["error.go"],
importpath = "github.com/prysmaticlabs/prysm/v5/api/server",
visibility = ["//visibility:public"],
)
go_test(
name = "go_default_test",
srcs = ["error_test.go"],
embed = [":go_default_library"],
deps = ["//testing/assert:go_default_library"],
)

47
api/server/error.go Normal file
View File

@@ -0,0 +1,47 @@
package server
import (
"errors"
"fmt"
"strings"
)
// DecodeError represents an error resulting from trying to decode an HTTP request.
// It tracks the full field name for which decoding failed.
type DecodeError struct {
path []string
err error
}
// NewDecodeError wraps an error (either the initial decoding error or another DecodeError).
// The current field that failed decoding must be passed in.
func NewDecodeError(err error, field string) *DecodeError {
var de *DecodeError
ok := errors.As(err, &de)
if ok {
return &DecodeError{path: append([]string{field}, de.path...), err: de.err}
}
return &DecodeError{path: []string{field}, err: err}
}
// Error returns the formatted error message which contains the full field name and the actual decoding error.
func (e *DecodeError) Error() string {
return fmt.Sprintf("could not decode %s: %s", strings.Join(e.path, "."), e.err.Error())
}
// IndexedVerificationFailureError wraps a collection of verification failures.
type IndexedVerificationFailureError struct {
Message string `json:"message"`
Code int `json:"code"`
Failures []*IndexedVerificationFailure `json:"failures"`
}
func (e *IndexedVerificationFailureError) StatusCode() int {
return e.Code
}
// IndexedVerificationFailure represents an issue when verifying a single indexed object e.g. an item in an array.
type IndexedVerificationFailure struct {
Index int `json:"index"`
Message string `json:"message"`
}

16
api/server/error_test.go Normal file
View File

@@ -0,0 +1,16 @@
package server
import (
"errors"
"testing"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
)
func TestDecodeError(t *testing.T) {
e := errors.New("not a number")
de := NewDecodeError(e, "Z")
de = NewDecodeError(de, "Y")
de = NewDecodeError(de, "X")
assert.Equal(t, "could not decode X.Y.Z: not a number", de.Error())
}

View File

@@ -0,0 +1,31 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"log.go",
"options.go",
"server.go",
],
importpath = "github.com/prysmaticlabs/prysm/v5/api/server/httprest",
visibility = ["//visibility:public"],
deps = [
"//api/server/middleware:go_default_library",
"//runtime:go_default_library",
"@com_github_pkg_errors//:go_default_library",
"@com_github_sirupsen_logrus//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["server_test.go"],
embed = [":go_default_library"],
deps = [
"//cmd/beacon-chain/flags:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
"@com_github_sirupsen_logrus//hooks/test:go_default_library",
"@com_github_urfave_cli_v2//:go_default_library",
],
)

View File

@@ -0,0 +1,5 @@
package httprest
import "github.com/sirupsen/logrus"
var log = logrus.WithField("prefix", "httprest")

View File

@@ -0,0 +1,44 @@
package httprest
import (
"time"
"net/http"
"github.com/prysmaticlabs/prysm/v5/api/server/middleware"
)
// Option is a http rest server functional parameter type.
type Option func(g *Server) error
// WithMiddlewares sets the list of middlewares to be applied on routes.
func WithMiddlewares(mw []middleware.Middleware) Option {
return func(g *Server) error {
g.cfg.middlewares = mw
return nil
}
}
// WithHTTPAddr sets the full address ( host and port ) of the server.
func WithHTTPAddr(addr string) Option {
return func(g *Server) error {
g.cfg.httpAddr = addr
return nil
}
}
// WithRouter sets the internal router of the server, this is required.
func WithRouter(r *http.ServeMux) Option {
return func(g *Server) error {
g.cfg.router = r
return nil
}
}
// WithTimeout allows changing the timeout value for API calls.
func WithTimeout(duration time.Duration) Option {
return func(g *Server) error {
g.cfg.timeout = duration
return nil
}
}

View File

@@ -0,0 +1,101 @@
package httprest
import (
"context"
"net/http"
"time"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v5/api/server/middleware"
"github.com/prysmaticlabs/prysm/v5/runtime"
)
var _ runtime.Service = (*Server)(nil)
// Config parameters for setting up the http-rest service.
type config struct {
httpAddr string
middlewares []middleware.Middleware
router http.Handler
timeout time.Duration
}
// Server serves HTTP traffic.
type Server struct {
cfg *config
server *http.Server
cancel context.CancelFunc
ctx context.Context
startFailure error
}
// New returns a new instance of the Server.
func New(ctx context.Context, opts ...Option) (*Server, error) {
g := &Server{
ctx: ctx,
cfg: &config{},
}
for _, opt := range opts {
if err := opt(g); err != nil {
return nil, err
}
}
if g.cfg.router == nil {
return nil, errors.New("router option not configured")
}
var handler http.Handler
defaultReadHeaderTimeout := time.Second
handler = middleware.MiddlewareChain(g.cfg.router, g.cfg.middlewares)
if g.cfg.timeout > 0*time.Second {
defaultReadHeaderTimeout = g.cfg.timeout
handler = http.TimeoutHandler(handler, g.cfg.timeout, "request timed out")
}
g.server = &http.Server{
Addr: g.cfg.httpAddr,
Handler: handler,
ReadHeaderTimeout: defaultReadHeaderTimeout,
}
return g, nil
}
// Start the http rest service.
func (g *Server) Start() {
g.ctx, g.cancel = context.WithCancel(g.ctx)
go func() {
log.WithField("address", g.cfg.httpAddr).Info("Starting HTTP server")
if err := g.server.ListenAndServe(); err != http.ErrServerClosed {
log.WithError(err).Error("Failed to start HTTP server")
g.startFailure = err
return
}
}()
}
// Status of the HTTP server. Returns an error if this service is unhealthy.
func (g *Server) Status() error {
if g.startFailure != nil {
return g.startFailure
}
return nil
}
// Stop the HTTP server with a graceful shutdown.
func (g *Server) Stop() error {
if g.server != nil {
shutdownCtx, shutdownCancel := context.WithTimeout(g.ctx, 2*time.Second)
defer shutdownCancel()
if err := g.server.Shutdown(shutdownCtx); err != nil {
if errors.Is(err, context.DeadlineExceeded) {
log.Warn("Existing connections terminated")
} else {
log.WithError(err).Error("Failed to gracefully shut down server")
}
}
}
if g.cancel != nil {
g.cancel()
}
return nil
}

View File

@@ -0,0 +1,71 @@
package httprest
import (
"context"
"flag"
"fmt"
"net"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/prysmaticlabs/prysm/v5/cmd/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
logTest "github.com/sirupsen/logrus/hooks/test"
"github.com/urfave/cli/v2"
)
func TestServer_StartStop(t *testing.T) {
hook := logTest.NewGlobal()
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
port := ctx.Int(flags.HTTPServerPort.Name)
portStr := fmt.Sprintf("%d", port) // Convert port to string
host := ctx.String(flags.HTTPServerHost.Name)
address := net.JoinHostPort(host, portStr)
handler := http.NewServeMux()
opts := []Option{
WithHTTPAddr(address),
WithRouter(handler),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
g.Start()
go func() {
require.LogsContain(t, hook, "Starting HTTP server")
require.LogsDoNotContain(t, hook, "Starting API middleware")
}()
err = g.Stop()
require.NoError(t, err)
}
func TestServer_NilHandler_NotFoundHandlerRegistered(t *testing.T) {
app := cli.App{}
set := flag.NewFlagSet("test", 0)
ctx := cli.NewContext(&app, set, nil)
handler := http.NewServeMux()
port := ctx.Int(flags.HTTPServerPort.Name)
portStr := fmt.Sprintf("%d", port) // Convert port to string
host := ctx.String(flags.HTTPServerHost.Name)
address := net.JoinHostPort(host, portStr)
opts := []Option{
WithHTTPAddr(address),
WithRouter(handler),
}
g, err := New(context.Background(), opts...)
require.NoError(t, err)
writer := httptest.NewRecorder()
g.cfg.router.ServeHTTP(writer, &http.Request{Method: "GET", Host: "localhost", URL: &url.URL{Path: "/foo"}})
assert.Equal(t, http.StatusNotFound, writer.Code)
}

View File

@@ -0,0 +1,26 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"middleware.go",
"util.go",
],
importpath = "github.com/prysmaticlabs/prysm/v5/api/server/middleware",
visibility = ["//visibility:public"],
deps = ["@com_github_rs_cors//:go_default_library"],
)
go_test(
name = "go_default_test",
srcs = [
"middleware_test.go",
"util_test.go",
],
embed = [":go_default_library"],
deps = [
"//api:go_default_library",
"//testing/assert:go_default_library",
"//testing/require:go_default_library",
],
)

View File

@@ -0,0 +1,125 @@
package middleware
import (
"fmt"
"net/http"
"strings"
"github.com/rs/cors"
)
type Middleware func(http.Handler) http.Handler
// NormalizeQueryValuesHandler normalizes an input query of "key=value1,value2,value3" to "key=value1&key=value2&key=value3"
func NormalizeQueryValuesHandler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
query := r.URL.Query()
NormalizeQueryValues(query)
r.URL.RawQuery = query.Encode()
next.ServeHTTP(w, r)
})
}
// CorsHandler sets the cors settings on api endpoints
func CorsHandler(allowOrigins []string) Middleware {
c := cors.New(cors.Options{
AllowedOrigins: allowOrigins,
AllowedMethods: []string{http.MethodPost, http.MethodGet, http.MethodDelete, http.MethodOptions},
AllowCredentials: true,
MaxAge: 600,
AllowedHeaders: []string{"*"},
})
return c.Handler
}
// ContentTypeHandler checks request for the appropriate media types otherwise returning a http.StatusUnsupportedMediaType error
func ContentTypeHandler(acceptedMediaTypes []string) Middleware {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// skip the GET request
if r.Method == http.MethodGet {
next.ServeHTTP(w, r)
return
}
contentType := r.Header.Get("Content-Type")
if contentType == "" {
http.Error(w, "Content-Type header is missing", http.StatusUnsupportedMediaType)
return
}
accepted := false
for _, acceptedType := range acceptedMediaTypes {
if strings.Contains(strings.TrimSpace(contentType), strings.TrimSpace(acceptedType)) {
accepted = true
break
}
}
if !accepted {
http.Error(w, fmt.Sprintf("Unsupported media type: %s", contentType), http.StatusUnsupportedMediaType)
return
}
next.ServeHTTP(w, r)
})
}
}
// AcceptHeaderHandler checks if the client's response preference is handled
func AcceptHeaderHandler(serverAcceptedTypes []string) Middleware {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
acceptHeader := r.Header.Get("Accept")
// header is optional and should skip if not provided
if acceptHeader == "" {
next.ServeHTTP(w, r)
return
}
accepted := false
acceptTypes := strings.Split(acceptHeader, ",")
// follows rules defined in https://datatracker.ietf.org/doc/html/rfc2616#section-14.1
for _, acceptType := range acceptTypes {
acceptType = strings.TrimSpace(acceptType)
if acceptType == "*/*" {
accepted = true
break
}
for _, serverAcceptedType := range serverAcceptedTypes {
if strings.HasPrefix(acceptType, serverAcceptedType) {
accepted = true
break
}
if acceptType != "/*" && strings.HasSuffix(acceptType, "/*") && strings.HasPrefix(serverAcceptedType, acceptType[:len(acceptType)-2]) {
accepted = true
break
}
}
if accepted {
break
}
}
if !accepted {
http.Error(w, fmt.Sprintf("Not Acceptable: %s", acceptHeader), http.StatusNotAcceptable)
return
}
next.ServeHTTP(w, r)
})
}
}
func MiddlewareChain(h http.Handler, mw []Middleware) http.Handler {
if len(mw) < 1 {
return h
}
wrapped := h
for i := len(mw) - 1; i >= 0; i-- {
wrapped = mw[i](wrapped)
}
return wrapped
}

View File

@@ -0,0 +1,204 @@
package middleware
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/prysmaticlabs/prysm/v5/api"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestNormalizeQueryValuesHandler(t *testing.T) {
nextHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, err := w.Write([]byte("next handler"))
require.NoError(t, err)
})
handler := NormalizeQueryValuesHandler(nextHandler)
tests := []struct {
name string
inputQuery string
expectedQuery string
}{
{
name: "3 values",
inputQuery: "key=value1,value2,value3",
expectedQuery: "key=value1&key=value2&key=value3", // replace with expected normalized value
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
req, err := http.NewRequest("GET", "/test?"+test.inputQuery, http.NoBody)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Errorf("handler returned wrong status code: got %v want %v", rr.Code, http.StatusOK)
}
if req.URL.RawQuery != test.expectedQuery {
t.Errorf("query not normalized: got %v want %v", req.URL.RawQuery, test.expectedQuery)
}
if rr.Body.String() != "next handler" {
t.Errorf("next handler was not executed")
}
})
}
}
func TestContentTypeHandler(t *testing.T) {
acceptedMediaTypes := []string{api.JsonMediaType, api.OctetStreamMediaType}
nextHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, err := w.Write([]byte("next handler"))
require.NoError(t, err)
})
handler := ContentTypeHandler(acceptedMediaTypes)(nextHandler)
tests := []struct {
name string
contentType string
expectedStatusCode int
isGet bool
}{
{
name: "Accepted Content-Type - application/json",
contentType: api.JsonMediaType,
expectedStatusCode: http.StatusOK,
},
{
name: "Accepted Content-Type - ssz format",
contentType: api.OctetStreamMediaType,
expectedStatusCode: http.StatusOK,
},
{
name: "Unsupported Content-Type - text/plain",
contentType: "text/plain",
expectedStatusCode: http.StatusUnsupportedMediaType,
},
{
name: "Missing Content-Type",
contentType: "",
expectedStatusCode: http.StatusUnsupportedMediaType,
},
{
name: "GET request skips content type check",
contentType: "",
expectedStatusCode: http.StatusOK,
isGet: true,
},
{
name: "Content type contains charset is ok",
contentType: "application/json; charset=utf-8",
expectedStatusCode: http.StatusOK,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
httpMethod := http.MethodPost
if tt.isGet {
httpMethod = http.MethodGet
}
req := httptest.NewRequest(httpMethod, "/", nil)
if tt.contentType != "" {
req.Header.Set("Content-Type", tt.contentType)
}
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if status := rr.Code; status != tt.expectedStatusCode {
t.Errorf("handler returned wrong status code: got %v want %v", status, tt.expectedStatusCode)
}
})
}
}
func TestAcceptHeaderHandler(t *testing.T) {
acceptedTypes := []string{"application/json", "application/octet-stream"}
nextHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, err := w.Write([]byte("next handler"))
require.NoError(t, err)
})
handler := AcceptHeaderHandler(acceptedTypes)(nextHandler)
tests := []struct {
name string
acceptHeader string
expectedStatusCode int
}{
{
name: "Accepted Accept-Type - application/json",
acceptHeader: "application/json",
expectedStatusCode: http.StatusOK,
},
{
name: "Accepted Accept-Type - application/octet-stream",
acceptHeader: "application/octet-stream",
expectedStatusCode: http.StatusOK,
},
{
name: "Accepted Accept-Type with parameters",
acceptHeader: "application/json;q=0.9, application/octet-stream;q=0.8",
expectedStatusCode: http.StatusOK,
},
{
name: "Unsupported Accept-Type - text/plain",
acceptHeader: "text/plain",
expectedStatusCode: http.StatusNotAcceptable,
},
{
name: "Missing Accept header",
acceptHeader: "",
expectedStatusCode: http.StatusOK,
},
{
name: "*/* is accepted",
acceptHeader: "*/*",
expectedStatusCode: http.StatusOK,
},
{
name: "application/* is accepted",
acceptHeader: "application/*",
expectedStatusCode: http.StatusOK,
},
{
name: "/* is unsupported",
acceptHeader: "/*",
expectedStatusCode: http.StatusNotAcceptable,
},
{
name: "application/ is unsupported",
acceptHeader: "application/",
expectedStatusCode: http.StatusNotAcceptable,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest("GET", "/", nil)
if tt.acceptHeader != "" {
req.Header.Set("Accept", tt.acceptHeader)
}
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if status := rr.Code; status != tt.expectedStatusCode {
t.Errorf("handler returned wrong status code: got %v want %v", status, tt.expectedStatusCode)
}
})
}
}

View File

@@ -0,0 +1,17 @@
package middleware
import (
"net/url"
"strings"
)
// NormalizeQueryValues replaces comma-separated values with individual values
func NormalizeQueryValues(queryParams url.Values) {
for key, vals := range queryParams {
splitVals := make([]string, 0)
for _, v := range vals {
splitVals = append(splitVals, strings.Split(v, ",")...)
}
queryParams[key] = splitVals
}
}

View File

@@ -0,0 +1,21 @@
package middleware
import (
"testing"
"github.com/prysmaticlabs/prysm/v5/testing/assert"
"github.com/prysmaticlabs/prysm/v5/testing/require"
)
func TestNormalizeQueryValues(t *testing.T) {
input := make(map[string][]string)
input["key"] = []string{"value1", "value2,value3,value4", "value5"}
NormalizeQueryValues(input)
require.Equal(t, 5, len(input["key"]))
assert.Equal(t, "value1", input["key"][0])
assert.Equal(t, "value2", input["key"][1])
assert.Equal(t, "value3", input["key"][2])
assert.Equal(t, "value4", input["key"][3])
assert.Equal(t, "value5", input["key"][4])
}

View File

@@ -0,0 +1,56 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"block.go",
"conversions.go",
"conversions_blob.go",
"conversions_block.go",
"conversions_lightclient.go",
"conversions_state.go",
"endpoints_beacon.go",
"endpoints_blob.go",
"endpoints_builder.go",
"endpoints_config.go",
"endpoints_debug.go",
"endpoints_events.go",
"endpoints_lightclient.go",
"endpoints_node.go",
"endpoints_rewards.go",
"endpoints_validator.go",
"other.go",
"state.go",
],
importpath = "github.com/prysmaticlabs/prysm/v5/api/server/structs",
visibility = ["//visibility:public"],
deps = [
"//api/server:go_default_library",
"//beacon-chain/state:go_default_library",
"//config/fieldparams:go_default_library",
"//consensus-types/interfaces:go_default_library",
"//consensus-types/primitives:go_default_library",
"//consensus-types/validator:go_default_library",
"//container/slice:go_default_library",
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/eth/v1:go_default_library",
"//proto/eth/v2:go_default_library",
"//proto/migration:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"@com_github_ethereum_go_ethereum//common:go_default_library",
"@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)
go_test(
name = "go_default_test",
srcs = ["conversions_test.go"],
embed = [":go_default_library"],
deps = [
"//proto/prysm/v1alpha1:go_default_library",
"//testing/require:go_default_library",
],
)

547
api/server/structs/block.go Normal file
View File

@@ -0,0 +1,547 @@
package structs
import "encoding/json"
// MessageJsoner describes a signed consensus type wrapper that can return the `.Message` field in a json envelope
// encoded as a []byte, for use as a json.RawMessage value when encoding the outer envelope.
type MessageJsoner interface {
MessageRawJson() ([]byte, error)
}
// SignedMessageJsoner embeds MessageJsoner and adds a method to also retrieve the Signature field as a string.
type SignedMessageJsoner interface {
MessageJsoner
SigString() string
}
type SignedBeaconBlock struct {
Message *BeaconBlock `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlock{}
func (s *SignedBeaconBlock) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlock) SigString() string {
return s.Signature
}
type BeaconBlock struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBody `json:"body"`
}
type BeaconBlockBody struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
}
type SignedBeaconBlockAltair struct {
Message *BeaconBlockAltair `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlockAltair{}
func (s *SignedBeaconBlockAltair) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlockAltair) SigString() string {
return s.Signature
}
type BeaconBlockAltair struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBodyAltair `json:"body"`
}
type BeaconBlockBodyAltair struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
}
type SignedBeaconBlockBellatrix struct {
Message *BeaconBlockBellatrix `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlockBellatrix{}
func (s *SignedBeaconBlockBellatrix) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlockBellatrix) SigString() string {
return s.Signature
}
type BeaconBlockBellatrix struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBodyBellatrix `json:"body"`
}
type BeaconBlockBodyBellatrix struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayload *ExecutionPayload `json:"execution_payload"`
}
type SignedBlindedBeaconBlockBellatrix struct {
Message *BlindedBeaconBlockBellatrix `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBlindedBeaconBlockBellatrix{}
func (s *SignedBlindedBeaconBlockBellatrix) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBlindedBeaconBlockBellatrix) SigString() string {
return s.Signature
}
type BlindedBeaconBlockBellatrix struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BlindedBeaconBlockBodyBellatrix `json:"body"`
}
type BlindedBeaconBlockBodyBellatrix struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeader `json:"execution_payload_header"`
}
type SignedBeaconBlockCapella struct {
Message *BeaconBlockCapella `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlockCapella{}
func (s *SignedBeaconBlockCapella) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlockCapella) SigString() string {
return s.Signature
}
type BeaconBlockCapella struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBodyCapella `json:"body"`
}
type BeaconBlockBodyCapella struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayload *ExecutionPayloadCapella `json:"execution_payload"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
}
type SignedBlindedBeaconBlockCapella struct {
Message *BlindedBeaconBlockCapella `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBlindedBeaconBlockCapella{}
func (s *SignedBlindedBeaconBlockCapella) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBlindedBeaconBlockCapella) SigString() string {
return s.Signature
}
type BlindedBeaconBlockCapella struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BlindedBeaconBlockBodyCapella `json:"body"`
}
type BlindedBeaconBlockBodyCapella struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeaderCapella `json:"execution_payload_header"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
}
type SignedBeaconBlockContentsDeneb struct {
SignedBlock *SignedBeaconBlockDeneb `json:"signed_block"`
KzgProofs []string `json:"kzg_proofs"`
Blobs []string `json:"blobs"`
}
type BeaconBlockContentsDeneb struct {
Block *BeaconBlockDeneb `json:"block"`
KzgProofs []string `json:"kzg_proofs"`
Blobs []string `json:"blobs"`
}
type SignedBeaconBlockDeneb struct {
Message *BeaconBlockDeneb `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlockDeneb{}
func (s *SignedBeaconBlockDeneb) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlockDeneb) SigString() string {
return s.Signature
}
type BeaconBlockDeneb struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBodyDeneb `json:"body"`
}
type BeaconBlockBodyDeneb struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayload *ExecutionPayloadDeneb `json:"execution_payload"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
BlobKzgCommitments []string `json:"blob_kzg_commitments"`
}
type BlindedBeaconBlockDeneb struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BlindedBeaconBlockBodyDeneb `json:"body"`
}
type SignedBlindedBeaconBlockDeneb struct {
Message *BlindedBeaconBlockDeneb `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBlindedBeaconBlockDeneb{}
func (s *SignedBlindedBeaconBlockDeneb) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBlindedBeaconBlockDeneb) SigString() string {
return s.Signature
}
type BlindedBeaconBlockBodyDeneb struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashing `json:"attester_slashings"`
Attestations []*Attestation `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeaderDeneb `json:"execution_payload_header"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
BlobKzgCommitments []string `json:"blob_kzg_commitments"`
}
type SignedBeaconBlockContentsElectra struct {
SignedBlock *SignedBeaconBlockElectra `json:"signed_block"`
KzgProofs []string `json:"kzg_proofs"`
Blobs []string `json:"blobs"`
}
type BeaconBlockContentsElectra struct {
Block *BeaconBlockElectra `json:"block"`
KzgProofs []string `json:"kzg_proofs"`
Blobs []string `json:"blobs"`
}
type SignedBeaconBlockElectra struct {
Message *BeaconBlockElectra `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBeaconBlockElectra{}
func (s *SignedBeaconBlockElectra) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBeaconBlockElectra) SigString() string {
return s.Signature
}
type BeaconBlockElectra struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BeaconBlockBodyElectra `json:"body"`
}
type BeaconBlockBodyElectra struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashingElectra `json:"attester_slashings"`
Attestations []*AttestationElectra `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayload *ExecutionPayloadElectra `json:"execution_payload"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
BlobKzgCommitments []string `json:"blob_kzg_commitments"`
ExecutionRequests *ExecutionRequests `json:"execution_requests"`
}
type BlindedBeaconBlockElectra struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
Body *BlindedBeaconBlockBodyElectra `json:"body"`
}
type SignedBlindedBeaconBlockElectra struct {
Message *BlindedBeaconBlockElectra `json:"message"`
Signature string `json:"signature"`
}
var _ SignedMessageJsoner = &SignedBlindedBeaconBlockElectra{}
func (s *SignedBlindedBeaconBlockElectra) MessageRawJson() ([]byte, error) {
return json.Marshal(s.Message)
}
func (s *SignedBlindedBeaconBlockElectra) SigString() string {
return s.Signature
}
type BlindedBeaconBlockBodyElectra struct {
RandaoReveal string `json:"randao_reveal"`
Eth1Data *Eth1Data `json:"eth1_data"`
Graffiti string `json:"graffiti"`
ProposerSlashings []*ProposerSlashing `json:"proposer_slashings"`
AttesterSlashings []*AttesterSlashingElectra `json:"attester_slashings"`
Attestations []*AttestationElectra `json:"attestations"`
Deposits []*Deposit `json:"deposits"`
VoluntaryExits []*SignedVoluntaryExit `json:"voluntary_exits"`
SyncAggregate *SyncAggregate `json:"sync_aggregate"`
ExecutionPayloadHeader *ExecutionPayloadHeaderElectra `json:"execution_payload_header"`
BLSToExecutionChanges []*SignedBLSToExecutionChange `json:"bls_to_execution_changes"`
BlobKzgCommitments []string `json:"blob_kzg_commitments"`
ExecutionRequests *ExecutionRequests `json:"execution_requests"`
}
type SignedBeaconBlockHeaderContainer struct {
Header *SignedBeaconBlockHeader `json:"header"`
Root string `json:"root"`
Canonical bool `json:"canonical"`
}
type SignedBeaconBlockHeader struct {
Message *BeaconBlockHeader `json:"message"`
Signature string `json:"signature"`
}
type BeaconBlockHeader struct {
Slot string `json:"slot"`
ProposerIndex string `json:"proposer_index"`
ParentRoot string `json:"parent_root"`
StateRoot string `json:"state_root"`
BodyRoot string `json:"body_root"`
}
type ExecutionPayload struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
Transactions []string `json:"transactions"`
}
type ExecutionPayloadHeader struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
TransactionsRoot string `json:"transactions_root"`
}
type ExecutionPayloadCapella struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
Transactions []string `json:"transactions"`
Withdrawals []*Withdrawal `json:"withdrawals"`
}
type ExecutionPayloadHeaderCapella struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
TransactionsRoot string `json:"transactions_root"`
WithdrawalsRoot string `json:"withdrawals_root"`
}
type ExecutionPayloadDeneb struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
Transactions []string `json:"transactions"`
Withdrawals []*Withdrawal `json:"withdrawals"`
BlobGasUsed string `json:"blob_gas_used"`
ExcessBlobGas string `json:"excess_blob_gas"`
}
type ExecutionPayloadElectra = ExecutionPayloadDeneb
type ExecutionPayloadHeaderDeneb struct {
ParentHash string `json:"parent_hash"`
FeeRecipient string `json:"fee_recipient"`
StateRoot string `json:"state_root"`
ReceiptsRoot string `json:"receipts_root"`
LogsBloom string `json:"logs_bloom"`
PrevRandao string `json:"prev_randao"`
BlockNumber string `json:"block_number"`
GasLimit string `json:"gas_limit"`
GasUsed string `json:"gas_used"`
Timestamp string `json:"timestamp"`
ExtraData string `json:"extra_data"`
BaseFeePerGas string `json:"base_fee_per_gas"`
BlockHash string `json:"block_hash"`
TransactionsRoot string `json:"transactions_root"`
WithdrawalsRoot string `json:"withdrawals_root"`
BlobGasUsed string `json:"blob_gas_used"`
ExcessBlobGas string `json:"excess_blob_gas"`
}
type ExecutionPayloadHeaderElectra = ExecutionPayloadHeaderDeneb
type ExecutionRequests struct {
Deposits []*DepositRequest `json:"deposits"`
Withdrawals []*WithdrawalRequest `json:"withdrawals"`
Consolidations []*ConsolidationRequest `json:"consolidations"`
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,61 @@
package structs
import (
"strconv"
"github.com/prysmaticlabs/prysm/v5/api/server"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
eth "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
)
func (sc *Sidecar) ToConsensus() (*eth.BlobSidecar, error) {
if sc == nil {
return nil, errNilValue
}
index, err := strconv.ParseUint(sc.Index, 10, 64)
if err != nil {
return nil, server.NewDecodeError(err, "Index")
}
blob, err := bytesutil.DecodeHexWithLength(sc.Blob, 131072)
if err != nil {
return nil, server.NewDecodeError(err, "Blob")
}
kzgCommitment, err := bytesutil.DecodeHexWithLength(sc.KzgCommitment, 48)
if err != nil {
return nil, server.NewDecodeError(err, "KzgCommitment")
}
kzgProof, err := bytesutil.DecodeHexWithLength(sc.KzgProof, 48)
if err != nil {
return nil, server.NewDecodeError(err, "KzgProof")
}
header, err := sc.SignedBeaconBlockHeader.ToConsensus()
if err != nil {
return nil, server.NewDecodeError(err, "SignedBeaconBlockHeader")
}
// decode the commitment inclusion proof
var commitmentInclusionProof [][]byte
for _, proof := range sc.CommitmentInclusionProof {
proofBytes, err := bytesutil.DecodeHexWithLength(proof, 32)
if err != nil {
return nil, server.NewDecodeError(err, "CommitmentInclusionProof")
}
commitmentInclusionProof = append(commitmentInclusionProof, proofBytes)
}
bsc := &eth.BlobSidecar{
Index: index,
Blob: blob,
KzgCommitment: kzgCommitment,
KzgProof: kzgProof,
SignedBlockHeader: header,
CommitmentInclusionProof: commitmentInclusionProof,
}
return bsc, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,127 @@
package structs
import (
"encoding/json"
"fmt"
"strconv"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/pkg/errors"
v1 "github.com/prysmaticlabs/prysm/v5/proto/eth/v1"
v2 "github.com/prysmaticlabs/prysm/v5/proto/eth/v2"
"github.com/prysmaticlabs/prysm/v5/proto/migration"
)
func LightClientUpdateFromConsensus(update *v2.LightClientUpdate) (*LightClientUpdate, error) {
attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
if err != nil {
return nil, errors.Wrap(err, "could not marshal attested light client header")
}
finalizedHeader, err := lightClientHeaderContainerToJSON(update.FinalizedHeader)
if err != nil {
return nil, errors.Wrap(err, "could not marshal finalized light client header")
}
return &LightClientUpdate{
AttestedHeader: attestedHeader,
NextSyncCommittee: SyncCommitteeFromConsensus(migration.V2SyncCommitteeToV1Alpha1(update.NextSyncCommittee)),
NextSyncCommitteeBranch: branchToJSON(update.NextSyncCommitteeBranch),
FinalizedHeader: finalizedHeader,
FinalityBranch: branchToJSON(update.FinalityBranch),
SyncAggregate: syncAggregateToJSON(update.SyncAggregate),
SignatureSlot: strconv.FormatUint(uint64(update.SignatureSlot), 10),
}, nil
}
func LightClientFinalityUpdateFromConsensus(update *v2.LightClientFinalityUpdate) (*LightClientFinalityUpdate, error) {
attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
if err != nil {
return nil, errors.Wrap(err, "could not marshal attested light client header")
}
finalizedHeader, err := lightClientHeaderContainerToJSON(update.FinalizedHeader)
if err != nil {
return nil, errors.Wrap(err, "could not marshal finalized light client header")
}
return &LightClientFinalityUpdate{
AttestedHeader: attestedHeader,
FinalizedHeader: finalizedHeader,
FinalityBranch: branchToJSON(update.FinalityBranch),
SyncAggregate: syncAggregateToJSON(update.SyncAggregate),
SignatureSlot: strconv.FormatUint(uint64(update.SignatureSlot), 10),
}, nil
}
func LightClientOptimisticUpdateFromConsensus(update *v2.LightClientOptimisticUpdate) (*LightClientOptimisticUpdate, error) {
attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
if err != nil {
return nil, errors.Wrap(err, "could not marshal attested light client header")
}
return &LightClientOptimisticUpdate{
AttestedHeader: attestedHeader,
SyncAggregate: syncAggregateToJSON(update.SyncAggregate),
SignatureSlot: strconv.FormatUint(uint64(update.SignatureSlot), 10),
}, nil
}
func branchToJSON(branchBytes [][]byte) []string {
if branchBytes == nil {
return nil
}
branch := make([]string, len(branchBytes))
for i, root := range branchBytes {
branch[i] = hexutil.Encode(root)
}
return branch
}
func syncAggregateToJSON(input *v1.SyncAggregate) *SyncAggregate {
return &SyncAggregate{
SyncCommitteeBits: hexutil.Encode(input.SyncCommitteeBits),
SyncCommitteeSignature: hexutil.Encode(input.SyncCommitteeSignature),
}
}
func lightClientHeaderContainerToJSON(container *v2.LightClientHeaderContainer) (json.RawMessage, error) {
// In the case that a finalizedHeader is nil.
if container == nil {
return nil, nil
}
beacon, err := container.GetBeacon()
if err != nil {
return nil, errors.Wrap(err, "could not get beacon block header")
}
var header any
switch t := (container.Header).(type) {
case *v2.LightClientHeaderContainer_HeaderAltair:
header = &LightClientHeader{Beacon: BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon))}
case *v2.LightClientHeaderContainer_HeaderCapella:
execution, err := ExecutionPayloadHeaderCapellaFromConsensus(t.HeaderCapella.Execution)
if err != nil {
return nil, err
}
header = &LightClientHeaderCapella{
Beacon: BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon)),
Execution: execution,
ExecutionBranch: branchToJSON(t.HeaderCapella.ExecutionBranch),
}
case *v2.LightClientHeaderContainer_HeaderDeneb:
execution, err := ExecutionPayloadHeaderDenebFromConsensus(t.HeaderDeneb.Execution)
if err != nil {
return nil, err
}
header = &LightClientHeaderDeneb{
Beacon: BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon)),
Execution: execution,
ExecutionBranch: branchToJSON(t.HeaderDeneb.ExecutionBranch),
}
default:
return nil, fmt.Errorf("unsupported header type %T", t)
}
return json.Marshal(header)
}

Some files were not shown because too many files have changed in this diff Show More