**What type of PR is this?**
Feature
What does this PR do? Why is it needed?
This PR takes @MarcoPolo 's PR at
https://github.com/OffchainLabs/prysm/pull/16130 to completion with
tests.
The description on his PR:
"""
a relatively small change to optimize network send order.
Without this, network writes tend to prioritize sending data for one
column to all peers before sending data for later columns (e.g for two
columns and 4 peers per column it would send A,A,A,A,B,B,B,B). With
batch publishing we can change the write order to round robin across
columns (e.g. A,B,A,B,A,B,A,B).
In cases where the process is sending at a rate over the network limit,
this approach allows at least some copies of the column to propagate
through the network. In early simulations with bandwidth limits of
50mbps for the publisher, this improved dissemination by ~20-30%.
"""
See the issue for some more context.
**Which issues(s) does this PR fix?**
Fixes https://github.com/OffchainLabs/prysm/issues/16129
Other notes for review
Acknowledgements
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [ ] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [ ] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
---------
Co-authored-by: Marco Munizaga <git@marcopolo.io>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
* adding compare beacon block test
* fixing bazel
* fixing evaluator import
* fixing imports
* changing package name
* fixing bazel
* adding logic to check for checking epoch
* fixing linting
* adding check for attester duties
* handle both blockv1 and blockv2
* making middleware objects public instead
* adding test for block attestations
* fixing typo
* adding blockroot test
* adding test for attestations
* fixing type value
* fixing test
* adding in node endpoints
* fixing bazel
* updating web3signer
* printing beacon blocks on request
* fixing struct
* temp log
* forgot string cast
* adding comparison function
* fixing bazel and evaulators, WIP
* fixing bazel
* changing how to minify json
* trying multiclient
* fixing port problem
* reverting evaluator and making test only for mainnet scenario testing
* removing test data
* fixing linting unused functions
git push
* changed to reflect
* adding in ssz comparison
* fixing tests
* fixing conflict
* fixing tests
* making v2 the standard
* adding better error logging
* fixing type
* adding lighthouse settings and fixing some deepsource items
* testing adding delay to evaluator
* testing without peers check
* changing target peers to try to fix lighthouse peer connections
* temp removing other tests
* fix lint issue
* adding peers connect back in
* adding in state version
* fixing bazel
* fixing path error
* testing changes to state
* fix unmarshal
* simplifying beacon api e2e execution
* fixing missed assertian checks
* improve logging and debugging issue
* trying to fix unmarshal
* still breaking more test edits
* removing fork to test unmarshal
* fixing pathing
* resolving error
* fixing beacon_api
* merging in debug api to beacon_api test
* fixing lint and temp commenting out endpoint
* adding in custom comparison function
* fixing custom evaluator
* adding test for block header data
* fixing header evaluation
* add node apis
* fixing linting,adding tests
* fixing bazel and temp removing unused functions
* fixing deepsource and linting issues
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* addressing review
* resolving more review comments
* fixing linting
* removing ssz return value as it's large and possibly not needed
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* fixing more review comments
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* fixing linting and review iteems
* fixing cognit complexity issue
* fixing linting
* fix log printout
* test build kite only with crossclient
* switching out evaluator to depositedvalidatorsareactive
* removed wrong evaluator switching correct one
* removing skip based on review comments
* fixing pathing issue
* test without participation at epoch
* testing without special lighthouse logic in evaluator
* reducing expected participation when multiclient
* fixing imports
* reducing epochs to see if less flaky
* testing with other tests added back in
* reducing epochs ran further
* testing only cross client again
* testing multi run again
* test reverted scenario for tests
* testing with cross client
* removing commented out function
* testing without peers connect
* adding optimization based on suggestions
* removed the wrong peers connect
* accidently commited something I shouldn't have
* fixing lighthouse flag
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update testing/endtoend/evaluators/beaconapi_evaluators/beacon_api.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* s
* s
* typo
* typo
* s
* s
* fixes based on PR feedback
* PR feedback
* reverting log changes
* adding flag per feedback
* conventions
* main fixes
* Update cmd/beacon-chain/jwt/jwt.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/beacon-chain/jwt/jwt.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* s
* tests
* test attempt
* test
* Update cmd/beacon-chain/jwt/jwt.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* err fix
* s
* further simplify
* cleanup
* namefix
* tests pass
* gaz
* rem deadcode
* Gaz
* shorthand
* naming
* test pass
* dedup
* success
* Ignore jwt.hex file
* logrus
* feedback
* junk
* Also check that no file was written
* local run config
* small fix
* jwt
* testfix
* s
* disabling test
* reverting main changes
* main revert
* removing temp folder
* comment
* gaz
* clarity
* rem
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Add AFL third_party libraries
* add beacon state fuzzing, add afl fuzz bundle
* rm fuzzing engine
* fix and lint
* Check for array out of bounds when calculating proposer delta
* failing test
* fix
* Checkpoint progress
* Add requirement that inclusion distance is not zero, add regression test
* No need for HTR since that is covered in process slots
* Removing some fuzzit logic, old fuzz tests
* Add ssz encoder test and fix
* Fuzzing checkpoint, adding fuzzing to the p2p layer
* ignore some libfuzzer files
* Full testing of p2p processing of blocks, with some mocked stuff
* use tmpdir and always process blocks
* use checkptr
* Update ethereumapis
* go mod tidy
* benchmarks for ferran's fast ssz hash tree root
* Update fastssz
* fmt
* gaz
* goimports
* Fix
* fix ethereumapis
* fix again
* kafka
* fix gen file
* fix compute signing root
* gofmt
* checkpoint progress
* progress
* checkpoint
* updates
* updates
* merge fix
* WIP
* merge
* fix build
* fix merge related issues
* cleanup
* revert unrelated
* lint
* lint
* lint
* manual tags for fuzz
* Commentary on upload script
* some import fixes, but not all
* fix //fuzz:fuzz_tests
* rm unused test
* update generated ssz
* Set // +build libfuzzer
* remove debug code
* A bit of refactoring ot explain why there is a committee_disabled file
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix a few deps to work with go.mod, check in generated files
* Update Gossipsub to 1.1 (#5998)
* update libs
* add new validators
* add new deps
* new set of deps
* tls
* further fix gossip update
* get everything to build
* clean up
* gaz
* fix build
* fix all tests
* add deps to images
* imports
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Beacon chain builds with go build
* fix bazel
* fix dep
* lint
* Add github action for testing go
* on PR for any branch
* fix libp2p test failure
* Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test
* Revert "Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test"
This reverts commit 43676894ab.
* Compute and set proposer index instead of hard code
* Add back go mod/sum, fix deps
* go build ./...
* Temporarily skip two tests
* Fix kafka confluent patch
* Fix kafka confluent patch
* fix kafka build
* fix kafka
* Add info in DEPENDENCIES. Added a stub link for Why Bazel? until https://github.com/prysmaticlabs/documentation/issues/138
* Update fuzz ssz files as well
* Update fuzz ssz files as well
* getting closer
* rollback rules_go and gazelle
* fix gogo protobuf
* install librdkafka-dev as part of github actions
* Update kafka to a recent version where librkafkfa is not required for go modules
* clarify comment
* fix kafka build
* disable go tests
* comment
* Fix geth dependencies for end to end
* rename word
* lint
* fix docker
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Remove most of the remaining geth code and set up bazel for this
* chmod +x
* Add flake check
* better flake detection
Former-commit-id: 5c332ecbf2923943f646f1fe40befa95be883329 [formerly 99590fc354514584700e5ce8d7d30a8a7d541f29]
Former-commit-id: e5f919b553fe698e98090965d34eb721990b5693
Events with just 1 argument fail before this change
Former-commit-id: 69cbb51c56054208417e464dc6e46a94ccb3bbeb [formerly fbb03244929beb0ce8f9c607ce33d107e8319b89]
Former-commit-id: 984154c1ba79f1fe8bf0106a604b2765ea312079
