Commit Graph

5224 Commits

Author SHA1 Message Date
Nishant Das
508c5fcf2f More Efficient Validation of Proposer Index (#8107)
* metric

* make it better

* make it better

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-15 16:27:05 +00:00
Victor Farazdagi
a0c475671c Add extra methods to shared/fileutil (#8117)
* add extra methods to fileutil

* Shay's suggestion

* Update shared/fileutil/fileutil.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* Adds reference to the original implementation

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-15 14:18:15 +00:00
Preston Van Loon
72a92fe708 CI: Disable remote caching of GoStdLib (#8111)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 05:58:52 +00:00
terence tsao
6a5589f99e Skip proposer indices cache update if exists (#8096)
* Check if cache is empty before update

* Add tests

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 00:09:30 +00:00
terence tsao
70c0bb106b Beacon node code health improvements (#8109)
* Apply code health fixes after code inspect

* Remove attestationRoot

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 23:41:24 +00:00
Victor Farazdagi
0f18867f08 Remove duplicate package imports (#8104)
* remove duplicate imports

* remove unused lookupLimit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-14 22:22:55 +00:00
Cipio
630d57377a Fix Slasher Backup DB panic on call. (#8099) 2020-12-14 21:39:26 +00:00
terence tsao
3e9d721280 Validator code health improvements (#8106) 2020-12-14 21:10:02 +00:00
Steven Allen
2428880058 Update go-libp2p to 0.12.0 (#8015)
* Update go-libp2p to 0.12.0

go-libp2p 0.12.0 made some significant changes to the stream interfaces around
stream closing:

* Close now closes in both directions and frees the stream. However, unlike
FullClose did, it doesn't _wait_ for the remote peer to respond with an EOF.
* To close for writing, call CloseWrite (like one would on a TCP connection, etc.).

This patch:

* Replaces calls to FullClose with Close where appropriate.
* Replaces calls to Close with CloseWrite where appropriate.
* Removes redundant Close calls.
* Calls Reset to where appropriate to indicate that the request/response was
  aborted. Unlike Close, this will not flush and will not cause the remote peer
  to read an EOF. Instead, the remote peer will read an ErrReset error.
* Ensures we always either close or reset streams. Send wasn't closing the
  stream on some error paths.
* Now that stream closing is async, we explicitly wait for a response when
  "hanging up" on a peer (so we don't hang up before they receive our
  response/goodbye message).

* update bazel

* Gazelle

* revert unintentional bazel workspace change

* appease an overzealous linter

* update to latest

* Refactor encoder

* gazelle

* Gazelle

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
v1.0.5
2020-12-14 17:22:25 +00:00
Potuz
4d1f01aacc Change block_arrival_latency buckets to exponential format (#8065)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-14 16:00:07 +00:00
Victor Farazdagi
b9848dc94f Remove unused vars in tests (#8103)
* update process_block_test

* service_attester_test

* update service and rpc_status tests

* go fmt
2020-12-13 05:23:13 +00:00
Nishant Das
579335f81a Remove Saving Of Target And Source Epoch (#8102)
* remove

* skip
2020-12-13 04:26:18 +00:00
Nishant Das
11bbf06d03 Add Inbound Peer Limit (#7942)
* add changes

* fix up

* fix

* add test

* fix test

* fix again

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-12 03:46:55 +00:00
Nishant Das
29804fa572 Refactor Subnet Search (#8048)
* checkpoint progress

* checkpoint

* clean up

* do better

* fix test

* fix

* fix

* preston's review

* fix

* fix

* fix

* add iterator

* go doc

* make it a config parameter

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 18:40:56 -08:00
Radosław Kapka
4ec396c025 Refetch validating keys if no keys are fetched (#8000)
* refetch validating keys every 30 seconds

* deduplicate error/log messages

* remove redundant break statement

* comment about execution flow

* move code to wait_for_activation.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 20:55:52 +01:00
Raul Jordan
1fbfd52e52 Simpler and Safer Attester Slashing Protection (#8086)
* att sign validations

* eliminate old cached methods and use a simple approach in the db

* redefined db methods

* db package builds

* add multilock to attest and propose

* gaz

* removed concurrency tests that are no longer relevant

* add cache to db functions for attesting history checks

* passing

* add in feature flag --disable-attesting-history-db-cache

* remove lock

* Revert "remove lock"

This reverts commit b1a65020e4.

* comment

* gaz
2020-12-11 12:31:35 -06:00
Mohamed Mansour
2e18df642d Expand Validator Input Errors (#8090)
The reason of why a validator input was failing was very generic,
this allows the actual implementation to state why it failed by
passing the exception forward.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-11 10:57:47 -06:00
Nishant Das
99b3835f19 Add Fallback Option for Eth1 Nodes (#8062)
* fix

* fix tests and change back

* gaz

* change

* ready again

* Update beacon-chain/flags/base.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* radek's review

* Update shared/cmd/helpers.go

* Update shared/cmd/helpers_test.go

* Update shared/cmd/helpers_test.go

* Endpoint/endpoint

Co-authored-by: Shay Zluf <thezluf@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-12-11 19:15:04 +08:00
Victor Farazdagi
46d99fdc00 Attestation aggregation: remove proper set duplicates (#8063)
* fix proper-set issue

* move test into dedicated test file

* move functionality from maxcover -> proposer

* adds dedup to proposer

* fix tests

* update tests

* remove redundant proper subset test

* fix bug with identical expression
2020-12-11 11:13:14 +03:00
Preston Van Loon
923e4d3a5e Attempt to reconnect when waiting for activation (#8057)
* Attempt to reconnect when waiting for activation

* rm fuzz/attestation_fuzz.go

* math.Min, not math.Max

* Gofmt

* resolve RVV-B0003
2020-12-10 19:26:31 -06:00
terence tsao
bb9e2ba12c Validator client logs time left to next duty (#8088) 2020-12-10 10:56:18 -08:00
Preston Van Loon
f44b2a35e4 Update waiting validator methods to be context aware (#8078)
* Update waiting validator methods to be context aware

* Enable debug logging for validator in e2e

* invert logic in deferred function, remove for loop

* return early if wait is 0 or less

* overwrite ctx

* t.Stop() chan closure is hanging. Doesnt make sense, so ignoring that cleanup task. It shouldnt happen at runtime anyway

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-10 10:00:48 -06:00
terence tsao
00dacbd00d Remove custom block body root and block root methods (#8069)
* Remove custom block body root and block root methods

* Add nil checks and fix tests

* Fmt

* Typo
2020-12-09 12:11:42 -06:00
Radosław Kapka
92736d0188 warn when creating an imported wallet (#8081) 2020-12-09 16:41:26 +01:00
Preston Van Loon
c96db1a122 Add spans to pre and post att signing updates (#8079)
* Add spans to pre and post att signing updates

* span on db method

* Add span to isNewAttSlashable
2020-12-08 23:47:02 -06:00
terence tsao
c5770a2e56 Add beacon block nil body checks (#8077)
* Add nil checks and fix tests

* Gaz

* Revert one minor typo

* Update test

* Remove extra space
2020-12-08 16:27:58 -06:00
terence tsao
ade3b2f2df Add state field count to config (#8068)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-07 21:49:28 -06:00
Radosław Kapka
9d7052796b Restore database CLI command (#8061)
* restore beacon node db

* revert image name

* move restore out of the kv folder

* remove files from kv folder

* go mod tidy

* Remove usage of prometheus testutil

* add yes/no to prompt text

* restore slasher db

* organize imports

* go mod tidy

* restore validator db

* close slasher db

* defer close backup db in tests

* simplify function literal
2020-12-07 21:36:43 +01:00
Shay Zluf
fbbdd94fea Resolve panic on shutdown with offline eth1 node (#8033)
* Resolve panic on shutdown with offline eth1 node

* Move fix tne right place nishant feedback

* fix log message

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-07 10:16:23 +00:00
Victor Farazdagi
b51aec6981 Init sync: minor fixes (#8060)
* fixes typo

* simplify code
2020-12-07 09:33:06 +00:00
terence tsao
b4437e6cec Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
v1.0.4
2020-12-04 23:15:12 +00:00
Arthur Burkart
8ad328d9b3 fix(grpcHeaders): accept values with "=" symbols (#8047)
* fix(grpcHeaders): accept values with equal signs

# What

Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.

Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.

# Testing

This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:

```go
package main

import (
	"log"
	"os"
	"strings"

	"github.com/davecgh/go-spew/spew"
	"github.com/urfave/cli/v2"
)

type Config struct {
	GrpcHeadersFlag string
}

func main() {
	app := &cli.App{
		Action: func(c *cli.Context) error {
			for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
				if hdr != "" {
					ss := strings.Split(hdr, "=")
					spew.Dump(ss[0])
					spew.Dump(strings.Join(ss[1:], "="))
				}
			}
			return nil
		},
		Flags: []cli.Flag{
			&cli.StringFlag{
				Name: "grpc-headers",
				Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
					"calls. Example: --grpc-headers=key=value",
			},
		},
	}

	err := app.Run(os.Args)
	if err != nil {
		log.Fatal(err)
	}
}
```

Example invocation:

```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```

* Adds tests to new gRPC header parsing code

* bazel run //:gazelle
2020-12-04 21:31:15 +00:00
Victor Farazdagi
21ede7634e Attestation aggregation: maxcover vs naive aggregation effectiveness test (#8043)
* maxcover performance test

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 17:24:57 +00:00
Victor Farazdagi
57b74283d3 Attestation aggregation: removes redundant code from max-cover benchmarks (#8044)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 16:51:31 +00:00
Victor Farazdagi
be078d6a16 Update TestMain(): do not call os.Exit() explicitly (#8046)
* update workspace

* update testmain
2020-12-04 16:10:07 +00:00
Victor Farazdagi
ccba8cfa5a Update rules_go to v0.24.9 and golang to v1.15.6 (#8045) 2020-12-04 15:28:03 +00:00
Victor Farazdagi
afbfaedea4 Unskip fixed init-sync service test (#8042) 2020-12-04 11:57:46 +00:00
terence tsao
3ce96701de Update attestation schedule log with total attester count (#8013)
* Log attesting total

* Use the right library

* Go fmt

* Use fmt

* No space is better

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 07:02:58 +00:00
simonatsn
d2ba45aad9 Update blst to v0.3.1 and incorporate subgroup changes (#7971)
* Update blst to v0.3.1 and incorporate subgroup changes

* go mod tidy

* gofmt

* Update bzl blst dependency

* Remove unnecessary check for nil

* Run bazel run //:gazelle -- fix

* Update blst to v0.3.2

* fix sha

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-04 06:46:08 +00:00
Raul Jordan
14e1f08208 Add Backup Webhooks to All Prysm Services With DBs (#8025)
* integrate backup webhooks

* pass slasher tests

* fix node

* cmd

* gaz

* read test passes

* radek feedback

* added comment

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-03 22:28:57 +00:00
Victor Farazdagi
647b4cf108 Round robin: half open interval in syncToFinalizedEpoch (#8039) 2020-12-03 21:55:42 +00:00
terence tsao
c090c6a1c5 Revert "Logging with PadLevelText to true" (#8036)
This reverts commit c51754fa8a.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 20:38:00 +00:00
Victor Farazdagi
7dd0c24fea Skip bogus init-sync test (#8038) 2020-12-03 20:20:04 +00:00
Victor Farazdagi
e1755b6066 Invert enable-sync-backtracking (#8034)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 15:43:26 +00:00
Victor Farazdagi
3092f75ec2 Init sync: conditional syncing to finalized slot (#7999)
* extract sync methods

* add unit test

* gazelle

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* better set back step

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-03 14:10:51 +00:00
Shay Zluf
821620c520 Add test for local protection genesis attestation (#7977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 01:02:51 +00:00
Victor Farazdagi
5417e8cf31 Init-sync: enable peer scorer by default (#7974)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:45:20 +00:00
Radosław Kapka
323769bf1a Make TLS connections to a remote wallet non-mandatory (#7953)
* disable-remote-signer-tls flag

* use flag in edit-config

* send requests without TLS

* change warning message

* fix account list output test

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:18:15 +00:00
terence tsao
c51754fa8a Logging with PadLevelText to true (#8003)
* Use logrus formatter with pad

* Validator

* Fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 23:49:13 +00:00
terence tsao
2153a2d7c3 Remove logging deposit inclusion slot (#8023)
* Remove logging deposit inclusion slot

* Remove old tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-02 21:17:47 +00:00