Commit Graph

484 Commits

Author SHA1 Message Date
terence tsao
756ccbe5e4 Update Attestation schedule... log for clarify (#8148)
* Don't use fraction, use different fields

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 19:12:41 +00:00
terence tsao
3e9d721280 Validator code health improvements (#8106) 2020-12-14 21:10:02 +00:00
Nishant Das
579335f81a Remove Saving Of Target And Source Epoch (#8102)
* remove

* skip
2020-12-13 04:26:18 +00:00
Radosław Kapka
4ec396c025 Refetch validating keys if no keys are fetched (#8000)
* refetch validating keys every 30 seconds

* deduplicate error/log messages

* remove redundant break statement

* comment about execution flow

* move code to wait_for_activation.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 20:55:52 +01:00
Raul Jordan
1fbfd52e52 Simpler and Safer Attester Slashing Protection (#8086)
* att sign validations

* eliminate old cached methods and use a simple approach in the db

* redefined db methods

* db package builds

* add multilock to attest and propose

* gaz

* removed concurrency tests that are no longer relevant

* add cache to db functions for attesting history checks

* passing

* add in feature flag --disable-attesting-history-db-cache

* remove lock

* Revert "remove lock"

This reverts commit b1a65020e4.

* comment

* gaz
2020-12-11 12:31:35 -06:00
Preston Van Loon
923e4d3a5e Attempt to reconnect when waiting for activation (#8057)
* Attempt to reconnect when waiting for activation

* rm fuzz/attestation_fuzz.go

* math.Min, not math.Max

* Gofmt

* resolve RVV-B0003
2020-12-10 19:26:31 -06:00
terence tsao
bb9e2ba12c Validator client logs time left to next duty (#8088) 2020-12-10 10:56:18 -08:00
Preston Van Loon
f44b2a35e4 Update waiting validator methods to be context aware (#8078)
* Update waiting validator methods to be context aware

* Enable debug logging for validator in e2e

* invert logic in deferred function, remove for loop

* return early if wait is 0 or less

* overwrite ctx

* t.Stop() chan closure is hanging. Doesnt make sense, so ignoring that cleanup task. It shouldnt happen at runtime anyway

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-10 10:00:48 -06:00
terence tsao
00dacbd00d Remove custom block body root and block root methods (#8069)
* Remove custom block body root and block root methods

* Add nil checks and fix tests

* Fmt

* Typo
2020-12-09 12:11:42 -06:00
Preston Van Loon
c96db1a122 Add spans to pre and post att signing updates (#8079)
* Add spans to pre and post att signing updates

* span on db method

* Add span to isNewAttSlashable
2020-12-08 23:47:02 -06:00
terence tsao
c5770a2e56 Add beacon block nil body checks (#8077)
* Add nil checks and fix tests

* Gaz

* Revert one minor typo

* Update test

* Remove extra space
2020-12-08 16:27:58 -06:00
terence tsao
b4437e6cec Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-04 23:15:12 +00:00
Arthur Burkart
8ad328d9b3 fix(grpcHeaders): accept values with "=" symbols (#8047)
* fix(grpcHeaders): accept values with equal signs

# What

Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.

Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.

# Testing

This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:

```go
package main

import (
	"log"
	"os"
	"strings"

	"github.com/davecgh/go-spew/spew"
	"github.com/urfave/cli/v2"
)

type Config struct {
	GrpcHeadersFlag string
}

func main() {
	app := &cli.App{
		Action: func(c *cli.Context) error {
			for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
				if hdr != "" {
					ss := strings.Split(hdr, "=")
					spew.Dump(ss[0])
					spew.Dump(strings.Join(ss[1:], "="))
				}
			}
			return nil
		},
		Flags: []cli.Flag{
			&cli.StringFlag{
				Name: "grpc-headers",
				Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
					"calls. Example: --grpc-headers=key=value",
			},
		},
	}

	err := app.Run(os.Args)
	if err != nil {
		log.Fatal(err)
	}
}
```

Example invocation:

```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```

* Adds tests to new gRPC header parsing code

* bazel run //:gazelle
2020-12-04 21:31:15 +00:00
terence tsao
3ce96701de Update attestation schedule log with total attester count (#8013)
* Log attesting total

* Use the right library

* Go fmt

* Use fmt

* No space is better

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 07:02:58 +00:00
Shay Zluf
821620c520 Add test for local protection genesis attestation (#7977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 01:02:51 +00:00
terence tsao
2153a2d7c3 Remove logging deposit inclusion slot (#8023)
* Remove logging deposit inclusion slot

* Remove old tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-02 21:17:47 +00:00
Raul Jordan
01bf97293f Fix Miscellaneous Deep Source Issues (#8007)
* deep source fixes

* sh fixes

* fix import

* test err

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-30 19:55:30 -06:00
terence tsao
ea10784a4a Validator logging: return early if no att included (#7979)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-30 23:28:12 +00:00
Raul Jordan
04615cb97b Add Validator RPC Endpoint to Retrieve Beacon + Validator Logs Websocket Endpoints (#7981)
* add logs endpoint

* commands to retrieve logs endpoints

* ensure works at runtime

* add auth
2020-11-27 18:28:45 +00:00
terence tsao
b243665d3e Save lowest source and target epochs in post (#7973)
* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Save lowest epcohs at post signature update

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 19:58:20 +00:00
Raul Jordan
ae8a619775 Allow Optional Signing Roots in Proposal History (#7960) 2020-11-25 22:24:07 +00:00
Raul Jordan
edb40ddea4 Add ValidatorDB Methods for Highest and Lowest Signed Proposals (#7957)
* add in highest and lowest signed proposal

* begin adding tests

* add in db tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 20:04:43 +00:00
Shay Zluf
0cf9800b75 Fix locks and fallback to db read if attestation history map is missing a pub key data (#7937)
* minimal change to handle nil attesterHistoryByPubKey

* Revert "Always Update Attesting History If Not Slashable (#7935)"

This reverts commit 3cc2ebc5d5.

* remove unused functions

* move save before propose

* wait before go func

* move wait into the go routine

* handling map mutation

* remove map handling in this case

* log in case it is still not found

* fix log

* fix locks

* Update validator/client/attest_protect.go

* remove code duplication

* remove method extraction

* move metrics to their appropriate place

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-24 11:24:23 -06:00
Raul Jordan
3cc2ebc5d5 Always Update Attesting History If Not Slashable (#7935)
* update attesting history always if not slashable

* initialize empty if no history found

* rem duplicate logic
2020-11-24 02:48:20 +00:00
Raul Jordan
2cb814648a Improve Slashing Protection for V1, More Tests and Observability (#7934)
* tests tests and more tests

* tests all passsss

* log for double vote
2020-11-23 19:03:04 -06:00
Radosław Kapka
64be627a6d Make grpc-headers flag work (#7932) 2020-11-23 20:38:32 +00:00
Ivan Martinez
5140ceec68 Hotfix for WaitForChainStart GenesisValidatorsRoot Check (#7870)
* Hotfix for genesis val root

* Add regression test

* Fix error message

* Remove comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-20 20:53:12 +00:00
terence tsao
97ad5cd5fd Reduce no attestation in pool to warn (#7863)
* Reduce no attestation in pool to warn

* Use NotFound

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-20 12:17:26 -08:00
Ivan Martinez
4dc65c5787 Save GenesisValidatorsRoot from WaitForChainStart (#7855)
* Add GenValRoot dbs

* Test genvalroot

* Fix names

* Add overwrite rejection

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-20 18:06:12 +00:00
Ivan Martinez
fec469291e Add GenesisValidatorRoot to ChainStartResponse (#7846)
* Add genesis validator root to chainstartresposne

* Deps

* Tidy

* Fix tests

* Fix test

* Fix test and add to ChainStartedData
2020-11-17 20:15:48 -06:00
Ivan Martinez
5889670cc7 Remove WaitForSynced (#7835)
* Remove waitforsynced

* Remove WaitForsynced entirely

* Fix bazel

* tidy
2020-11-16 20:48:16 -06:00
Raul Jordan
7449eba612 Refactor HD Wallets for Enhanced Security (#7821)
* begin hd wallet refactor

* further simplify the new derived keymanager

* make it almost a full wrapper around an imported keymanager

* fix up the EIP test

* deprecated derived

* fixing keymanager tests

* fix up derived tests

* refactor initialize keymanager

* simplify hd

* pass some tests

* pass accounts list test

* gaz

* regenerate protos without create account privilege

* enforce account recovery on wallet create

* allow accounts delete to work

* remove mentions of accounts create

* resolve comments and go mod

* fix up tests

* build fixes

* remove insecure warning

* revert

* fix proto file

* remove create account message

* gaz

* remove account create

* update web api protos

* fix up imports

* change func sig

* tidy

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-16 22:26:04 +00:00
Fabrice Cheng
16bccf05cf [Feature] enable/disable validator accounts (#7746)
* add --enable --disable flags for validator accounts

* refactor DeleteAccountConfig into AccountConfig to be used for enable and disable feature

* add `disable` flag for validator accounts

* [wip] add method to disable account

* refactor account delete

* add disable & enable with proper filters

* fix keymanager unit tests

* update DisabledPublicKeys to be a string instead of [][]byte

* fix FetchValidatingPrivateKeys to only fetch active keys with new string format

* fix FetchValidationPrivateKeys with new DisabledPublicKeys format (as a string)

* rename file + update AccountsConfig to include Disable, Enable and Delete distinct attributes

* rename accounts_activation -> accounts_enable_disable

* revert changes from using string to [][]byte for DisabledPublicKeys

* add FetchAllValidatingPublicKeys to preserve the functionality for accounts list, backup and delete

* fix unit tests

* convert publickeys from [][]byte to str before passing it to pb message

* add unit tests for disable keys

* add unit tests for EnableAccounts

* revert WORKSPACE LLM for now

* ran gazelle

* move function to convert KeymanagerOpts to Config inside rpc and run gazelle

* add unit tests for FetchAllValidatingPublicKeys

* fix keymanageropts for InteropKey

* Fix mistake for enable accounts

* add docstring to DisableAccountsCli and EnableAccountsCli

* remove previous testnet and add toledo & pyrmont
2020-11-13 10:06:24 -06:00
Shay Zluf
18be4a4e3e Immediate Slashing Protection Data Storage (#7789)
* Immediate save of validator protection data

* fix error log

* separate delete from save

* remove logs

* rename delete into reset

* comment fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 22:07:32 +00:00
Victor Farazdagi
2f11e55869 Use t.TempDir() in tests (#7769)
* use t.TempDir()

* remove redundant delete

* simplify setupDB()

* simplify db/testing/setup_db

* fix tests
2020-11-10 22:45:17 +00:00
Potuz
e22dd3758d Attestation performance metrics (#7709)
* call LogValidatorGainsAndLosses at end of epoch

* Reviewer fixes

* Reviewer fixes

* Reviewer fixes

* Export Inclusion Distance to Prometheus

* changed default value to 1

* removed default value

* Added other performance metrics

* add slot

* get rid of inclusion_slot

* Fix fmt test

* Reviewer changes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-10 20:13:36 +00:00
Victor Farazdagi
09e3f0360e Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
Shay Zluf
7b0ee3adfe Use new attestation protection (#7605)
* Use new attestation protection

* tests fixes

* fix tests

* fix comment

* fix TestSetTargetData

* fix tests

* empty history handling

* fix another test

* mock domain request

* fix empty handling

* use far future epoch

* use far future epoch

* migrate data

* copy byte array to resolve sigbus error

* init validator protection on pre validation

* raul feedback

* rename fixes

* nishant feedback

* map with values

* fix tests

* lock and add test

* add and fix concurrency tests

* added tests error msg

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-10 14:14:11 +00:00
Preston Van Loon
ce75b2f684 Add more validation to AllValidatorsAreExited (#7755)
* Add more validation to AllValidatorsAreExited

* gofmt

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2020-11-10 11:07:35 +00:00
dv8silencer
b4bce7c726 Correct how AllValidatorsAreExited creates status request (#7758)
* fix and regression test

* address feedback

* gofmt

* improve test -- feedback

Co-authored-by: dv8silencer <15720668+dv8silencer@users.noreply.github.com>
2020-11-10 04:46:28 +00:00
Potuz
15706a36cb Allow exiting validators to attest (#7747)
* Allow exiting validators to attest

* Added regression test

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-09 18:01:38 +00:00
Radosław Kapka
3485f3b8b0 Check if all validators are exited on every slot (#7719)
* check if validators are exited in every loop iteration

* regression test

* fix mock validator

* handle context.WithValue key in a more idiomatic way

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-04 11:41:47 -08:00
Potuz
b996824446 call LogValidatorGainsAndLosses at end of epoch (#7708)
* call LogValidatorGainsAndLosses at end of epoch

* Reviewer fixes

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-03 23:47:12 +00:00
Nishant Das
211d9bc0b9 Update BLST And Herumi (#7632)
* fix build from source

* clean up

* update again

* change everything

* workaround for now

* fix versioning

* all passing now

* fix build issues

* clean up

* revert use of MulVerify

* gaz

* stub

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix all

* fix test

* todo

* fix stub

* revert back

* make deep source happy

* Update shared/bls/herumi/public_key.go

* Update shared/bls/blst/signature.go

* Update shared/bls/blst/signature_test.go

* imports

* move iface to common, export errors

* rm iface build

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-30 19:06:33 +00:00
Raul Jordan
8c2fff3a75 Allow for 25th Word Passphrases in Mnemonics (#7645)
* advanced functionality, enable 25th word mnemonic passphrase

* 25th word passphrase

* add test

* added test to ensure differences when using the mnemonic 25th word

* better message

* passing tests

* fix up logic
2020-10-27 20:51:29 +00:00
Radosław Kapka
21c5ba8ed8 display that all validators are exited (#7651) 2020-10-27 09:29:58 -05:00
Raul Jordan
a688b9e030 Miscellaneous Docs Improvements (#7606)
* misc docs fixes

* revert readme

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 19:45:31 +00:00
Raul Jordan
bec91d348e Use Password Specific to Web UI Instead of Wallet Password (#7569)
* use password specific to web UI

* fix up a few more tests

* tests passing

* gaz

* fix fmt
2020-10-20 02:26:31 +00:00
Raul Jordan
7e44d1eec7 Rename Direct Keymanager to Imported (#7549)
* rename direct to imported

* km

* fix more refs

* rename all instances appropriately

* rename instances of nonhd
2020-10-16 13:45:14 -05:00
Raul Jordan
a81c863ddb Rename Accounts-V2 to Accounts (#7545)
* rename accounts v2

* rename keymanager and fix imports

* rename accounts-v2 instances

* imports

* build

* build fix

* deepsource

* fix up broken aliases

* imports

* gaz

* Update validator/accounts/accounts_import_test.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fmt

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-10-15 22:31:52 +00:00