67 Commits

Author SHA1 Message Date
Nazar Hussain
bbfdcb4cbe chore(docker): security upgrade grafana/grafana from 8.5.25 to 8.5.27 (#5924)
fix: docker/grafana/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-ALPINE315-NCURSES-5606598
- https://snyk.io/vuln/SNYK-ALPINE315-NCURSES-5606598
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-5661569
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-5661569
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-5788364

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-09-11 12:01:56 +02:00
Snyk bot
b8c239f020 chore: [Snyk] Security upgrade grafana/grafana from 8.5.22 to 8.5.25 (#5571)
* fix: docker/grafana/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3368753
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3368753
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-5291790
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-5291790

* Update docker/grafana/Dockerfile

---------

Co-authored-by: Cayman <caymannava@gmail.com>
2023-05-30 19:17:46 +00:00
Snyk bot
7e34b462b6 [Snyk] Security upgrade grafana/grafana from 8.5.20 to 8.5.22 (#5321)
fix: docker/grafana/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-ALPINE315-E2FSPROGS-3339845
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314621
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314621
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314622
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314629
2023-03-30 09:19:59 -04:00
Nazar Hussain
91473d3724 [Snyk] Security upgrade grafana/grafana from 8.5.16 to 8.5.20 (#5166)
* fix: docker/grafana/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314621
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314622
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314628
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314629
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-3314629

* Update docker/grafana/Dockerfile

---------

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
Co-authored-by: Cayman <caymannava@gmail.com>
2023-02-20 14:42:35 +01:00
Snyk bot
a0da8cd996 [Snyk] Security upgrade grafana/grafana from 8.4.2 to 8.5.16 (#5071)
fix: docker/grafana/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-2426331
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-2426331
- https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-2426331
- https://snyk.io/vuln/SNYK-ALPINE315-ZLIB-2434420
- https://snyk.io/vuln/SNYK-ALPINE315-ZLIB-2976173
2023-01-31 06:48:34 +08:00
Micah Zoltu
84b798dac4 Publishes Grafana and Prometheus images for monitoring. (#4927) 2023-01-04 10:04:16 -05:00
Lion - dapplion
d0a78a209c Re-org dashboards into common folder (#3905) 2022-04-10 21:07:08 +05:30
Lion - dapplion
55180870d9 Add Gossipsub debug charts (#3904) 2022-04-10 18:33:44 +05:30
dadepo
3e1f9b26fa Support running metrics via (grafana/prometheus) on macos (#3868)
* Support running local monitoring (grafana/prometheus) via docker on macos

* create seperate prometheus/grafana config files since host.docker.internal does not work crossplatform

* removed extra_hosts
2022-04-04 00:39:35 +05:30
Lion - dapplion
f63ae9d4f0 Add node exporter metrics section (#3816)
* Add node exporter metrics section

* Set examplar false
2022-03-01 09:54:19 -06:00
Lion - dapplion
0efbf1d671 Add more range sync metrics (#3803)
* Add more sync metrics

* Bump to 8.4.2

* Lock Grafana version

* Add Sync - Range charts

* Set exemplar false
2022-02-28 22:39:26 +05:30
Lion - dapplion
fa5ffea989 Update Grafana dashboard (#3795)
* Update Grafana dashboard

* Add common datasource uid

* Set exemplar to false
2022-02-28 10:48:23 +05:30
Lion - dapplion
9a58ce3c8b Run prettier on entire repo (#3720) 2022-02-07 09:57:26 -06:00
g11tech
0e47cd31af backfill sync from an anchor checkpoint state (#3384)
* Rebased mpetrunic's PR #2637 with fixes on current master

* fixing the remove peer error

* refactoring to solve sync stuck issues on not anchored kind of errors

* read from db, validate wsCheckpoint

* backfill sequences in db to skip redoing previous backfill work

* syncrange improvs

* feedback cleanup, modular refac of sync function and metrics update

* cleanup

* Graphana Dashboard

* renaming sequences to ranges

* rebase cleanup

* shortneing comment

* using initialize from's return as the anchorState

* Fix metrics

* Add Aborted enum value in lodestar_backfill_sync_status

* Only use JSDoc comment notation for JSDocs

* Simplify nullish values to be only null

* WIP

* refactoring the backfill sync, with parent-child linkage verfication of last previous unverified finalized or wscheckpoint block

* cleanup and simplification of checkpoint/prev finalized checks

* initializing backfillwritten to avoid previous overwriting with a ahead value

* prev finalized or wscheckpoint lookup fix

* missing initializtion

* better assignment of prev fin or ws checkpoint

* don't verify sig on genesis block

* making the extractPreviousFinOrWsCheckpoint lighter

* simplfication of extractPreviousFinOrWsCheckpoint

* improving messaging

* metric for prev fin or ws block slot validation

* dashboard entry for prev fin or ws checkpoint slot for validation

* dashed line for prev fin or ws slot for better clairty

* comments cleanup and always backfill

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
2022-01-06 18:32:38 +01:00
Lion - dapplion
f5d6f9f2e7 Add multi-node dashboard (#3503)
* Add ad hoc filter to dashboard

* Add multinode dashboard

* Enable auto-refresh

* Scan all dashboards for exemplar
2021-12-11 09:26:40 -06:00
g11tech
1c5047c126 changing points to line in graphana graphs (#3458) 2021-11-30 08:46:46 +01:00
tuyennhv
ac812ab38b Monitor validator balance (#3430)
* Monitor validator balance per epoch

* Grafana: add Correct Head Percentage panel
2021-11-16 09:25:08 +07:00
tuyennhv
40d47c1121 Grafana Dashboard: add Precompute Epoch Transition panels (#3421)
* Grafana Dashboard: add Precompute Epoch Transition panels

* Apply rate() to counter  metrics
2021-11-10 15:00:01 +01:00
Lion - dapplion
08dbb21538 Refactor discovery consumer (#3403)
* Integrate discv5 into discovery consumer

* Start discovery

* Update test types

* Add metrics for find node queries

* Add cachedENRsSize metric

* Add dashboard

* Track dropped ENRs

* Track peersToConnect metric

* Improve metrics

* Set exemplar to false

* More charts

* Fix e2e tests

* Tune charts

* WIP test

* Uncomment retry

* Track count of sync peers

* Review libp2p options

* Disable libp2p latency monitor

* Improve PeerManager peer data

* Overshoot when connecting to peers

* Skip discv5 e2e test
2021-11-04 12:03:21 -05:00
g11tech
13b61a32d2 prefixing lodestar to lodestar metrics, suffixing with a quantifier like count,total if missing in gauge metrics (#3404) 2021-11-01 09:19:33 +07:00
g11tech
861f6e4531 regen fn metrices collapse and negative cache artificats fx (#3261) 2021-10-28 17:11:14 +02:00
g11tech
694a6562a6 tracking UnhandledPromiseRejection(s) (#3386) 2021-10-25 16:28:42 -05:00
tuyennhv
20d4cab311 Epoch transition count metric (#3310)
* Epoch transition count metric

* Fix exemplar

* Address PR comment
2021-10-06 13:07:49 +07:00
tuyennhv
957dfb2aff Add Unknown Block Sync metrics to Grafana dashboard (#3244)
* Add Unknown Block Sync metrics to Grafana dashboard

* Revert title and uid
2021-09-24 18:58:58 +02:00
tuyennhv
f86a4381bd Add Gossip Block metrics (#3214)
* Add Gossip Block metrics

* Fix lint

* Fix check types

* Capture seenTimestamp before gossip queues

* Fix merge issue

* Calculate elappsedTimeTillProcessed in gossip handler

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
2021-09-21 10:48:36 +02:00
g11tech
1041a5d2a0 updating Dicv5 legends (#3181) 2021-09-16 09:27:28 +02:00
Cayman
e612b00a94 Add discv5 metrics to grafana dashboard (#3103)
* Add discv5 metrics to grafana dashboard

* Fix lint error

* Fix metrics

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
2021-09-13 22:25:57 +02:00
g11tech
ec7e4192e1 rearrangement of regen fn stats panel (#3119) 2021-09-13 10:49:15 +02:00
g11tech
2c9ebf748c switching off exemplar in queries (#3114)
Co-authored-by: gajinder <gajinder@g11>
2021-09-12 12:06:45 +02:00
g11tech
71b44e667b regen metrics reference impl (#2852)
* state cache and checkpoint cache metrics across all entrypoints

* Reduce diff

* refac regen metrics based on the latest jobprocessor queue

* regen cache dashboard

* regen fn stats

* removing labels from the cache metrics

* additional state/checkpoint state cache add, size metrics

* grafana dashboard update as well as new metrics for state and statecheckpoint

* Review PR

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
2021-08-28 15:36:02 +02:00
Lion - dapplion
0add9d11d6 Update Grafana dashboard (#2947) 2021-08-11 09:58:29 +02:00
Lion - dapplion
8a61ea0a86 Harden docker setup (#2891)
* Disable anonymous login in Grafana

* Don't expose API port by default

* Remove cli link in package.json

* Move NODE_OPTIONS to ENV to split beacon_node and validator limits

* Revert "Remove cli link in package.json"

This reverts commit 74c9b2ec9a.
2021-07-27 09:45:51 -05:00
Lion - dapplion
6fe84e427f Add generic batch BLS verification (#2801)
* Buffer jobs in BLS queue + batch them

* Mark some gossip objects as batchable

* Chunkify batchable

* Guard against missing jobResult

* Add more comments and review

* Update BLS grafana charts

* Just assert error happens

* Fix chart equation

* Don't call metric.inc() with 0
2021-07-26 15:20:22 -05:00
Lion - dapplion
623994c533 Add head drift chart (#2844) 2021-07-19 10:23:58 -05:00
Lion - dapplion
34f55490e5 Review gossipsub handlers (#2803)
* Handle onAttestation error

* Simplify gossip validation fns

* Move gossip topic handling to validate functions

* Re-org gossip handlers

* Cleanup

* Override validate function completely

* Fix tests

* Add StrictNoSign validation

* Add gossipMeshPeersBySyncCommitteeSubnet metric

* Handle multiple forks in meshPeers metrics

* Update tests

* Rename allForksAfterAltair

* Fix merge issues

* Fix merge issues in e2e tests
2021-07-07 08:48:16 +07:00
g11tech
3d1f78f182 block production metrics (#2811) 2021-07-06 09:16:11 -05:00
Lion - dapplion
5b3ff3a9fe Add DB metrics charts (#2802) 2021-07-05 15:20:38 -05:00
g11tech
7a3c282ce1 ReqResp metrics (#2774)
* ReqResp metrics

feedback changes

* fixes
2021-07-02 10:26:58 -05:00
g11tech
3392d7089f Add more fork-choice metrics (#2722)
* drill down find head/update head run times by lables syncStatus, resStatus

* Refactoring labels to guage and adding fork choice panels to grafana

* refactoring

* panel feedback changes
2021-06-28 10:58:28 -05:00
Lion - dapplion
e7048bbac5 Fix dashboard issues (#2492) 2021-05-05 10:17:58 -05:00
Cayman
0d67abd5e8 Add charts for new metrics (#2485)
* Add charts for new metrics

* Tweak charts

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
2021-05-04 16:54:24 -05:00
Lion - dapplion
c16174dff3 Improve BLS thread pool metrics (#2447)
* Improve BLS thread pool metrics

* Change new metric label

* Add red line to show process restarts
2021-05-03 15:52:51 -05:00
Lion - dapplion
5bdb118e0b Add clock slot and genesis time metrics (#2465)
* Add clock slot and genesis time metrics

* Update chart
2021-05-03 10:13:44 -05:00
Lion - dapplion
3f19346c0f Update grafana dashboard with REST API metrics (#2418) 2021-04-26 16:38:16 -05:00
Lion - dapplion
e8465816c5 Add queue charts to dashboard (#2332)
* Add queue charts to dashboard

* Left-pad subnet label

* Add network to Grafana dashboard

* Improve local version data fetching
2021-04-06 07:41:34 -05:00
Lion - dapplion
55ff6fefb4 Refactor metrics (#2312)
* Refactor metrics

* Fix metric name in grafana dashboard
2021-03-31 13:24:21 -05:00
Lion - dapplion
ce48eafd92 Refactor job queue (#2275)
* Refactor job queue

* Update metrics http server
2021-03-26 09:48:44 -05:00
Cayman
94c8304bd3 Merge pull request #2057 from ChainSafe/P0/docker-validator
Add validator service to docker-compose
2021-03-16 12:49:06 -05:00
Cayman
fe16231b4d Tweak beacon committee mesh size metric 2021-03-06 10:57:22 -06:00
Cayman
13f89d2eac Add mesh peer count to dashboard 2021-03-05 18:55:55 -06:00