Commit Graph

114 Commits

Author SHA1 Message Date
Fabio Di Fabio
ba86ce1181 Add a method to check if a metric category is enabled to the plugin API (#7832)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
2024-10-30 13:12:10 -04:00
Fabio Di Fabio
22a570eda4 Fix registering new metric categories from plugins (#7825)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
2024-10-30 16:21:10 +01:00
Matilda-Clerke
2169985ee2 7311: add peertask foundation code (#7628)
* 7311: Add PeerTask system for use in future PRs

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
2024-10-18 11:33:15 +10:00
Justin Florentine
911f12efa0 Repackage options (#7688)
Updates MetricsOptions to implement CLIOptions, and to contain its unstable options.
---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
2024-10-04 11:17:46 +10:00
Justin Florentine
7d3e376771 shift creation of plugin context to BesuCommand for now (#7625)
* shift creation of plugin context to BesuCommand for now
* mock component will provide a no-op metrics sys

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
2024-09-18 14:31:03 +00:00
Matilda-Clerke
da98fa5541 Reintroduce engine get client version v1 with commit in manifest (#7548)
* Include Commit-Hash in manifests
* Add commit method in BesuInfo
* Use 8 character hash in EngineGetClientVersionV1 and revert calculateVersion back to original spec

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
2024-08-30 18:13:53 +00:00
Stefan Pingel
be8f4945b0 Metrics for sync phases (#7390)
* add metrics to measure the time for chain and world state sync

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2024-08-28 14:30:56 +00:00
Karim Taam
30c96c7a1d Optimistic parallelization of transactions to improve performance (#7296)
Optimistic transaction parallelization execution during block processing to improve the performances. This feature can enabled with a flag --Xbonsai-parallel-tx-processing-enabled=true

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Co-authored-by: Ameziane H <ameziane.hamlat@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
2024-07-19 09:11:30 -07:00
Usman Saleem
42e9139265 build: Add default constructor and javadoc for metrics, nat, pki, plugins, services, testutil, util (#7071)
Signed-off-by: Usman Saleem <usman@usmans.info>
2024-05-08 04:39:10 +00:00
Usman Saleem
f0cb26eb31 Enable spotless license header check (#7040)
* Enable spotless license header

Signed-off-by: Usman Saleem <usman@usmans.info>

* Apply spotless license header

Signed-off-by: Usman Saleem <usman@usmans.info>

---------

Signed-off-by: Usman Saleem <usman@usmans.info>
2024-05-03 01:38:42 +00:00
StevenMia
2bb0a59860 chore: fix some typos in comments (#6951)
* chore: fix some typos in comments

Signed-off-by: StevenMia <flite@foxmail.com>

* Apply suggestions from code review

fix typo

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: StevenMia <flite@foxmail.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2024-04-17 03:09:48 +00:00
Danno Ferrin
001080a72b Upgrade spotless to 1.22.0 (#6898)
Upgrade spotless to 1.22.0 and reformat.
This is required for Java21 support.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Co-authored-by: Danno Ferrin <danno@numisight.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2024-04-09 04:12:56 +00:00
Usman Saleem
80c6c04db6 fix: Use HttpRequest authority instead of host (#6879)
Use HttpRequest authority method to determine the hostname from header instead of using deprecated host method

Signed-off-by: Usman Saleem <usman@usmans.info>
2024-04-05 01:25:47 +00:00
Usman Saleem
e954537fcc build - Refactor Besu custom error prone dependency (#6692)
Move Besu custom error-prone checks into its own repository and use it as an external dependency. This allows to move to a newer version of Google errorprone checks as well while cleaning up build.gradle file.

Key changes resulted due to this change:

* String toLowerCase and toUpperCase to use Locale.ROOT as argument
* Use interface such as List,Map or NavigatableMap instead of concrete class where appropriate.
* Simplify StringBuilder to plain String
* Suppress warnings where appropriate.
-----
Signed-off-by: Usman Saleem <usman@usmans.info>
2024-03-27 06:17:40 +10:00
Danno Ferrin
9ef48d4399 Remove uses of org.jetbrains annotations (#6558)
* Remove uses of org.jetbrains annotations

We have a mix of org.jetbrains.annotations.NotNull and
javax.annotations.Nonnull. Change all to the latter. Add an errorprone
check to enforce this.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>

* spotless

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>

* spdx

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>

---------

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
2024-02-13 08:52:18 +10:00
Fabio Di Fabio
a563cf23c1 Upgrade Prometheus and Opentelemetry dependencies (#6422)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
2024-01-30 18:34:06 +10:00
Fabio Di Fabio
0b4b81595e Disable txpool when not in sync (#6302)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
2024-01-10 14:56:37 +01:00
Sally MacFarlane
db173ebd98 services - migrate to junit 5 (#5613)
* services to junit5

* removed some junit4 engine imports

* updated some plugins test since these extend from testutil KV storage

* one more form of EPL v2

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-06-21 09:12:51 +10:00
Nischal Sharma
857c9ac0b8 junit4 to junit5 (#5600)
Signed-off-by: Nischal Sharma <nischal@web3labs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-06-15 09:43:12 +10:00
Justin Florentine
25ab21128e Metrics via dagger (#5244)
* dagger component
* pushed up MetricsSystemModule
* passes around the BesuComponent as an interim application context till more things are managed by Dagger

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
2023-04-20 10:37:39 -04:00
ahamlat
ae04d46885 Upgrade RocksDB version from 7.7.3 to 8.0.0 (#5262)
Upgrade rocksdb to 8.0.0
Remove deprecated/obsolete RocksDB statistics in version 8.0.0

Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
2023-03-24 11:23:46 -07:00
matkt
77b55ed7d0 init heal code (#5059)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-02-08 14:30:11 +10:00
Usman Saleem
9eb32836b7 Fix javadocs to allow build to pass in JDK 17 (#4834)
- Added missing javadocs so that javadoc doclint passes against JDK 17 (invoke by Besu gradle build).
- Exclude following packages from javadoc lint:
org.hyperledger.besu.privacy.contracts.generated
org.hyperledger.besu.tests.acceptance.*
- Temporarily exclude ethereum and evm submodule for doc lint checks.
- Run the javadoc task using GitHub actions (use Java 17) to report any javadoc errors during the PR builds
- Updating plugin-api build.gradle with new hash as javadoc comments caused it to change

Signed-off-by: Usman Saleem <usman@usmans.info>
2023-01-18 22:51:00 +10:00
Justin Traglia
b1f5a7df36 Use value comparisons instead of reference comparisons in some places (#4757)
Signed-off-by: Justin Traglia <jtraglia@pm.me>

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2022-12-02 12:28:42 +10:00
garyschulte
36333b3db2 don't re-disable the OpenTelemetry global if it has already been set/disabled. Fix for evmtool usage (#4721)
Signed-off-by: garyschulte <garyschulte@gmail.com>
2022-11-22 13:21:17 -08:00
Antoine Toulme
5b462af0a7 Otel take 2 (#4075)
* Revert "Revert "Upgrade OpenTelemetry (#3675)" (#4031)"

This reverts commit 17de636fe2.

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Make sure we don't initialize the OpenTelemetry global singleton by mistake

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* disable global otel singleton explicitly

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* make sure to set GlobalOpenTelemetry at most once to avoid test failures

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* reset for tests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* fix changelog

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2022-11-14 19:01:43 +10:00
Sally MacFarlane
17de636fe2 Revert "Upgrade OpenTelemetry (#3675)" (#4031)
This reverts commit 78717ade1d.

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-06-30 11:07:14 +10:00
Antoine Toulme
78717ade1d Upgrade OpenTelemetry (#3675)
* Upgrade OpenTelemetry

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* remove a single sleep, poll with a for loop instead

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* use the new approach to send trace requests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-06-27 01:01:06 +02:00
Danno Ferrin
bdb00b299e Upgrade Errorprone (#3851)
* Upgrade Errorprone

Upgrade errorprone to 2.13.1.  Like all errorprone upgrades there are
new checks requiring code changes.

* Unused methods now cause compilation errors
* fields must be static and final
* Effectively constant booleans must now be returned as true/false.
* longs should not auto-cast to double.
* turn off errorprone javadocs

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-18 23:15:33 -04:00
Jim McDonald
37f44049d6 Log host and port when starting metrics service. (#3799)
* Log host and port when starting metrics service.

The metrics service start message was logging the port twice, rater than
the host and port as expected.  This changes the log message to print
the correct information in standard host:port format.

Signed-off-by: Jim McDonald <Jim@mcdee.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-09 03:28:11 +02:00
Sally MacFarlane
fb25cdb198 typos (#3782)
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-04 11:24:58 +10:00
Fabio Di Fabio
e01f63bbfa Adapt Fast sync, and Snap sync, to use finalized block as pivot after the Merge (#3655)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
2022-04-25 13:00:37 +02:00
matkt
d5658c2a65 add snapsync as experimental feature (#3710)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
2022-04-14 15:55:15 +02:00
Danno Ferrin
b4027d6e39 Use JUnit 5 for unit test execution (#3620)
Change the unit test execution to use the Junit5 JUnitPlatform. This
allows for a mix of junit 4 and junit 5 tests and for a gradual
migration to junit 5 instead of a big bang. One class depended on
junit 4 exceptions and was updated. Two tests depending on 
native libraries fail gracefully on mac (and only mac).

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2022-04-02 00:34:48 +10:00
Sally MacFarlane
db232418c2 to remove deprecated proto dependency, needed to add jaeger-proto dependency (#3485)
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-02-25 15:34:19 +13:00
Diego López León
ed1329cf84 Introduce SLF4J for logging (#3285)
* Bump SLF4J version

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Replace log4j2 API with SLF4j API

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Replace usage of LogManager#getFormatterLogger

This is for keeping compatibility with SLF4J. If neccesary, a specific formatter can be created for the RlpBlockImporter class

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Unset the default logging value for the retesteth

This is because it's not possible to resolve the root logger level into a Log4J2 field

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Prevent creation of Logger context outside SLF4J

org.hyperledger.besu.cli.BesuCommand#setAllLevels was taken from
https://github.com/apache/logging-log4j2/blob/rel%2F2.17.1/log4j-core/src/main/java/org/apache/logging/log4j/core/config/Configurator.java#L309

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Add FATAL level deprecation message

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* [Sonar] Fix java:S2139

Exceptions should be either logged or rethrown but not both

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* [Sonar] Fix java:S3457

Printf-style format strings should be used correctly

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Add changelog

Signed-off-by: Diego López León <dieguitoll@gmail.com>
2022-01-25 20:14:54 -03:00
Giuseppe Bertone
ce1c8ef22c Metrics idle timeout is now configurable (#2752)
Signed-off-by: Giuseppe Bertone <bertone.giuseppe@gmail.com>
2021-09-13 06:59:21 +10:00
Paul Harris
75324f3817 Cleanup metrics connections after idle timeout (#2748)
When the prometheus service restarts, connections can be left open and never closed. These connections should close after an idle timeout to avoid causing issues with running out of ports.

Signed-off-by: Paul Harris <paul.harris@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2021-09-10 12:45:16 +10:00
Antoine Toulme
cda70d3051 Upgrade to OpenTelemetry 1.5 (#2666)
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-08-21 09:25:48 -07:00
Paul Harris
92955d2894 Add a labelled gauge to metrics (#2646)
The labelled gauge functionality will allow monitoring of gauge type data without creating many gauges, which can be particularly handy in tracking related data.

Signed-off-by: Paul Harris <paul.harris@consensys.net>

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2021-08-20 12:58:41 +10:00
Antoine Toulme
670a19e7d8 Add stratum metrics (#2583)
* Add stratum metrics

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Use a more descriptive name for the target difficulty divisor
Use a double gauge so all difficulty values can be represented

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* rename field

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-07-29 08:49:51 +10:00
Antoine Toulme
5670264f12 Upgrade OpenTelemetry to 1.2.0 (#2313)
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2021-05-26 08:23:44 +10:00
Danno Ferrin
a30d8accb1 Use Http Compression (#1884)
Where vertx allows turn on gzip compression for HTTP.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2021-02-05 14:14:25 -07:00
Antoine Toulme
cd66968b6d Add tracing support for internals and JSON-RPC (#1557)
* Add tracing support for internals and JSON-RPC

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Remove rocksdb tracing as it slows down execution too much

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add B3 headers extraction on JSON-RPC requests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Remove traces around trie tree as they slow down syncing significantly

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add tracing to fast sync pipeline

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add tracing for all pipelines

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Address code review

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add acceptance tests and break out the shaded dependency

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Fix tracer id

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Revert changes to trie

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Upgrade otel to latest, remove old tracing

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review comments

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-01-04 11:11:13 -08:00
Antoine Toulme
45fd9f861e Add OTLP metrics support (#1492)
* Add OTLP metrics support

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Don't automatically push to grpc. Only allow if pushEnabled is set to true

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Missed refactoring

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add missing header

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

*  Don't make otel depend on the push enabled flag

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Expose JUL logs to log4j2

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review fixes and make sure not to start push gateway if not set to prometheus

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* spotless

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Fix unit test

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* code review feedback

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add changelog entry

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2020-11-12 13:22:22 -08:00
Adrian Sutton
80173d6fe6 Ignore accept header in requests to metrics service (#1345)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2020-08-28 08:12:38 +10:00
Sally MacFarlane
df647a7a6e [MINOR] Metrics: reinstated get/setWhitelist with Deprecated warning (#1142)
* reinstated get/setWhitelist with Deprecated warning

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-25 11:48:50 +10:00
Sally MacFarlane
9e224ed7f3 rename more "whitelist" occurrences to "allowlist" (#1133)
* rename more whitelist occurrences; change allowlisted to allowed and reword where we ended up with allowlisting

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-24 13:29:56 +10:00
Danno Ferrin
256e5c914d Errorprone version upgrade (#1096)
Upgrade to ErrorProne 2.4.0

* public constructors on abstract classes are removed
* Javadoc must have meaningfull documentation
* lambdas should not be variables
* Added to the list of confusing inner class names (Entry and Type)
* no assert keyword in tests
* Obsolete JDK classes produce errors now

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-06-17 10:34:56 -06:00
Sally MacFarlane
4eef568b4c Replace whitelist/blacklist with allowlist/denylist (#1077)
* replace whitelist/blacklist with allowlist/denylist and support both versions in CLI options

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-16 10:17:45 +10:00