Commit Graph

97 Commits

Author SHA1 Message Date
Sally MacFarlane
db173ebd98 services - migrate to junit 5 (#5613)
* services to junit5

* removed some junit4 engine imports

* updated some plugins test since these extend from testutil KV storage

* one more form of EPL v2

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-06-21 09:12:51 +10:00
Nischal Sharma
857c9ac0b8 junit4 to junit5 (#5600)
Signed-off-by: Nischal Sharma <nischal@web3labs.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-06-15 09:43:12 +10:00
Justin Florentine
25ab21128e Metrics via dagger (#5244)
* dagger component
* pushed up MetricsSystemModule
* passes around the BesuComponent as an interim application context till more things are managed by Dagger

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
2023-04-20 10:37:39 -04:00
ahamlat
ae04d46885 Upgrade RocksDB version from 7.7.3 to 8.0.0 (#5262)
Upgrade rocksdb to 8.0.0
Remove deprecated/obsolete RocksDB statistics in version 8.0.0

Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
2023-03-24 11:23:46 -07:00
matkt
77b55ed7d0 init heal code (#5059)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2023-02-08 14:30:11 +10:00
Usman Saleem
9eb32836b7 Fix javadocs to allow build to pass in JDK 17 (#4834)
- Added missing javadocs so that javadoc doclint passes against JDK 17 (invoke by Besu gradle build).
- Exclude following packages from javadoc lint:
org.hyperledger.besu.privacy.contracts.generated
org.hyperledger.besu.tests.acceptance.*
- Temporarily exclude ethereum and evm submodule for doc lint checks.
- Run the javadoc task using GitHub actions (use Java 17) to report any javadoc errors during the PR builds
- Updating plugin-api build.gradle with new hash as javadoc comments caused it to change

Signed-off-by: Usman Saleem <usman@usmans.info>
2023-01-18 22:51:00 +10:00
Justin Traglia
b1f5a7df36 Use value comparisons instead of reference comparisons in some places (#4757)
Signed-off-by: Justin Traglia <jtraglia@pm.me>

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
2022-12-02 12:28:42 +10:00
garyschulte
36333b3db2 don't re-disable the OpenTelemetry global if it has already been set/disabled. Fix for evmtool usage (#4721)
Signed-off-by: garyschulte <garyschulte@gmail.com>
2022-11-22 13:21:17 -08:00
Antoine Toulme
5b462af0a7 Otel take 2 (#4075)
* Revert "Revert "Upgrade OpenTelemetry (#3675)" (#4031)"

This reverts commit 17de636fe2.

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Make sure we don't initialize the OpenTelemetry global singleton by mistake

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* disable global otel singleton explicitly

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* make sure to set GlobalOpenTelemetry at most once to avoid test failures

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* reset for tests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* fix changelog

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2022-11-14 19:01:43 +10:00
Sally MacFarlane
17de636fe2 Revert "Upgrade OpenTelemetry (#3675)" (#4031)
This reverts commit 78717ade1d.

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-06-30 11:07:14 +10:00
Antoine Toulme
78717ade1d Upgrade OpenTelemetry (#3675)
* Upgrade OpenTelemetry

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* remove a single sleep, poll with a for loop instead

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* use the new approach to send trace requests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-06-27 01:01:06 +02:00
Danno Ferrin
bdb00b299e Upgrade Errorprone (#3851)
* Upgrade Errorprone

Upgrade errorprone to 2.13.1.  Like all errorprone upgrades there are
new checks requiring code changes.

* Unused methods now cause compilation errors
* fields must be static and final
* Effectively constant booleans must now be returned as true/false.
* longs should not auto-cast to double.
* turn off errorprone javadocs

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-18 23:15:33 -04:00
Jim McDonald
37f44049d6 Log host and port when starting metrics service. (#3799)
* Log host and port when starting metrics service.

The metrics service start message was logging the port twice, rater than
the host and port as expected.  This changes the log message to print
the correct information in standard host:port format.

Signed-off-by: Jim McDonald <Jim@mcdee.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-09 03:28:11 +02:00
Sally MacFarlane
fb25cdb198 typos (#3782)
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-05-04 11:24:58 +10:00
Fabio Di Fabio
e01f63bbfa Adapt Fast sync, and Snap sync, to use finalized block as pivot after the Merge (#3655)
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
2022-04-25 13:00:37 +02:00
matkt
d5658c2a65 add snapsync as experimental feature (#3710)
Signed-off-by: Karim TAAM <karim.t2am@gmail.com>
2022-04-14 15:55:15 +02:00
Danno Ferrin
b4027d6e39 Use JUnit 5 for unit test execution (#3620)
Change the unit test execution to use the Junit5 JUnitPlatform. This
allows for a mix of junit 4 and junit 5 tests and for a gradual
migration to junit 5 instead of a big bang. One class depended on
junit 4 exceptions and was updated. Two tests depending on 
native libraries fail gracefully on mac (and only mac).

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2022-04-02 00:34:48 +10:00
Sally MacFarlane
db232418c2 to remove deprecated proto dependency, needed to add jaeger-proto dependency (#3485)
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2022-02-25 15:34:19 +13:00
Diego López León
ed1329cf84 Introduce SLF4J for logging (#3285)
* Bump SLF4J version

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Replace log4j2 API with SLF4j API

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Replace usage of LogManager#getFormatterLogger

This is for keeping compatibility with SLF4J. If neccesary, a specific formatter can be created for the RlpBlockImporter class

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Unset the default logging value for the retesteth

This is because it's not possible to resolve the root logger level into a Log4J2 field

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Prevent creation of Logger context outside SLF4J

org.hyperledger.besu.cli.BesuCommand#setAllLevels was taken from
https://github.com/apache/logging-log4j2/blob/rel%2F2.17.1/log4j-core/src/main/java/org/apache/logging/log4j/core/config/Configurator.java#L309

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Add FATAL level deprecation message

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* [Sonar] Fix java:S2139

Exceptions should be either logged or rethrown but not both

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* [Sonar] Fix java:S3457

Printf-style format strings should be used correctly

Signed-off-by: Diego López León <dieguitoll@gmail.com>

* Add changelog

Signed-off-by: Diego López León <dieguitoll@gmail.com>
2022-01-25 20:14:54 -03:00
Giuseppe Bertone
ce1c8ef22c Metrics idle timeout is now configurable (#2752)
Signed-off-by: Giuseppe Bertone <bertone.giuseppe@gmail.com>
2021-09-13 06:59:21 +10:00
Paul Harris
75324f3817 Cleanup metrics connections after idle timeout (#2748)
When the prometheus service restarts, connections can be left open and never closed. These connections should close after an idle timeout to avoid causing issues with running out of ports.

Signed-off-by: Paul Harris <paul.harris@consensys.net>
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2021-09-10 12:45:16 +10:00
Antoine Toulme
cda70d3051 Upgrade to OpenTelemetry 1.5 (#2666)
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-08-21 09:25:48 -07:00
Paul Harris
92955d2894 Add a labelled gauge to metrics (#2646)
The labelled gauge functionality will allow monitoring of gauge type data without creating many gauges, which can be particularly handy in tracking related data.

Signed-off-by: Paul Harris <paul.harris@consensys.net>

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2021-08-20 12:58:41 +10:00
Antoine Toulme
670a19e7d8 Add stratum metrics (#2583)
* Add stratum metrics

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Use a more descriptive name for the target difficulty divisor
Use a double gauge so all difficulty values can be represented

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* rename field

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-07-29 08:49:51 +10:00
Antoine Toulme
5670264f12 Upgrade OpenTelemetry to 1.2.0 (#2313)
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2021-05-26 08:23:44 +10:00
Danno Ferrin
a30d8accb1 Use Http Compression (#1884)
Where vertx allows turn on gzip compression for HTTP.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2021-02-05 14:14:25 -07:00
Antoine Toulme
cd66968b6d Add tracing support for internals and JSON-RPC (#1557)
* Add tracing support for internals and JSON-RPC

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Remove rocksdb tracing as it slows down execution too much

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add B3 headers extraction on JSON-RPC requests

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Remove traces around trie tree as they slow down syncing significantly

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add tracing to fast sync pipeline

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add tracing for all pipelines

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Address code review

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add acceptance tests and break out the shaded dependency

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Fix tracer id

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Revert changes to trie

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Upgrade otel to latest, remove old tracing

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review comments

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2021-01-04 11:11:13 -08:00
Antoine Toulme
45fd9f861e Add OTLP metrics support (#1492)
* Add OTLP metrics support

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Don't automatically push to grpc. Only allow if pushEnabled is set to true

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Missed refactoring

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add missing header

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

*  Don't make otel depend on the push enabled flag

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Expose JUL logs to log4j2

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Code review fixes and make sure not to start push gateway if not set to prometheus

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* spotless

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Fix unit test

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* code review feedback

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>

* Add changelog entry

Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2020-11-12 13:22:22 -08:00
Adrian Sutton
80173d6fe6 Ignore accept header in requests to metrics service (#1345)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2020-08-28 08:12:38 +10:00
Sally MacFarlane
df647a7a6e [MINOR] Metrics: reinstated get/setWhitelist with Deprecated warning (#1142)
* reinstated get/setWhitelist with Deprecated warning

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-25 11:48:50 +10:00
Sally MacFarlane
9e224ed7f3 rename more "whitelist" occurrences to "allowlist" (#1133)
* rename more whitelist occurrences; change allowlisted to allowed and reword where we ended up with allowlisting

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-24 13:29:56 +10:00
Danno Ferrin
256e5c914d Errorprone version upgrade (#1096)
Upgrade to ErrorProne 2.4.0

* public constructors on abstract classes are removed
* Javadoc must have meaningfull documentation
* lambdas should not be variables
* Added to the list of confusing inner class names (Entry and Type)
* no assert keyword in tests
* Obsolete JDK classes produce errors now

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-06-17 10:34:56 -06:00
Sally MacFarlane
4eef568b4c Replace whitelist/blacklist with allowlist/denylist (#1077)
* replace whitelist/blacklist with allowlist/denylist and support both versions in CLI options

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-06-16 10:17:45 +10:00
Adrian Sutton
f2244ee531 Handle requests when the host header is not specified at all correctly. (#901)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2020-05-12 08:34:45 +10:00
Sally MacFarlane
b9b09319ae typos (#755)
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
2020-04-22 05:45:34 +10:00
Danno Ferrin
a66b244cc7 Log to trace denied because of host whitelisting (#663)
When we deny a connection based on HTTP hostname log to trace the
rejected value.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
Co-authored-by: Usman Saleem <usman@usmans.info>
2020-04-08 00:36:07 -06:00
Danno Ferrin
a4143dc410 Check to see if vertx response is closed before responding (#613)
Occasionally the other side of an HTTP connection will drop the
connection before we are done processing.  This results in a harmless
but annoying exception showing up in the log.  If we check before we
write we won't get that exception.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-03-31 14:09:07 -06:00
Danno Ferrin
838f7aa5e0 Fix Gradle 7.0 compatibility issues and build warnings (#288)
Removes as many Gradle 7.0 compatibility issues as possible
* `baseName` -> `archiveBaseName`
* `extension` -> `archiveExtension`
* `destinationDir` -> `destinationDirectory`
* `runtime` -> `runtimeOnly`
* Change some log4j-api and log4j-core dependencies
* Remove an unneeded and outdated plugin (`net.ltgt.apt`)
* tweak the plugin-api change detector's property annotations.

Warnings still exist with one external plugin used for license file
checking that we do not control the source code for.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-01-13 08:28:50 -07:00
Danno Ferrin
e493b91742 Restore "FieldCanBeFinal" errorprone check (#287)
This was turned off a number of months ago because of long build times.
However it appears either because of code structure changes or fixed
errorprone that this finishes in a reasonable time now.

Code that violated this check is also fixed.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-01-02 10:27:54 -07:00
Danno Ferrin
dfad8bff78 Update Dependencies (#281)
Update dependencies to most current version
 - except picocli which is a major version update
Alphabetize dependencies

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
2020-01-01 20:42:13 +13:00
Antoine Toulme
376ec111a3 Make sure to close Vert.x instances that are opened for testing (#280)
Signed-off-by: Antoine Toulme <antoine@lunar-ocean.com>
2019-12-23 10:33:16 -07:00
Trent Mohay
3d867532de Repaired Metrics name collision between Privacy and RocksDB (#187)
It was identified that when both KV_ROCKSDB metrics and Privacy were enabled,
that Besu failed to start due to a naming collision in Metrics Collators - this was ultimately due to Besu having 2 keyvalue stores - one for public state, and another for private state - and _both_ using the same metrics.

To overcome this issue, the metrics used in the private kv store are prefixed with the word "private".

Signed-off-by: Trent Mohay <trent.mohay@consensys.net>
2019-11-15 08:47:22 +11:00
Lucas Saldanha
11cb619a50 Fixing rocksdb artifact name conflicts (#73)
Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>
2019-10-02 09:18:20 +13:00
mark-terry
e1e720d905 PIE-1858: Infrastructure for exposing PoA metrics for plugins. (#37)
[PIE-1858] Added functionality to register custom metrics categories and exposed some PoA data for metrics.

Signed-off-by: Mark Terry <mark.terry@consensys.net>
2019-09-30 15:58:51 +10:00
Joshua Fernandes
bac5c673d3 Adding a spdx license check gradle task (#30)
* adding in spdx-license-identifier & updated check for the same; removing license check from spotless

Signed-off-by: Joshua Fernandes <joshua.fernandes@consensys.net>

* Change CheckSpdxHeader to a task.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-20 14:59:57 +10:00
Joshua Fernandes
d6a2394e2d Migrating Pantheon to Besu (#1945)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-16 11:14:10 +10:00
CJ Hare
dec01db6f9 Refactoring Rocksdb as a module (#1889)
Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-12 12:34:57 +10:00
Danno Ferrin
ac2db5a205 [PIE-1859] - remove metrics from plugin registration (#1918)
Remove metrics from plugin registration.  Doing it this way here always results in a
NoOpsMetrics being inited because CLI options have not been parsed.

Add an acceptance test for metrics.  This is what should have been in place and
would have detected metrics not working earlier.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-10 21:20:44 -06:00
Edward
9856cd32b0 Ensure plugin-api module gets published at the correct maven path (#1905)
* Ensure `plugin-api` module gets published at the correct maven path
* Move `plugins` to `plugin-api`

Signed-off-by: Edward Evans <edward.evans@consensys.net>

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-05 10:29:30 +10:00
Edward
9601f44ac8 Return the plugin-apis to this repo (#1900)
* Return plugin-api to the main repo
* Spotless
* Migrate all external plugin-api references to the project in this repo
* Add licence header
* Update repo reference for publish, even if commented
* Use real configuration for publishing plugin-api

This was tested with the
`:plugins:publishMavenJavaPublicationToMavenLocal` task and checking the
local Maven repo to make sure it was using the correct paths

Signed-off-by: Edward Evans <edward.evans@consensys.net>

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
2019-09-04 12:12:27 +10:00