107 Commits

Author SHA1 Message Date
jonesho
0b863e2a6e feat: update trace-limits and besu images (#1129)
* feat: update trace-limits and besu images

* update prover image

* feat: update trace-limits ROM to the power of 2

* feat: update trace-limits to be at max 2147483647

---------

Co-authored-by: gusiri <dreamerty@postech.ac.kr>
2025-06-11 00:09:20 +08:00
Fluent Crafter
194742eda8 coordinator: update inage on local stack (#1113) 2025-06-06 15:34:09 +01:00
Fluent Crafter
a9bc257baa Localstack web3j default signer (#1111)
* localstack: use we3bj by default to make it lightweight

* localstack: update coordinator image
2025-06-06 14:41:42 +01:00
Fluent Crafter
0c7c736830 coordinator: adds config v2 (#853)
* coordinator: adds config v2 sample

* coordinator: update config V2

* coordinator: update config v2

* coordinator: update config v2

* coordinator: config v2 wip

* spotless fix

* coordinator: adds config v2 parsers

* coordinator: adds config v2 classes and parsing tests

* coordinator: adds config v2 classes and parsing tests

* remove end2end changes from staterecovery test trigger

* coordinator: config v2 - fix prover directory config

* coordinator: add getChaindId to EthApiClient.kt

* coordinator: improve EIP1559GasProvider validation

* coordinator: add createReadOnly to Web3JL2MessageServiceSmartContractClient

* coordinator: add validation to FeeHistoryFetcherImpl

* coordinator: add more configs to v2

* coordinator: extend Web3JFactory

* coordinator: addapt CoordinatorApp to new V2 configs

* coordinator: adapt local stack coordinator configs

* coordinator: log4j clients.l1 debug

* coordinator: revert attempt to use web3signer on CI ONly :(

* coordinator: fix test and configs

* coordinator: fix test and configs

* coordinator: fix traces node address

* coordinator: remove unnecessary file

* coordinator: hardcode tracesVersion to v2.1.0 to match prover regex

* Update config/coordinator/coordinator-config-v2.toml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Fluent Crafter <205769460+fluentcrafter@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>
Signed-off-by: Fluent Crafter <205769460+fluentcrafter@users.noreply.github.com>

* coordinator: default targetBlobsPerTransaction=7u

* coordinator: hardcode tracesVersion to 2.1.0 to match prover regex

* coordinator: fix agg configs

* coordinator: strict configs log warning when config is not used

* coordinator: add carved out config files

* coordinator: add opt-in to avoid annoying warning log

* feat: update Makefile for new coordinator config file and variable name

* localstack: remove coordinator forced platform

* coordinator: add missing config on gas-price-cap-calculation

* coordinator: remove old configs

---------

Signed-off-by: Fluent Crafter <205769460+fluentcrafter@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>
Co-authored-by: jonesho <81145364+jonesho@users.noreply.github.com>
Co-authored-by: jonesho <jones.ho@consensys.net>
2025-06-06 11:15:01 +01:00
jonesho
62b422b634 feat: add script for assemble linea besu packages and makefile (#1050)
* feat: add script for assemble linea besu packages and makefile

* feat: update README.md for make commands to build linea-besu-package locally

* feat: revise the make commands for build and run e2e tests

* feat: add timeout for shomei frontend e2e test

* feat: revise README

* feat: update e2e test timeout and reduce log

* feat: update e2e test timeout

* feat: revise Makefile for compatible sed -i command on MacOS and Linux

* feat: update e2e test timeout

* feat: added log info for e2e test

* feat: revise the e2e test to avoid timeout

* feat: revise the e2e test to avoid timeout
2025-06-06 02:22:27 +08:00
Victorien Gauch
86f9bbf819 fix: update postman docker image tag in the local stack (#1098) 2025-06-05 11:20:46 +02:00
jonesho
953e7a90a5 feat: added shomei-frontend linea_getProof e2e test (#1077)
* feat: added shomei-frontend linea_getProof e2e test

* feat: revise the e2e test with comments

* feat: make linked library name human readable
2025-06-03 15:29:23 +08:00
Victorien Gauch
19735ced28 fix: update blob compressor lib + update dependencies + use tsup (#976)
* fix: update blob compressor lib + update dependencies + use tsup

* fix: update dict path

* fix: lockfile issue

* fix: bump package json version
2025-06-02 14:43:53 +02:00
jonesho
7e306e2658 feat: remove l2-node in e2e test and make it to run in local stack op… (#1001)
* feat: remove l2-node in e2e test and make it to run in local stack optionally

* fix: l2 spec transaction data size limit e2e test

* feat: revise blockscout configs

* feat: revised comment in Makefile

* feat: update transaction-exclusion-api and postman image

* feat: remove expected-traces-api-version-v2 from coordinator local overrides toml

* feat: remove shomei-frontend from ci and add disabled option in Type2StateProofProviderConfig

* feat: add endpoint for type2-state-proof-provider

* feat: revise coordinator local-dev override configs

* feat: remove white space
2025-05-22 23:27:14 +08:00
FlorianHuc
af0471b953 update shomei plugin mode 2025-05-22 11:18:39 +00:00
Fluent Crafter
f6cb0f2a05 Besu packgae update sequencer version (#1026)
* besu-package: update sequencer version

* besu-package: update sequencer version

* besu-package: update sequencer version

* besu-package: sequencer plugin update to v2.1-rc16.2.3

* besu-package: update besu package

* localstack: update coordinator version

* localstack: remove unnecessary

* localstack: revert l2 node configs on profitability check

* localstack: update e2e README

* besu-package: update sequencer plugin

* Revert "localstack: revert l2 node configs on profitability check"

This reverts commit 24ac5f6b746b85c4f27c02aaa63d992549aa44ac.

* besu-package: update plugins

* besu-package: update besu image

* fix flaky test

* besu-package: remove besu/plugins from classpath

* expected-traces-api-version-v2="beta-v2.1-rc16.2"

* update shomei plugin

* update besu-package image

* besu-package add back plugins folder to classpath

* localstack: update besu package
2025-05-21 14:39:50 +01:00
Fluent Crafter
cd49558fcf besu-package: fix dockerfile classpath (#1021)
* besu-package: fix dockerfile classpath

* besu-package: update local stack

* fid tracer version on coordinator configs beta-v2.1-rc16.1

* fid tracer version on coordinator configs beta-v2.1-rc16.1

* feat: revise workflow to trigger e2e on dockerfile change, update sed command, and update coordinator config

* feat: revert the coordinator config for l2-network-gas-pricing

* update local images

* staterecovery: synchronize latch variable

---------

Co-authored-by: jonesho <jones.ho@consensys.net>
2025-05-19 12:08:42 +01:00
jonesho
38efa5fdff feat: update l1-el-node image to a recent linea-besu-package (#1018) 2025-05-16 16:08:11 +08:00
jonesho
9bf4b5cd17 feat: add manual inputs to set custom expected-traces-api-version (#1016)
* feat: add manual inputs to set custom expected-traces-api-version

* feat: revise comment

* feat: revert the l1-el-node image hardcoded tag and update L1 genesis for Prague

* feat: update linea-besu-package image tag and coordinator config
2025-05-16 15:42:54 +08:00
Fluent Crafter
2f5d08b7bc Coordinator dictionary update (#962)
* update compressor libraries with new dictionary

* update blob compressor library version

* coordinator: add new dictionary to blob compressor and decompressor

* coordinator: revert compressor lib update

* coordinator: compressor change dependencies from implementation to compileOnly

* coordinator: update compressor dependencies to released in maven

* update local stack images

* localstack: revert L2 node endpoit

* localstack: update postman image

* staterecovery: add missing dependencies to uberjar

* staterecovery: update version in linea-besu-package

---------

Co-authored-by: Tabaie <arya.pourtabatabaie@gmail.com>
2025-05-14 16:20:48 +01:00
Fluent Crafter
522bd26f59 localstack: update prover and coordinator (#993) 2025-05-13 11:18:05 +01:00
Arya Tabaie
b448d95ab0 feat: more explicit "dictionary not found" error (#985) 2025-05-13 06:11:04 +02:00
Fluent Crafter
255b3ac670 coordinator: update image in local stack (#977)
* coordinator: update image in local stack

* coordinator: move hardcoded version to libs versions

* coordinator: fix dependency
2025-05-09 16:43:33 +01:00
jonesho
47a9d07d4f feat: remove switch-to-linea-besu, counter-v1, conflation-v1, file-ma… (#952)
* feat: remove switch-to-linea-besu, counter-v1, conflation-v1, file-manager related configs and their reference codes in L1DependentApp

* fix: spotless

* feat: remove traces-api-facade module

* feat: remove traces-counter-v1 from coordinator config and removed TracesFileManager, TracesRpcClientV1, TraceCounterV1WatcherClient classes

* feat: removed TracesCounterV1 and update tests

* feat: removed TracingModuleV1 and traces-limits-v1 toml files

* feat: removed TracesClientV1, TracesClientV2Adapter, and revised codes accordingly

* feat: removed FileManager in CoordinatorConfig and updated test case for assertConsecutiveBlocksRange

* feat: updated l2-node-besu docker-compose configs

* feat: rename traces-node-v2 to traces-node in all places

* feat: use prover v3 compression proofs for tests

* fix: spotless
2025-05-09 19:52:46 +08:00
The Dark Jester
1b2040fe1e [Feat] Use 9 blobs for Pectra (#944)
* use 9 blobs for Pectra

* add linting to script

* turn prague on by default

* remove non-prague configuration

* testing not-yet-prague is the reason for failure

* using pectra from genesis

* reset to 9 for testing

* lowering blobs per tick

* add missing 9s and update blobscan

* spotless

* expect 9 blobs in one tx

* Update coordinator/app/src/main/kotlin/net/consensys/zkevm/coordinator/app/config/CoordinatorConfig.kt

Co-authored-by: Julien Marchand <julien-marchand@users.noreply.github.com>
Signed-off-by: The Dark Jester <thedarkjester@users.noreply.github.com>

* refactor testdata folders and use 9 blob agg

* refactor function for ease of reading

* move arguments up

* rename function

* format parameters

* validate at least 1 blob exists

* more spotless

* simplify blob hash computation

---------

Signed-off-by: The Dark Jester <thedarkjester@users.noreply.github.com>
Co-authored-by: Julien Marchand <julien-marchand@users.noreply.github.com>
2025-05-09 01:47:54 -07:00
jonesho
1b874b45fb feat: removed all tracing-v1 related codes from github workflow (#932)
* feat: removed all tracing-v1 related codes from github workflow

* feat: removed all tracing-v1 related codes from makefile, test files, and docker yml files

* feat: removed all old prover, geth traces node, and traces-api in docker files

* feat: removed traces-api-facade related codes from workflow

* feat: update image tag of coordinator and others
2025-05-06 01:19:33 +08:00
kyzooghost
19880cd8a9 [Feat] Postman Sponsorship Metrics (#902)
* added claimTxGasUsed attribute

* add test code for message mapper

* add two more table entries

* added isForSponsorship db entity change

* remove isForSponsorship

* working postman tests

* working tests for add isForSponsorship attribute

* fix positioning

* refactor MessageClaimingPersister

* remove bad comment

* added migration:create command

* generate migration docs

* added new migration for sponsorship

* typo

* passing tests for MessageMetricsService & MetricsService refactor

* remove console logs

* added subscriber code for isForSponsorship

* working e2e and local test for sponsorship messages metrics

* add comment

* added SponsorshipFeesWei and SponsorshipFeesGwei metrics

* did sponsorshipfeesubscriber

* register sponsorshipfeessubscriber

* string fixes

* remove claimtxgasused and claimtxgasprice db columns, refactor metricsservice design pattern

* SponsorshipMetricsUpdater.ts

* change isForSponsorship semantics

* remove isForSponsorship from DB

* Revert "remove isForSponsorship from DB"

This reverts commit 79f4887e3492a43443bb5dba78f66f78606241a5.

* remove isForSponsorship public function

* remove isForSponsorship label from message metric

* add getSponsoredMessagesTotal method

* added sponsorshipmetricsupdater to messageclaimingpersister

* amend base postman image for e2e test

* add promql queries in comments

* empty

* remove === true
2025-05-05 20:26:37 +10:00
Roman Vaseev
d43b3a965c Fixing the arithmetization version conflict between the sequencer and tracer plugins (#914) 2025-04-30 15:24:16 +02:00
kyzooghost
b021601c61 [Fix] Postman Sponsoring config (#892)
* change PROFIT_MARGIN to 0.01

* fix misconfig
2025-04-23 08:31:59 +10:00
Victorien Gauch
b2990e9a3c Feat: Add metrics to postman (#871)
* feat: add metrics to postman

* fix: issue with get message to claim query

* fix: import issues

* fix: update unit tests and fix minor issues

* fix: refator metrics

* fix: update typeorm query

* fix: update postman docker config
2025-04-22 11:44:53 +02:00
kyzooghost
9a16c5e152 [Feat] Postman Sponsor (#860)
* remove isTokenCanonicalUSDC

* move logic to handleTokenClick

* add clause

* remove string for ClaimType

* remove todos

* rename for clarity

* added switch for auto_free

* replaced AUTO_PAID

* some more auto_free

* fix typo

* did free text

* comment fixes

* add test todos

* added config values to postman

* added isForSponsorship

* add isForSponsorship condition to MessageClaimingProcessor

* removed config from messageclaimingprocessor

* did tests for transactionvalidationservice

* did one messageclaimingprocessor test

* added cases to MessageClaimingProcessor test

* rm valueAndFee magic value from e2e test

* new test

* working e2e test

* test refactor

* test refactor

* new e2e test

* add new e2e case

* remove .only

* new test case

* test 2 workers for e2e

* Revert "test 2 workers for e2e"

This reverts commit 8256043df9613015e98c8b5507fcf87f3a8ccc06.

* empty

* adjust for comments 1

* empty

* adjust for comment 2
2025-04-22 19:28:53 +10:00
jonesho
74703cfd64 feat: move plugins into besu-plugins (#879)
* feat: move plugins into besu-plugins

* fix: naming of the besu-plugins folder

* feat: update description for pluginName

* feat: comment out unused task in gradle.build

* feat: align the same besu version used across all modules

* feat: removed vertx-helper dependency in message-anchoring module
2025-04-17 16:30:25 +08:00
jonesho
31afe41f77 feat: add send-bundle e2e test case with workaround to support traces… (#771)
* feat: add send-bundle e2e test case with workaround to support traces-v1 sequencer

* feat: linting and removed io.consensys in maven gradle

* feat: RPC besu node to forward sendBundle to sequencer in e2e

* feat: update besu nodes to devnet-9d6e914 and coordinator version update

* feat: update coordinator version and l2-node-besu plugins config

* feat: remove gas-limit e2e tests as already moved to 2b block gas limit for all envs

* feat: remove opcode test contract related variables

* feat: update coordinator version

* feat: update linea-besu-package version

* feat: revise test cases based on review comment

* feat: skip send bundle tests explicitly if traces-v1

* feat: always pass send bundle tests if traces-v1

* feat: remove unused helper function

* feat: skip send bundle tests explicitly if traces-v1

* feat: update to use l2-node-besu log4j.xml in l2-node-besu

* feat: use describe.skip instead of it.skip for skipping bundle tests
2025-04-10 18:30:12 +08:00
Sally MacFarlane
a2b5e31318 replace deprecated option (#769)
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: jonesho <81145364+jonesho@users.noreply.github.com>
2025-04-08 22:58:48 +08:00
Fluent Crafter
4f5af620c5 staterecovery: fix 2B block gas limit (#837)
* staterecovery: fix 2B block gas limit

* staterecovery: Add better failure message for fake client

* staterecovery: update besu image

* staterecovery: udpate traces version on coordinator file

* staterecovery: add explicit timeout in test

* staterecovery: fix CLI typo
2025-04-07 09:41:41 +01:00
Fluent Crafter
811743bb5a coordinator: update Shomei frontend at LATEST finalization (#831)
* coordinators: add finalization monitor on LATEST to update shomei frontend faster.

* coordinators: fix spotless

* coordinators: adapt local configs to not try to update disabled services in docker local stack
2025-04-02 09:57:53 +01:00
jonesho
869c0c63d4 feat: updated linea-besu-package in local stack (#794) 2025-03-20 02:51:52 +08:00
jonesho
6aa305f500 feat: update build.gradle to org.hyperledger and linea-besu-upstream (#779)
* feat: update build.gradle to org.hyperledger and linea-besu-upstream

* feat: add org.hyperledger as included group in maven repositories url

* feat: update linea-besu-package in compose files for linea-besu-upstream with new plugin releases
2025-03-17 16:33:04 +08:00
jonesho
5c8ea31d2a feat: add support of eth_estimateGas for L1 transactions (#694)
* feat: add support of eth_estimateGas for L1 transactions

* feat: revise log message for calling eth estimate gas

* chore: update function signatures

* fix: unit test for merge conflicts

* fix: merge conflicts

* feat: use web3J async helper to encapsulate useEthEstimateGas option

* feat: remove calling eth_estimateGas in eth_call for aggregation tx and disable eth estimate gas on data submission

* feat: seperate useEthEstimateGas option for data submission and aggregation and remove calling eth_estimateGas on eth_call for data submission

* feat: add new function in LineaRollupSmartContractClient and remove unnecessary parameter

* feat: correct coordinator config and update sequencer config
2025-02-28 02:40:53 +08:00
jonesho
845ddedd5f feat: add metrics in GlobalBlobAwareConflationCalculator and move bri… (#654)
* feat: add metrics in GlobalBlobAwareConflationCalculator and move bridge logs and state root hash retrieval from execution prover client to coordinator

* fix: unit test

* feat: testing for docker cache pipeline

* feat: disabled docker cache

* feat: move encoder back to file-based prover client

* fix: compare with hex string instead of ulong for bridgelog block number

* feat: add test case for ExecutionProofRequestDtoMapper

* feat: revise ExecutionProofRequestDtoMapper test case

* feat: revise metric names and add testing for the added metrics

* feat: added metricscheck

* feat: handle zero transactions in empty blocks for avg-tx-data-size histogram metrics

* feat: resolve conflicts from latest main and fix warning on missing env vars in docker compose

* fix: unit tests

* feat: revise unit tests with fake implementation of histogram to reduce the use of mock

* feat: resuse filtered block counters
2025-02-25 18:10:36 +08:00
Pedro Novais
45a41b455d increase block gas limit as in sepolia (#720)
* increase block gas limit as in sepolia

* increase block gas limit as in sepolia

* staterecovery: update test data to match new genesis with 2B gas block

* staterecovery: update test data to match new genesis with 2B gas block

* staterecovery: reduce amount of blobs sent to L1

* staterecovery: increase l1 besu memory limit

* staterecovery: disable parallel tests in staterecovery

* staterecovery: donwgrade runner to large

* staterecovery: fix some integration tests

* staterecovery: fix integration tests
2025-02-24 14:26:15 +00:00
Pedro Novais
3c4f4c26db chore: improve package consistency jvm lib (#719)
* chore: improve package consistency jvm lib

* chore: improve package consistency jvm lib

* chore: improve package consistency domain models

* package rename improve consistency

* chore: fix docker config
2025-02-20 15:20:02 +00:00
Pedro Novais
3caceed6f6 Staterecovery part 13 blockhash fixes (#675)
staterecovery: fix blockHash opcode and improve performance of blob fetching from L1

---------

Signed-off-by: Pedro Novais <1478752+jpnovais@users.noreply.github.com>
Co-authored-by: Roman Vaseev <4833306+Filter94@users.noreply.github.com>
2025-02-20 10:14:38 +00:00
kyzooghost
2a38663956 implemented feature flag for prague local stack (#715) 2025-02-20 17:39:14 +11:00
Roman Vaseev
4af51c0193 Updating sequencer (#708)
* Updating sequencer

* Updating tracer as well

* Updating besu node as well
2025-02-19 15:43:27 +01:00
Pedro Novais
2f5dd4930e finalized-tag-plugin-update (#685)
* rename to linea-finalized-tag-plugin
---------

Co-authored-by: jonesho <jones.ho@consensys.net>
Co-authored-by: jonesho <81145364+jonesho@users.noreply.github.com>
2025-02-19 11:07:23 +00:00
jonesho
67cf57cd2d feat: remove image build for l1-node-genesis-generator (#686)
* feat: use alpine as base image for 1-node-genesis-generator

* feat: remove image build for l1-node-genesis-generator in local stack

* chore: change genesis.json name

* feat: set timestamp in el genesis file as zero
2025-02-19 16:26:19 +08:00
Roman Vaseev
cd7228ecdc Updating tracer plugin and traces API (#705)
* Updating tracer plugin and traces API
2025-02-18 15:41:16 +01:00
jonesho
c7f6e5943e feat: use already-built image in Dockerfile of l1-node-genesis-generator (#669)
* feat: use already-built image in Dockerfile of l1-node-genesis-generator

* feat: get genesis time automatically in generate-genesis.sh and make L1_GENESIS_TIME optional

* feat: make sed in-place command portable with both OS in generate-genesis.sh
2025-02-14 03:12:59 +08:00
jonesho
0eb6eb4ff4 feat: use linea_estimateGas instead of getFeeData for all L2 transact… (#660)
* feat: use linea_estimateGas instead of getFeeData for all L2 transactions

* feat: use linea_estimateGas instead of getFeeData for L2 transactionswith call data and enable json-rpc-pricing-propagation

* feat: set DISABLE_JSON_RPC_PRICING_PROPAGATION of coordinator as false for e2e ci
2025-02-12 01:05:09 +08:00
Pedro Novais
bf8a00dfc6 update besu libs; improve gradle build; improve make file stack start up (#674) 2025-02-11 16:36:01 +00:00
The Dark Jester
4700e9078c [Chore] - use defined account for opcode testing (#671)
* use defined account for opcode testing

* add output to show execution
2025-02-11 05:57:10 -08:00
Pedro Novais
4d85082d33 Staterecovery adapt to new besu (#642)
* staterecovery: adapt to Besu 25.6-delivery46 and support forced stop sync for debug purposes
2025-02-06 10:28:03 +00:00
Pedro Novais
b7d7e45c6a staterecovery: remove goerli deprecated files (#657) 2025-02-05 15:40:14 +00:00
Pedro Novais
a63965e32c docker: breakdown into smaller files and more efficient resources (#626)
* docker: breakdown into smaller files and more efficient resources
2025-01-30 15:09:47 +00:00