152 Commits

Author SHA1 Message Date
PearsonWhite
4fbe63bf18 Add deploy to BaseExperiment and add --dry-run (#132)
* Move some logic for deploying to BaseExperiment

* Add --dry-run to BaseExperiment and kube_utils
2025-09-22 11:49:56 -04:00
PearsonWhite
4d7d9ea1e3 Make _metadata_event use dict events (#131)
* Make _metadata_event use dict events

* Looks for events by comparing dicts

* Separate out parsing logic for metadata event

* Make class method _get_metadata_event

* typing
2025-09-11 13:22:07 -04:00
PearsonWhite
acdafe0586 Add cls.name when registering experiments (#136) 2025-09-11 13:21:02 -04:00
PearsonWhite
0a8b0ce735 Make --config param optional (#135) 2025-09-11 13:17:28 -04:00
PearsonWhite
eb48287958 Add lightpush client and getAddress fixes (#133)
* Add lightpush client and getAddress fixes

* Add lightpush client values in command presets

* Combined _getEnr.tpl and _getAddress.tpl into one template

* Fix some indents in nodes.yaml

* Remove unused lightpush.values.yaml

* Rename getEorA to getEnrOrAddress
2025-08-22 09:53:24 -04:00
PearsonWhite
71cf504d28 Organize experiment output better (#134)
* Organize experiment output better

* Remove --out-log flag

* Automatically pipe logging to out_folder/out.log

* Default out_folder to current datetime under ./out

* Copy values file into out_folder/cli_values.yaml

* Update README
2025-08-22 09:18:10 -04:00
Alberto Soutullo
e74babf6c5 Replace requirements.txt with uv and cleanup (#130)
* Removed mesh_creation from the repo

* Removed visualizer from the repo

* Removed old python files

* Rename kubernetes.py file to avoid clashes with kubernetes library import

* Update kubernetes imports

* Add uv files

* Update README.md

* Update pyproject.toml

Co-authored-by: PearsonWhite <PearsonWhite602@gmail.com>

* Delete old requirements.txt

---------

Co-authored-by: PearsonWhite <PearsonWhite602@gmail.com>
2025-08-19 15:30:21 +02:00
Alberto Soutullo
1bb251d6a2 Parse all special characters, not only whitespaces (#128) 2025-08-13 17:37:08 +02:00
Alberto Soutullo
4ded69bdfc Update getenr (#127)
* Update bash script

* Update build script version

* Update README.md

* Addressed review comments
2025-08-13 17:36:40 +02:00
PearsonWhite
2be0c1f05a Reorganize repo and update documentation (#124)
- Move old experiment shell scripts into deployment-utilities/experiment_scripts/

- Gather all waku kubernetes services in deployment-utilities/kubernetes-utilities/waku/zerotesting-services

- Rename ./experiments/ -> ./deployments/

- Move waku cleanup.sh with other shell scripts in
  deployment-utilities/experiment_scripts/waku

- Move analysis files to <root>/analysis/

- Add README.md to root folder

- Cleanup unused experiments/regression_tests/

- Update READMEs
2025-08-12 12:58:25 -04:00
PearsonWhite
84e701440c Add events logging for waku regression experiment (#78) 2025-08-12 10:27:52 -04:00
PearsonWhite
e50f445c90 Make bootstrap replicas a variable (#77) 2025-08-05 09:32:33 -04:00
PearsonWhite
f1899a3005 Changed plot stat labels use 'scale-x' instead of hardcoded 0.001 (#76) 2025-07-15 09:43:10 -04:00
PearsonWhite
8f2e9d6a98 Add get_filter_messages and get_store_messages. (#75)
* Add get_filter_messages and get_store_messages.

* Add option to not hyphenate commands in _args.tpl

* Add _command.tpl for generic commands
2025-07-07 11:21:23 -04:00
PearsonWhite
1603e9ad2b Update publisher lightpush API from v1 to v3. (#73)
* Update publisher lightpush API from v1 to v3.

* Added working versions and changed exception.
2025-07-03 15:37:49 -04:00
PearsonWhite
1477962c74 Make regression experiments self contained (#74)
Move .values.yaml files from node/publisher/etc to experiment folder.
2025-07-03 15:37:31 -04:00
PearsonWhite
2d469761f6 Adapt regression tests to new deployment scaffold (#72)
* Adapt regression tests to new deployment scaffold

* Fix formatting for _metrics.tpl and add `nice -n <int>` option in _waku_container_command.tpl.
2025-06-26 10:20:32 -04:00
Alberto Soutullo
4d219d4e0d Add mix intermediate log tracing analysis (#71)
* Add mix analysis to nimlibp2p_tracer

* Add analyze_mix_trace to Nimlibp2pAnalyzer

* Refactor _assert_num_nodes and calls

* Remove Exit and change log to error
2025-06-26 15:26:10 +02:00
PearsonWhite
0c3aded84f Refactor python scaffold MVP #70
Minimum Viable Product for scaffold refactor.

Changes stucture of scaffold to:
- Run experiments by building from different types of services
- Copy whole folder to workdir for more organized debugging
- Use decorator to scan for experiment and add subparsers
2025-06-19 08:52:57 -04:00
Pearson White
1d64902ebc Refactor python scaffold MVP
Minimum Viable Product for scaffold refactor.

Changes stucture of scaffold to:
- Run experiments by building from different types of services
- Copy whole folder to workdir for more organized debugging
- Use decorator to scan for experiment and add subparsers
2025-06-18 13:05:19 -04:00
PearsonWhite
059e361a10 Add .gitignore (#67) 2025-06-17 10:06:31 +02:00
Alberto Soutullo
877e1ee36a Analysis refactor (#66)
* Add base class for monitoring client configurations

* Add victoria metrics monitoring client configurations

* WIP: Minor refactor to readers, add extra fields to decouple from an always K8s scenario

* WIP: Move tracers inside readers, as it depends on Victoria. Start decoupling tracing from analysis.

* WIP: Create nimlibp2p tracer

* WIP: Create statusgo tracer

* WIP: Start creating stack classes, depending on the monitoring stack used, analysis will be done differently.

* WIP: Decouple analysis from stack. Add extra plot functions.

* WIP: Delete monitoring clients

* WIP: Add number of processes to File Reader

* WIP: Add number of processes Stack Analysis

* WIP: Decouple patterns with extra arguments from cluster

* WIP: Move waku analyzer

* WIP: Add container name as parameter

* WIP: Correct query selection when doing query

* WIP: Correct dispatching call in waku analyzer

* WIP: Move waku plotting to its own file

* WIP: Refactor in File Reader

* WIP: Minor refactor in victoria reader loop

* WIP: Add info to trace method

* WIP: Add builder to base stack analysis class

* WIP: Change fields in waku tracer

* WIP: Refactor due to previous change

* WIP: Extract ownership of parsing logs to upper class for better decoupling

* WIP: Move functions to different class

* WIP: Move extra fields to Victoria builder, from kwargs.

* WIP: Add chainable instantiation pattern to tracers

* WIP: Renamed method to match standard naming

* WIP: Remove old TODO

* WIP: Add notes for queries and patterns

* WIP: Remove stack from class variable and refactor get number of nodes

* WIP: Add victoria reader builder

* WIP: Log pod name instead of peer id for better debugging

* WIP: Change logging options

* WIP: Change read_logs function name to get_dataframes

* WIP: Make VictoriaReader implement Reader class

* WIP: Allow _make_queries to work with a single query in params

* WIP: Allow _make_queries to work with a single query in params

* WIP: Add comments to victoria reader

* WIP: Improve VictoriaReaderBuilder

* WIP: Download logs from malfunctioning pods

* WIP: Updated check_store_messages and check_filter_messages

* WIP: Add comments to Waku analyzer

* WIP: Improve WakuTracer

* WIP: Improve how vaclab_stack_analysis.py was handling log retrieval

* WIP: Rename functions in stack_analysis.py

* WIP: Renamed Plotter to MetricsPlotter

* WIP: Minor changes to waku_plots

* WIP: Minor changes to scrapper

* WIP: Rewrite README.md

* WIP: Add image for diagram

* WIP: Moved information to new readme

* WIP: Update example file

* WIP: Add readme for plotting metrics

* WIP: Expand nimlibp2p_analyzer.py

* WIP: Expand nimlibp2p_tracer.py

* Expand example_log_analysis.py
2025-06-06 15:35:41 +02:00
PearsonWhite
994f3e6034 Create Python deployment scaffold (#68) (#68) 2025-05-29 17:46:10 +02:00
PearsonWhite
74a4b30782 Add zerotesting service kubernetes configs (#69) 2025-05-27 21:38:07 +02:00
Alberto Soutullo
371c071e48 Update waku regression example (#65) 2025-05-13 14:52:34 +02:00
Alberto Soutullo
43f6c07dd2 Topology generator (#64)
* Add current state of topology generator

* Refactor topology generator

* Refactor readme

* Add tests

* Add tests
2025-03-17 13:00:58 +01:00
Alberto Soutullo
9be2a2be74 Add pod name to log error (#63) 2025-03-10 09:55:52 +01:00
Alberto Soutullo
2ba8307e1e Lightpush dataframe (#62)
* Add lightpush query to victoria config

* Modify patterns to match victoria config

* Modify trace to match patterns

* Add _trace_lightpush_in_logs

* Modify merge dfs function to match previous changes

* Adapt query tracing to match previous changes
2025-03-04 11:03:12 +01:00
Alberto Soutullo
c5d230488b Fix log information (#61) 2025-02-20 00:01:54 +01:00
Alberto Soutullo
ee601b686b Scale value in plot function util (#58)
* Move function and add scale_by argument

* Change imports and parameteres
2025-02-17 16:43:18 +01:00
Alberto Soutullo
47927a25ea Update publisher (#59)
* Update traffic.py

* Update README.md

* Update build script version
2025-02-17 16:43:06 +01:00
Alberto Soutullo
b511cf635e Update logging when parsing pods. (#60) 2025-02-13 21:44:47 +01:00
Alberto Soutullo
febe21a326 Set outliers as config. (#57) 2025-02-07 11:14:21 +01:00
Alberto Soutullo
9e924589e9 Fix README.md link (#56) 2025-02-04 17:08:01 +01:00
Alberto Soutullo
1573417eac Deployment utils (#54)
* Delete midstrap yaml

* Add yamls from experiments

* Delete old yamls
2025-02-04 17:07:42 +01:00
Alberto Soutullo
197d3c9ad6 Cleaning and documenting repo (#52)
* Delete old scripts

* Moved waku publisher under docker folder

* Moved waku get multiaddress under docker folder

* Fixed mistakes in get_address README

* Moved waku get enr under docker folder

* Moved waku get filter retriever under docker folder

* Moved waku get store messages under docker folder

* Moved waku get publisher under docker folder

* Moved debug image under docker folder

* Removed old dockerfile

* Updated main README

* Removed nimlibp2p folder
2025-01-31 12:15:43 +01:00
Alberto Soutullo
6da581c550 Violin distribution comparison (#51)
* Change _validate_path to private function in order to create different decorators

* Delete wrong code

* Add check_params_path_exists_by_position

* Add check_patams_path_exists_by_position_or_kwargs

* Refactor rime to reach value plot
2025-01-23 12:38:26 +01:00
Alberto Soutullo
8aa5bea98d Update queries (#50) 2025-01-21 18:55:17 +01:00
Alberto Soutullo
176a610d70 Connection & Time to reach healthy distribution (#48)
* Create class decorator for check if path exists.

* Create plot function to use for analysis of reach Nº of connections
2025-01-21 16:12:23 +01:00
Anton Iakimov
8f47c8f95b vacdst host rename (#49) 2025-01-20 17:24:12 +01:00
Alberto Soutullo
d1b9c24586 Message distribution plot improvement (#47)
* Create function to add data to boxplot in plot/utils module

* Refactor plotter.py to use previously created function

* Enhance plot_message_distribution function.
2025-01-20 17:22:06 +01:00
Alberto Soutullo
d76b0f9412 Mixed improvements for analysis (#46)
* Delete repeated docker utility

* Add shards for filter message retriever script

* Fix default content topic for store message retriever

* Change service to query so store message retriever can query any relay node

* Reduce complexity of publisher since now shards are implicit in stateful set names

* Delete unnecessary sleeps

* Add shard to dataframe, and infer the shard from the stateful set name

* Refactor code to match changes from sharding

* Add helper function to plot message distribution time

* Improve logging when a peer missed messages by showing the message hash

* Delete unnused import

* Update example file with the newest changes
2025-01-12 12:15:36 +01:00
Alberto Soutullo
e2d9b5746e Shard improvements for analysis (#45)
* Improve publisher to work with shards including lightpush

* Append kubernetes_host_name to the dataframe information

* Use number of shards and number of nodes as a class internal data

* fixup! Append kubernetes_host_name to the dataframe information

* Add shard to missing message checking

* Fix missing keyword in get files from local path

* Add extra information in log line

* Remove duplicated dump information

* Add code to check if a message has appeared in multiple shards

* Update python docker image for publisher
2024-12-19 16:27:23 +01:00
Alberto Soutullo
3033cd6cbd Debug Visualizer PR (#44)
* Rewrite visualizer in Dash instead of using jupyter notebook

* Update requirements.txt

* Delete old visualizer

* Add README.md with image
2024-12-11 22:44:19 +01:00
Alberto Soutullo
17a6efd121 Filter lightpush (#43)
* Change get enr container to use rest API and add service selection

* Add protocols selection, port and sharding to traffic script

* Improve victoria query

* Add function to check filter messages from get-filter-messages container

* Filter lightpush-servers until they don't have propper logging

* Create docker image to get addresses instead of enrs

* Create docker image to get messages from filter nodes

* Fix typo in comment

* Mark filter and lightpush as done
2024-11-04 09:22:08 +01:00
Alberto Soutullo
f51ad0b10f documentation + store (#40)
* Set main to work with current code

* Add yaml explanation + mark store as ready

* Clean scrape yaml

* Add yaml for store nodes

* Add container for get store messages

* Update README

* Add more logging to store_msg_retriever.py

* Rename list to list_utils and add function to dump list to file

* Rename import from list to list_utils

* Take into account container name for some metrics

* Dump received and send info as csv

* Add function to check messages received with store archived

* Refactor get_store_messages
2024-10-17 22:19:08 +02:00
Alberto Soutullo
6aefa6d91e Minor plot improvements (#41)
* Add more parameters to yaml configuration

* Refactor function to also show min and max values in boxplots
2024-10-02 13:29:06 +02:00
Alberto Soutullo
ae5142f119 Message store check (#39)
* Add container for store message retrieval

* Add store analysis example

* Add method to check store messages

* Fix bug when using cursor
2024-09-08 19:57:20 +02:00
Benjamin Arntzen
6df3860386 Create README.md (#38) 2024-09-05 10:14:32 +02:00
Alberto Soutullo
689795bede Divided prepare path for file and folder (#37) 2024-09-05 10:13:50 +02:00