Commit Graph

12 Commits

Author SHA1 Message Date
David Testé
238f7d51f6 chore(ci): fix data extraction on backends comparison for docs 2025-12-12 12:24:47 +01:00
David Testé
e969fb5a1a refactor(ci): make data extractor formatters modular on layers
This declutters formatter.py file and would help to add formatting
cases that are specific to a layer like ZK-PoK benchmarks on the
integer layer.
2025-12-11 09:35:56 +01:00
David Testé
182aad99f1 chore(ci): add backends comparison table generation for docs
This adds backends comparison in data extractor. It performs
comparison on a fixed list (CPU, GPU, HPU) for 64 bits precision
ciphertext as displayed in tfhe-rs public documentation.

SVG table generation is automated via the documentation benchmark
workflow.
2025-12-04 17:59:27 +01:00
David Testé
87bea9dcf1 chore(ci): remove 2m40 p-fail from core_crypto array generation 2025-11-24 14:03:08 +01:00
David Testé
3c76dd8cad chore(ci): small fixes on data_extractor filename generation
This is done to ease automated SVG tables for tfhe-rs public documentation.
2025-11-24 14:03:08 +01:00
David Testé
c5bac7994a chore(ci): fix typos in newly checked ci folder 2025-11-19 11:51:02 +01:00
David Testé
0c977a3996 chore(bench): insert params name in bench id for hlapi
To ease parsing and filtering by third parties.
2025-11-04 10:53:25 +01:00
David Testé
e17c481736 chore(ci): prefix regression ops results with layer name
This is done to avoid confusion for operations that might have the same between layer. For example, 'bitand' operation have the same name for shortint and integer layers
2025-10-30 09:51:44 +01:00
David Testé
2542ef38e6 chore(ci): add parameters filtering for data extractor
When doing regression generation, one can provide a global parameters set name pattern to filter head branch benchmark results.
This fixes the issue encountered when there are more than one parameters' set used to benchmark an operation, for example, in core_crypto or shortint tfhe-rs layer.
2025-10-30 09:51:44 +01:00
David Testé
67130646ad chore(ci): support shortint layer name parsing in data extractor 2025-10-29 15:33:33 +01:00
David Testé
fd6323b311 chore(ci): add throughput and hpu support to data extractor
Now throughput results can be fetched.
HPU backend is supported for integer formatting
2025-10-27 14:39:46 +01:00
David Testé
206553e9ee chore(ci): check for performance regression and create report
After running performances regression benchmarks, a performance
changes checking is executed. It will fetch results data with an
external tool then it will look for anomaly in changes.
Finally it will produce a report as an issue comment with any
anomaly display in a Markdown array. A folded section of the
report message contains all the results from the benchmark.

Note that a fully custom benchmark triggered from an issue comment
would not generate a report. In addition HPU performance
regression benchmark is not supported yet.
2025-10-17 15:05:24 +02:00