This adds backends comparison in data extractor. It performs
comparison on a fixed list (CPU, GPU, HPU) for 64 bits precision
ciphertext as displayed in tfhe-rs public documentation.
SVG table generation is automated via the documentation benchmark
workflow.
The workflow might succeed even if it doesn't scan any parameters
set or a smaller one. In this case the analysis duration would be
much lower than usual. As being sent to a Slack channel, this
improved message would warn maintainers about a potential issue.
This new workflow can trigger all the required benchmarks needed
to populate benchmarks tables in documentation.
It also can generate SVG tables and store them as artifacts.
Optionally, it can open a pull-request to update the current
tables in documentation.
In case of a re-run attempt, if one asks for only failing jobs to
be re-run (e.g. MacOS job on GitHub hosted runner) and the
previous attempt was successful regarding setup-build-teardown,
then we would try to teardown again an instance already gone.
This would cause an error in the CI.
By checking the 'github.run_attempt' number, we ensure that we run
the teardown only if the setup has occurred during the same run
attempt.
Instance teardown might fail regardless the result of build
commands. To make CI green in pull-request we need to check the
result of the build commands instead of the whole workflow result.
Doing so ensure a PR could pass branch protection rules even if
the teardown fails.
It seems that boolean output set by user is not coerced into
boolean type by the runner. Rather it checks if the string
associated ("true" or "false") is not empty.
This is done to be able to execute CI in further development.
Also, we won't have to temporary lift the branch protection rules
to be able to merge since this upcoming development is a rework
cargo_build.yml workflow.
The steps responsible for setting the OP_FLAVOR and ALL_PRECISION
variables were never executed due to usage of non-existing env
variable.
This causes OP_FLAVOR value to be null and thus would trigger
error on benchmarks that doesn't handle unknown values for
BENCH_OP_FLAVOR.
Also fixes filename to parse for additional boolean benchmark.
Following the same pattern as GPU benchmarks, HPU benchmarks rely
on a common workflow. All the manual launches via
workflow_dispatch event are now done in one place. That way, one
doesn't have to browse the workflow tree to find the right HPU
benchmark to trigger.
Following the same pattern as GPU benchmarks, CPU benchmarks rely on a common workflow. Weekly benchmarks are all gathered in one place. Also, all the manual launches via workflow_dispatch event are now done in one place. That way, one doesn't have to browse the workflow tree to find the right CPU benchmark to trigger.
Signed-off-by: David Testé <david.teste@zama.ai>
