Commit Graph

5 Commits

Author SHA1 Message Date
David Testé
99082d1473 chore(ci): use name field instead of run_name in benchmark results
In case of benchmark_repetitions > 1 using 'run_name' will return
always the same name for a given benchmark case. It won't make the
distinction between the iterations and the aggregated values like
mean, median or stdev. Using 'name' field fix this behavior.
2022-12-20 13:14:44 +01:00
David Testé
1e8c0df381 chore(ci): change benchmark parser input name
The use of "schema" was incorrect since it's meant to be used as
database name when sending data to Slab.
2022-11-23 16:22:17 +01:00
David Testé
77729514e0 chore(ci): parse benchmark results to send to postgres instance
Previousely, we were sending parsed benchmark results to a
Prometheus instance. Do to its time-series nature, Prometheus would
downsample database content to avoid having to much data points
for a given range of time. While this behavior is good for a
continuous stream of data, like monitoring CPU load, it's not suited
for benchmarks. Indeed benchmarks are discrete events that would
occurr once in a while (i.e once a day). Downsampling would, at
some point, simply omit some of benchmarks results.
Using a regular SQL database like PostgreSQL solves this issue.
2022-11-15 16:18:25 +01:00
David Testé
d1db4a5e45 tests(benchmarks): reduce number of test cases to speed-up execution
Bench just one compilation option for automatic benchmarks. Only 'loop'
option is tested to take advantage of hardware with a lot of available
CPUs. Running benchmarks with 'default' option is suboptimal for this
kind of hardware since it uses only one CPU.

This also remove time consuming MNIST test, as it should be in ML benchmarks.

Moreover Makefile is fixed to use provided Python executable instead of
relying on system one to generate MLIR Yaml files.
2022-11-03 19:07:39 +01:00
David Testé
89d7f065ae ci(compiler): schedule end-to-end benchmarks
Run end-to-end benchmarks on a weekly basis. Results are parsed and
then sent to Slab CI bot to be later plotted into Grafana.
2022-10-07 14:45:56 +02:00