From e957f63b812ea2d2904776a97b124328c0b20043 Mon Sep 17 00:00:00 2001 From: Benoit Chevallier-Mames Date: Mon, 20 Sep 2021 11:58:24 +0200 Subject: [PATCH] doc: add a benchmarks section closes #423 --- docs/benchmarks.md | 18 ++++++++++++++++++ docs/index.rst | 3 ++- 2 files changed, 20 insertions(+), 1 deletion(-) create mode 100644 docs/benchmarks.md diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 000000000..867d652e2 --- /dev/null +++ b/docs/benchmarks.md @@ -0,0 +1,18 @@ +# Benchmarks + +In order to track our progress over time, we have set a [public benchmark](https://progress.zama.ai) containing: +- a list of functions that we want to compile +- status on the compilation of these functions +- compilation time +- evaluation time on different hardware's +- accuracy of the functions for which it makes sense +- loss of the functions for which it makes sense + +Remark that we are not limited to these, and we'll certainly add more information later, as key generation time, encryption and decryption time, and more evaluation time once the explicit inference API is available. + +The benchmark can be used by competitive frameworks or technologies, in order to compare fairly with the **Concrete Framework**. Notably, one can see: +- if the same functions can be compiled +- what are discrepancies in the exactness of the evaluations +- how do evaluation times compare + +If one wants to see more functions in the benchmark or if there is more information you would like the benchmark to track, don't hesitate to drop an email to . \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index 967bb94d7..e75fb5d81 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -4,10 +4,11 @@ Concrete Framework's documentation .. toctree:: :maxdepth: 2 - :caption: Basics + :caption: Getting Started README.md user/howto/INSTALLING.md + benchmarks.md .. toctree:: :maxdepth: 2