Commit Graph

89 Commits

Author SHA1 Message Date
David Testé
d2b20ca551 ci(bench): fix end-to-end benchmarks workflow
This includes several fixes and add some functionalities:
 * EC2 instance type can be selected when workflow is triggered manually
 * benchmarks will be run on each push on main branch
 * docker is not used any more due to building issues
 * 10 repetitions are made during the benchmarks then results are aggregated
 * more tags are used to identify benchmarks configuration
2022-10-18 12:02:35 +02:00
Andi Drebes
eae898b56f enhance(build): Add convenience targets LLVM / MLIR tools to toplevel Makefile
This adds four new targets `opt`, `mlir-cpu-runner`, `mlir-opt`, and
`mlir-translate` to the toplevel Makefile of the compiler to
conveniently build the corresponding LLVM / MLIR utilities (e.g., for
debugging purposes).
2022-10-11 17:09:47 +02:00
Quentin Bourgerie
fe7cdf9d11 chore: Go back to rc.5 to fix the mac os CI 2022-10-10 12:30:27 +02:00
Quentin Bourgerie
f2cbb1e719 chore: ./build as default build directory 2022-10-10 09:33:45 +02:00
David Testé
89d7f065ae ci(compiler): schedule end-to-end benchmarks
Run end-to-end benchmarks on a weekly basis. Results are parsed and
then sent to Slab CI bot to be later plotted into Grafana.
2022-10-07 14:45:56 +02:00
Ayoub Benaissa
6a5e65631c chore: update concrete-ffi version
no major breaking changes, just enhancements
check https://github.com/zama-ai/concrete-core/releases
2022-10-06 12:48:17 +01:00
Andi Drebes
e35c5bddcf fix(test): Add missing dependency from run-mlbench to generate-mlbench
The target `run-mlbench` indirectly depends on the contents of
`tests.ml/bench.zip` which are extracted by `generate-mlbench`. If
`generate-mlbench` has not been built pefore, `run-mlbench`
fails. This adds a the missing dependency from `run-mlbench` to
`generate-mlbench`.
2022-10-04 16:40:27 +02:00
Andi Drebes
85442f425d test(compiler): Move clear text tests to YAML test specification
This moves all tests from
`tests/end_to_end_tests/end_to_end_jit_clear_tensor.cc` to the test
specification in YAML format in
`tests/end_to_end_fixture/end_to_end_clear_tensor.yaml`. Parametric
tests and tests invoking lambdas in loops have been fully unrolled.
2022-10-04 14:40:40 +02:00
Andi Drebes
847e8607f3 enhance(compiler): Add support for debug builds in Makefile
This adds a new variable `BUILD_TYPE` to the Makefile, controlling
whether the build should be a debug or a release build (values `Debug`
and `Release`, respectively). The default mode is `Release`. Depending
on the build type, the build directory is set to `build-Debug` or
`build-Release`. This enables debug and release builds to co-exist and
to switch easily between the two.
2022-10-04 14:40:40 +02:00
youben11
628df8849c refactor: unify Makefile target naming style 2022-09-19 13:24:20 +01:00
youben11
8f05f9ea72 ci: change package release process
it should be now faster using the same compilation cache as the test
build.
2022-09-19 13:24:20 +01:00
Antoniu Pop
09184077d9 benchmarking: add YAML generator and ML benchmarks (see issue https://github.com/zama-ai/concrete-ml-internal/issues/1543). 2022-09-15 14:38:56 +01:00
youben11
0cd9a7984d feat: support dataflow execution in python packages
use latest image manylinux_2_28_x86_64 using libc 2.28
2022-09-13 15:18:15 +01:00
youben11
4b584ca64e ci: create new keysetcache
last commit updated which keys we serialize
2022-09-06 07:18:34 +01:00
youben11
661d33c2b6 feat: keep std bsk and conv to fourier when needed 2022-09-06 07:18:34 +01:00
Mayeul@Zama
eb4b0753dc enhance(runtime): Move to the official concrete-core-ffi release 0.2.0-rc2
Co-authored-by: Mayeul@Zama <mayeul.debellabre@zama.ai>
Co-authored-by: Quentin Bourgerie <bourgerie.quentin@gmail.com>
2022-08-30 18:08:47 +02:00
Quentin Bourgerie
46a914e181 chore(optimizer): Always call the makefile of the optimizer 2022-08-25 09:46:51 +02:00
Quentin Bourgerie
046c0ad5d9 chore: Fix makefile build-unit-tests target 2022-08-25 09:46:51 +02:00
Quentin Bourgerie
c08a06ed8e chore: Just rename configuration variable to enable the dataflow runtime 2022-08-24 11:35:05 +02:00
Quentin Bourgerie
d1694648c5 chore: Add a target to install hpx from source 2022-08-24 11:35:05 +02:00
Antoniu Pop
ba60d74da0 feat(dfr): add timing measurements. 2022-08-04 17:06:09 +01:00
Antoniu Pop
2f5f9f6cf1 test(dfr): add AES test. 2022-08-04 17:06:09 +01:00
Antoniu Pop
93d5a06557 test: add a test for distributed execution. 2022-08-04 17:06:09 +01:00
Antoniu Pop
8beb596fab test(dfr): remove DFR unittests which are now obsolete. 2022-08-04 17:06:09 +01:00
youben11
63d84a3e4a refactor: remove code related to our custom conv2D named op
it was introduced as a workaround while linalng couldn't support other
types than int/float
2022-08-04 11:19:18 +01:00
youben11
ff3a3cbe8b fix: fail tests if any of the test binaries fail 2022-07-27 10:29:47 +01:00
rudy
b12331634c fix(Makefile): Python3_EXECUTABLE need to be defined on some setup 2022-07-20 15:34:46 +02:00
Quentin Bourgerie
2cfccd8f89 refactor: restructure the tests directory layout 2022-07-07 17:10:09 +02:00
youben11
5668e447e7 feat: support python 3.7 2022-07-07 09:55:44 +01:00
Quentin Bourgerie
d0d98c8228 chore: Add build-benchmarks to the all target 2022-06-27 16:45:56 +02:00
Quentin Bourgerie
f06957a197 tests(compiler): Add a benchmark tool builded thanks google-benchmarks 2022-06-27 16:45:56 +02:00
Andi Drebes
45577fb79e Rebase onto llvm-project f69328049e9e with local changes
This commit rebases the compiler onto commit f69328049e9e from
llvm-project.

Changes:

* Use of the one-shot bufferizer for improved memory management

* A new pass `OneShotBufferizeDPSWrapper` that converts functions
  returning tensors to destination-passing-style as required by the
  one-shot bufferizer

* A new pass `LinalgGenericOpWithTensorsToLoopsPass` that converts
  `linalg.generic` operations with value semantics to loop nests

* Rebase onto a fork of llvm-project at f69328049e9e with local
  modifications to enable bufferization of `linalg.generic` operations
  with value semantics

* Workaround for the absence of type propagation after type conversion
  via extra patterns in all dialect conversion passes

* Printer, parser and verifier definitions moved from inline
  declarations in ODS to the respective source files as required by
  upstream changes

* New tests for functions with a large number of inputs

* Increase the number of allowed task inputs as required by new tests

* Use upstream function `mlir_configure_python_dev_packages()` to
  locate Python development files for compatibility with various CMake
  versions

Co-authored-by: Quentin Bourgerie <quentin.bourgerie@zama.ai>
Co-authored-by: Ayoub Benaissa <ayoub.benaissa@zama.ai>
Co-authored-by: Antoniu Pop <antoniu.pop@zama.ai>
2022-06-14 14:35:25 +02:00
Mayeul@Zama
ee64b68109 chore(optimier): use optimizer dependency as a submodule 2022-05-31 10:01:26 +02:00
Mayeul@Zama
f5faf4c421 feat(optimizer): update concrete_optimizer dependency
fix breaking changes
2022-05-18 14:04:48 +02:00
Mayeul@Zama
ba315f4cf0 compat(optimizer): use compat branch before merging breaking change to master 2022-05-18 11:27:16 +02:00
rudy
6f02d1248d feat(optimizer): update to use the new cpp brige 2022-05-10 15:36:40 +02:00
rudy
61052a237b fix(Makefile): concrete-optimizer, rm target dir from dependencies 2022-05-09 15:04:24 +02:00
rudy
eb1f5b1842 fix(keysetcache): use a portable hash function
will help to make macos faster on the ci
2022-05-06 10:00:11 +02:00
rudy
b45e469a44 feat(ci): macos, fix keysetcache use 2022-05-02 17:20:22 +02:00
rudy
923a1b58e1 feat(optimizer): plug online parameter estimation 2022-04-29 15:11:42 +02:00
youben11
0d043d0606 chore(tests): separate compiler out test from python
tests/python should include tests related to the python package
2022-04-08 19:05:06 +01:00
youben11
91ccea9b6b chore: remove external header target
there is no need to update the header file, and we already patched it in
925745f0e3 so new version won't work
unless patched
2022-04-08 17:18:18 +01:00
youben11
01f6a334ff chore: remove macos workaround for linking runtimelib 2022-04-04 11:52:23 +01:00
youben11
308504566f chore: add linting step with pylint 2022-04-04 09:15:31 +01:00
youben11
51308058c1 chore: format python code with black 2022-04-04 09:15:31 +01:00
rudy
2c4d399575 feat(KeySetCache): use a artifacts to store the cache 2022-03-30 10:11:38 +02:00
Quentin Bourgerie
6717c4f5ff chore(test): Missing lambda tests in test target 2022-03-24 11:00:36 +01:00
Quentin Bourgerie
e5cec23868 chore(test): Missing test-end-to-end-jit-fhe to test-end-to-end-jit target 2022-03-24 11:00:36 +01:00
youben11
5b37ec640c feat: support parallelization in python 2022-03-17 16:15:02 +01:00
Mayeul@Zama
ca8d4fb110 feat(compiler): use engine concrete C API
remove ConcreteToConcreteCAPI and ConcreteUnparametrize passes
2022-03-15 18:14:35 +01:00