Commit Graph

99 Commits

Author SHA1 Message Date
Luis Montero
aad2a42634 chore(ffi): support ARM64 chips 2022-11-17 01:04:36 +01:00
Samuel Tap
8a231974f7 chore(ffi): update to concrete-ffi without fftw 2022-11-16 11:44:46 +01:00
youben11
472e762fbf feat(rust): enhance API to create func with FHE dialect 2022-11-09 12:53:25 +01:00
Quentin Bourgerie
84041557ab enhance(bench): Add benchmark generator for leveled operators 2022-11-08 14:27:30 +01:00
David Testé
d1db4a5e45 tests(benchmarks): reduce number of test cases to speed-up execution
Bench just one compilation option for automatic benchmarks. Only 'loop'
option is tested to take advantage of hardware with a lot of available
CPUs. Running benchmarks with 'default' option is suboptimal for this
kind of hardware since it uses only one CPU.

This also remove time consuming MNIST test, as it should be in ML benchmarks.

Moreover Makefile is fixed to use provided Python executable instead of
relying on system one to generate MLIR Yaml files.
2022-11-03 19:07:39 +01:00
youben11
0ac21fd037 feat: add rust bindings
The rust bindings are intented to access both LLVM/MLIR CAPI as well as
the concrete-compiler one. This initial commit provide the API for
LLVM/MLIR only. Tests should be used as an example to how to generate a
valid DAG of operations in MLIR.
2022-11-02 14:46:38 +01:00
Antoniu Pop
746b7c2641 fix(dataflow build): fix HPX installation path when built and installed locally. 2022-10-25 09:50:51 +01:00
Quentin Bourgerie
a5047586f4 test(benchmark): Add more micro benchmarks on parallel lookup table 2022-10-20 10:36:32 +01:00
youben11
7cd45d1514 test: add GPU end2end tests 2022-10-20 10:36:32 +01:00
youben11
7e08614e6c build: add options to build with CUDA support 2022-10-20 10:36:32 +01:00
David Testé
d2b20ca551 ci(bench): fix end-to-end benchmarks workflow
This includes several fixes and add some functionalities:
 * EC2 instance type can be selected when workflow is triggered manually
 * benchmarks will be run on each push on main branch
 * docker is not used any more due to building issues
 * 10 repetitions are made during the benchmarks then results are aggregated
 * more tags are used to identify benchmarks configuration
2022-10-18 12:02:35 +02:00
Andi Drebes
eae898b56f enhance(build): Add convenience targets LLVM / MLIR tools to toplevel Makefile
This adds four new targets `opt`, `mlir-cpu-runner`, `mlir-opt`, and
`mlir-translate` to the toplevel Makefile of the compiler to
conveniently build the corresponding LLVM / MLIR utilities (e.g., for
debugging purposes).
2022-10-11 17:09:47 +02:00
Quentin Bourgerie
fe7cdf9d11 chore: Go back to rc.5 to fix the mac os CI 2022-10-10 12:30:27 +02:00
Quentin Bourgerie
f2cbb1e719 chore: ./build as default build directory 2022-10-10 09:33:45 +02:00
David Testé
89d7f065ae ci(compiler): schedule end-to-end benchmarks
Run end-to-end benchmarks on a weekly basis. Results are parsed and
then sent to Slab CI bot to be later plotted into Grafana.
2022-10-07 14:45:56 +02:00
Ayoub Benaissa
6a5e65631c chore: update concrete-ffi version
no major breaking changes, just enhancements
check https://github.com/zama-ai/concrete-core/releases
2022-10-06 12:48:17 +01:00
Andi Drebes
e35c5bddcf fix(test): Add missing dependency from run-mlbench to generate-mlbench
The target `run-mlbench` indirectly depends on the contents of
`tests.ml/bench.zip` which are extracted by `generate-mlbench`. If
`generate-mlbench` has not been built pefore, `run-mlbench`
fails. This adds a the missing dependency from `run-mlbench` to
`generate-mlbench`.
2022-10-04 16:40:27 +02:00
Andi Drebes
85442f425d test(compiler): Move clear text tests to YAML test specification
This moves all tests from
`tests/end_to_end_tests/end_to_end_jit_clear_tensor.cc` to the test
specification in YAML format in
`tests/end_to_end_fixture/end_to_end_clear_tensor.yaml`. Parametric
tests and tests invoking lambdas in loops have been fully unrolled.
2022-10-04 14:40:40 +02:00
Andi Drebes
847e8607f3 enhance(compiler): Add support for debug builds in Makefile
This adds a new variable `BUILD_TYPE` to the Makefile, controlling
whether the build should be a debug or a release build (values `Debug`
and `Release`, respectively). The default mode is `Release`. Depending
on the build type, the build directory is set to `build-Debug` or
`build-Release`. This enables debug and release builds to co-exist and
to switch easily between the two.
2022-10-04 14:40:40 +02:00
youben11
628df8849c refactor: unify Makefile target naming style 2022-09-19 13:24:20 +01:00
youben11
8f05f9ea72 ci: change package release process
it should be now faster using the same compilation cache as the test
build.
2022-09-19 13:24:20 +01:00
Antoniu Pop
09184077d9 benchmarking: add YAML generator and ML benchmarks (see issue https://github.com/zama-ai/concrete-ml-internal/issues/1543). 2022-09-15 14:38:56 +01:00
youben11
0cd9a7984d feat: support dataflow execution in python packages
use latest image manylinux_2_28_x86_64 using libc 2.28
2022-09-13 15:18:15 +01:00
youben11
4b584ca64e ci: create new keysetcache
last commit updated which keys we serialize
2022-09-06 07:18:34 +01:00
youben11
661d33c2b6 feat: keep std bsk and conv to fourier when needed 2022-09-06 07:18:34 +01:00
Mayeul@Zama
eb4b0753dc enhance(runtime): Move to the official concrete-core-ffi release 0.2.0-rc2
Co-authored-by: Mayeul@Zama <mayeul.debellabre@zama.ai>
Co-authored-by: Quentin Bourgerie <bourgerie.quentin@gmail.com>
2022-08-30 18:08:47 +02:00
Quentin Bourgerie
46a914e181 chore(optimizer): Always call the makefile of the optimizer 2022-08-25 09:46:51 +02:00
Quentin Bourgerie
046c0ad5d9 chore: Fix makefile build-unit-tests target 2022-08-25 09:46:51 +02:00
Quentin Bourgerie
c08a06ed8e chore: Just rename configuration variable to enable the dataflow runtime 2022-08-24 11:35:05 +02:00
Quentin Bourgerie
d1694648c5 chore: Add a target to install hpx from source 2022-08-24 11:35:05 +02:00
Antoniu Pop
ba60d74da0 feat(dfr): add timing measurements. 2022-08-04 17:06:09 +01:00
Antoniu Pop
2f5f9f6cf1 test(dfr): add AES test. 2022-08-04 17:06:09 +01:00
Antoniu Pop
93d5a06557 test: add a test for distributed execution. 2022-08-04 17:06:09 +01:00
Antoniu Pop
8beb596fab test(dfr): remove DFR unittests which are now obsolete. 2022-08-04 17:06:09 +01:00
youben11
63d84a3e4a refactor: remove code related to our custom conv2D named op
it was introduced as a workaround while linalng couldn't support other
types than int/float
2022-08-04 11:19:18 +01:00
youben11
ff3a3cbe8b fix: fail tests if any of the test binaries fail 2022-07-27 10:29:47 +01:00
rudy
b12331634c fix(Makefile): Python3_EXECUTABLE need to be defined on some setup 2022-07-20 15:34:46 +02:00
Quentin Bourgerie
2cfccd8f89 refactor: restructure the tests directory layout 2022-07-07 17:10:09 +02:00
youben11
5668e447e7 feat: support python 3.7 2022-07-07 09:55:44 +01:00
Quentin Bourgerie
d0d98c8228 chore: Add build-benchmarks to the all target 2022-06-27 16:45:56 +02:00
Quentin Bourgerie
f06957a197 tests(compiler): Add a benchmark tool builded thanks google-benchmarks 2022-06-27 16:45:56 +02:00
Andi Drebes
45577fb79e Rebase onto llvm-project f69328049e9e with local changes
This commit rebases the compiler onto commit f69328049e9e from
llvm-project.

Changes:

* Use of the one-shot bufferizer for improved memory management

* A new pass `OneShotBufferizeDPSWrapper` that converts functions
  returning tensors to destination-passing-style as required by the
  one-shot bufferizer

* A new pass `LinalgGenericOpWithTensorsToLoopsPass` that converts
  `linalg.generic` operations with value semantics to loop nests

* Rebase onto a fork of llvm-project at f69328049e9e with local
  modifications to enable bufferization of `linalg.generic` operations
  with value semantics

* Workaround for the absence of type propagation after type conversion
  via extra patterns in all dialect conversion passes

* Printer, parser and verifier definitions moved from inline
  declarations in ODS to the respective source files as required by
  upstream changes

* New tests for functions with a large number of inputs

* Increase the number of allowed task inputs as required by new tests

* Use upstream function `mlir_configure_python_dev_packages()` to
  locate Python development files for compatibility with various CMake
  versions

Co-authored-by: Quentin Bourgerie <quentin.bourgerie@zama.ai>
Co-authored-by: Ayoub Benaissa <ayoub.benaissa@zama.ai>
Co-authored-by: Antoniu Pop <antoniu.pop@zama.ai>
2022-06-14 14:35:25 +02:00
Mayeul@Zama
ee64b68109 chore(optimier): use optimizer dependency as a submodule 2022-05-31 10:01:26 +02:00
Mayeul@Zama
f5faf4c421 feat(optimizer): update concrete_optimizer dependency
fix breaking changes
2022-05-18 14:04:48 +02:00
Mayeul@Zama
ba315f4cf0 compat(optimizer): use compat branch before merging breaking change to master 2022-05-18 11:27:16 +02:00
rudy
6f02d1248d feat(optimizer): update to use the new cpp brige 2022-05-10 15:36:40 +02:00
rudy
61052a237b fix(Makefile): concrete-optimizer, rm target dir from dependencies 2022-05-09 15:04:24 +02:00
rudy
eb1f5b1842 fix(keysetcache): use a portable hash function
will help to make macos faster on the ci
2022-05-06 10:00:11 +02:00
rudy
b45e469a44 feat(ci): macos, fix keysetcache use 2022-05-02 17:20:22 +02:00
rudy
923a1b58e1 feat(optimizer): plug online parameter estimation 2022-04-29 15:11:42 +02:00