Commit Graph

122 Commits

Author SHA1 Message Date
David Testé
ea5000aecb chore(ci) fix build on aws ec2
Use the right make rule target for testing the GPU and CPU.
2023-01-11 17:39:05 +01:00
tmontaigu
add68ccf84 fix(Makefile): update install target to work on macOS 2023-01-10 18:05:46 +01:00
rudy
a2c6763d80 fix: make all builds rust-bindings 2022-12-22 20:08:28 +01:00
rudy
b742ac35ae fix(ci): force crt for 9, 10, 11 for table lookup 2022-12-13 13:21:30 +01:00
Quentin Bourgerie
37c2627feb chore(ci): Regenerate the keyset cache 2022-12-12 10:38:44 +01:00
Quentin Bourgerie
07aa334d8d chore(benchmarks): Refactor the benchmark tools 2022-12-09 09:51:23 +01:00
youben11
65235408bc chore: build rust package using install dir 2022-12-08 07:45:55 +01:00
youben11
ff4a0076a1 ci: fix release tarball process
add install target in Makefile to copy necessary libs, bins, and
includes to an installation directory. Use this install target to
package deps into a tarball with new installation instructions.
2022-12-08 07:45:55 +01:00
Quentin Bourgerie
9cf506b0aa chore(tests): Add testing for gpu 2022-12-07 21:32:01 +01:00
Quentin Bourgerie
351bab0bc9 chore(tests): Add a make target for testing gpu 2022-12-07 21:32:01 +01:00
Quentin Bourgerie
71b24b1255 chore(ci/bench): Generate gpu benchmarks 2022-12-07 21:32:01 +01:00
Quentin Bourgerie
11cef2086d chore(cuda): Add concrete-core as submodule in order to integrate concrete-cuda as a cmake subproject 2022-12-07 21:32:01 +01:00
tmontaigu
188642b153 chore(ci): add rustfmt check 2022-12-07 13:55:02 +01:00
rudy
75dd26d0ab feat: bench do not keygen on all keys 2022-12-05 13:57:54 +01:00
Quentin Bourgerie
07c1e8347e refactor(tests): Refactor test tools to easiest run tests with description files and different compilation options 2022-12-05 13:57:54 +01:00
Quentin Bourgerie
0f88dd4bcb fix(ci): Fixing mac os ci 2022-11-30 11:17:48 +01:00
Quentin Bourgerie
fccb6da5b1 fix(check-tests): Just run llvm-lit in check_tests subdirectory 2022-11-30 10:29:19 +01:00
Quentin Bourgerie
6eb4cec706 fix(wopbs): Fixing woppbs with tensor operators (close #789)
- Missing offset in woppbs routine
- Better error message for check of tensor result in end to end fixture
- Modify fixture generator for testing purpose
2022-11-30 10:02:07 +01:00
rudy
ddb2b7fa58 fix(ci): more robust keysetcache download 2022-11-29 09:28:57 +01:00
Luis Montero
30be8cf4ae fix: fix MacOS test launching
* Update makefile to use appropriate arguments to find executables on
  macos systems.
* Add `set -e` to make sure that the tests crash if something goes wrong
  in the build or test of the macos job of the CI.

closes #783
2022-11-25 09:48:45 +01:00
youben11
7f65f4f527 fix(ci): set env var to look for lib in build/lib
LD_LIBRARY_PATH is set to BUILD_DIR/lib during rust tests in order to
find the runtime lib which it depends on
2022-11-23 14:01:25 +01:00
youben11
5661b758d7 feat(CAPI): add initial API to do round-tripping with CompilerEngine 2022-11-23 14:01:25 +01:00
David Testé
3c2a75186f chore(ci): run ml benchmarks in a matrix with slab
This CI "feature" is meant to circumvent the 6 hours hard-limit
for a job in GitHub Action.
The benchmark is done using a matrix which is handled by Slab.
Here's the workflow:

  1. ML benchmarks are started in a fire and forget fashion via
     start_ml_benchmarks.yml
  2. Slab will read ci/slab.toml to get the AWS EC2 configuration
     and the matrix parameters
  3. Slab will launch at most max_parallel_jobs EC2 instances in
     parallel
  4. Each job will trigger ml_benchmark_subset.yml which will run
     only one of the generated YAML file via make generate-mlbench,
     based on the value of the matrix item they were given.
  5. As soon as a job is completed, the next one in the matrix
     will start promptly.

This is done until all the matrix items are exhausted.
2022-11-21 11:25:40 +01:00
Luis Montero
aad2a42634 chore(ffi): support ARM64 chips 2022-11-17 01:04:36 +01:00
Samuel Tap
8a231974f7 chore(ffi): update to concrete-ffi without fftw 2022-11-16 11:44:46 +01:00
youben11
472e762fbf feat(rust): enhance API to create func with FHE dialect 2022-11-09 12:53:25 +01:00
Quentin Bourgerie
84041557ab enhance(bench): Add benchmark generator for leveled operators 2022-11-08 14:27:30 +01:00
David Testé
d1db4a5e45 tests(benchmarks): reduce number of test cases to speed-up execution
Bench just one compilation option for automatic benchmarks. Only 'loop'
option is tested to take advantage of hardware with a lot of available
CPUs. Running benchmarks with 'default' option is suboptimal for this
kind of hardware since it uses only one CPU.

This also remove time consuming MNIST test, as it should be in ML benchmarks.

Moreover Makefile is fixed to use provided Python executable instead of
relying on system one to generate MLIR Yaml files.
2022-11-03 19:07:39 +01:00
youben11
0ac21fd037 feat: add rust bindings
The rust bindings are intented to access both LLVM/MLIR CAPI as well as
the concrete-compiler one. This initial commit provide the API for
LLVM/MLIR only. Tests should be used as an example to how to generate a
valid DAG of operations in MLIR.
2022-11-02 14:46:38 +01:00
Antoniu Pop
746b7c2641 fix(dataflow build): fix HPX installation path when built and installed locally. 2022-10-25 09:50:51 +01:00
Quentin Bourgerie
a5047586f4 test(benchmark): Add more micro benchmarks on parallel lookup table 2022-10-20 10:36:32 +01:00
youben11
7cd45d1514 test: add GPU end2end tests 2022-10-20 10:36:32 +01:00
youben11
7e08614e6c build: add options to build with CUDA support 2022-10-20 10:36:32 +01:00
David Testé
d2b20ca551 ci(bench): fix end-to-end benchmarks workflow
This includes several fixes and add some functionalities:
 * EC2 instance type can be selected when workflow is triggered manually
 * benchmarks will be run on each push on main branch
 * docker is not used any more due to building issues
 * 10 repetitions are made during the benchmarks then results are aggregated
 * more tags are used to identify benchmarks configuration
2022-10-18 12:02:35 +02:00
Andi Drebes
eae898b56f enhance(build): Add convenience targets LLVM / MLIR tools to toplevel Makefile
This adds four new targets `opt`, `mlir-cpu-runner`, `mlir-opt`, and
`mlir-translate` to the toplevel Makefile of the compiler to
conveniently build the corresponding LLVM / MLIR utilities (e.g., for
debugging purposes).
2022-10-11 17:09:47 +02:00
Quentin Bourgerie
fe7cdf9d11 chore: Go back to rc.5 to fix the mac os CI 2022-10-10 12:30:27 +02:00
Quentin Bourgerie
f2cbb1e719 chore: ./build as default build directory 2022-10-10 09:33:45 +02:00
David Testé
89d7f065ae ci(compiler): schedule end-to-end benchmarks
Run end-to-end benchmarks on a weekly basis. Results are parsed and
then sent to Slab CI bot to be later plotted into Grafana.
2022-10-07 14:45:56 +02:00
Ayoub Benaissa
6a5e65631c chore: update concrete-ffi version
no major breaking changes, just enhancements
check https://github.com/zama-ai/concrete-core/releases
2022-10-06 12:48:17 +01:00
Andi Drebes
e35c5bddcf fix(test): Add missing dependency from run-mlbench to generate-mlbench
The target `run-mlbench` indirectly depends on the contents of
`tests.ml/bench.zip` which are extracted by `generate-mlbench`. If
`generate-mlbench` has not been built pefore, `run-mlbench`
fails. This adds a the missing dependency from `run-mlbench` to
`generate-mlbench`.
2022-10-04 16:40:27 +02:00
Andi Drebes
85442f425d test(compiler): Move clear text tests to YAML test specification
This moves all tests from
`tests/end_to_end_tests/end_to_end_jit_clear_tensor.cc` to the test
specification in YAML format in
`tests/end_to_end_fixture/end_to_end_clear_tensor.yaml`. Parametric
tests and tests invoking lambdas in loops have been fully unrolled.
2022-10-04 14:40:40 +02:00
Andi Drebes
847e8607f3 enhance(compiler): Add support for debug builds in Makefile
This adds a new variable `BUILD_TYPE` to the Makefile, controlling
whether the build should be a debug or a release build (values `Debug`
and `Release`, respectively). The default mode is `Release`. Depending
on the build type, the build directory is set to `build-Debug` or
`build-Release`. This enables debug and release builds to co-exist and
to switch easily between the two.
2022-10-04 14:40:40 +02:00
youben11
628df8849c refactor: unify Makefile target naming style 2022-09-19 13:24:20 +01:00
youben11
8f05f9ea72 ci: change package release process
it should be now faster using the same compilation cache as the test
build.
2022-09-19 13:24:20 +01:00
Antoniu Pop
09184077d9 benchmarking: add YAML generator and ML benchmarks (see issue https://github.com/zama-ai/concrete-ml-internal/issues/1543). 2022-09-15 14:38:56 +01:00
youben11
0cd9a7984d feat: support dataflow execution in python packages
use latest image manylinux_2_28_x86_64 using libc 2.28
2022-09-13 15:18:15 +01:00
youben11
4b584ca64e ci: create new keysetcache
last commit updated which keys we serialize
2022-09-06 07:18:34 +01:00
youben11
661d33c2b6 feat: keep std bsk and conv to fourier when needed 2022-09-06 07:18:34 +01:00
Mayeul@Zama
eb4b0753dc enhance(runtime): Move to the official concrete-core-ffi release 0.2.0-rc2
Co-authored-by: Mayeul@Zama <mayeul.debellabre@zama.ai>
Co-authored-by: Quentin Bourgerie <bourgerie.quentin@gmail.com>
2022-08-30 18:08:47 +02:00
Quentin Bourgerie
46a914e181 chore(optimizer): Always call the makefile of the optimizer 2022-08-25 09:46:51 +02:00