Commit Graph

240 Commits

Author SHA1 Message Date
Bourgerie Quentin
1dec886770 feat(compiler/frontend-python): Expose default GPU CompilerOptions set and use it in concrete-python 2024-01-05 13:15:12 +00:00
aPere3
6691c8f107 feat(frontend): add support for gpu in concrete-python 2024-01-05 13:15:12 +00:00
Bourgerie Quentin
15835719b9 fix(compiler/optimizer): Returns error for optimization strategy that doesn't support composition 2024-01-04 15:00:25 +01:00
Bourgerie Quentin
398af2783d test(compiler): Add disabled test on composition (to fix later) 2024-01-04 15:00:25 +01:00
Bourgerie Quentin
f3ec1976ef fix(compiler/optimizer): Returns NotComposable error if not lookup table in dag 2024-01-04 15:00:25 +01:00
Alexandre Péré
60da713312 feat(optimizer): adds support for function composition 2024-01-04 15:00:25 +01:00
Umut
1edd341fb0 feat(frontend-python): printing assignment 2023-12-29 10:19:06 +03:00
Bourgerie Quentin
e69a9ca520 fix(compiler/tests): bad options in end-to-end-test in macos 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
fc10062b4b test(compiler): Add --input-compression to the end-to-end-test 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
4d1d7bde7d test(compiler): Add --simulation options to end-to-end-test tool 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
c67fac4082 feat(compiler): Introduce input compression and use tfhe-rs as backend
- added --compress-input compiler option which forces the use of seeded
  bootstrap keys and keyswitch keys
- replaced the concrete-cpu FHE implementation with tfhe-rs

Co-authored-by: Nikita Frolov <nf@mkmks.org>
2023-12-22 15:51:02 +01:00
Bourgerie Quentin
0379f3676c fix(compiler/tests): Recreate the entire testCicuit when retrying a test as the keyset is also used by the clientProgram 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
4a8356c18d feat(compiler/tests): Add optimizer-key-sharing option to the end-to-end tests tool 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
8ecea9d625 fix(compiler/tests): Print values when a test fail 2023-12-22 15:51:02 +01:00
Alexandre Péré
e82a8f6ddb chore(compiler): fix broken retry for flaky tests 2023-12-18 15:54:25 +01:00
Bourgerie Quentin
39e27d3766 chore(compiler/bench): Remove useless bench 2023-12-15 11:00:45 +01:00
Bourgerie Quentin
23c721996a refactor(compiler/tests): Refactor internal test utils 2023-12-15 11:00:45 +01:00
rudy
687502d4e2 feat(compiler): tests showing ReLU scenario 2023-12-04 10:20:06 +01:00
rudy
ae2eec1b18 chore(compiler): use same bit extract in round as lsb 2023-12-04 10:20:06 +01:00
rudy
34ffd6f97a feat(compiler): new ops, reinterpret_precision and lsb 2023-12-04 10:20:06 +01:00
Umut
c040b665cf fix(compiler): print parameters correctly in the bindings 2023-11-29 13:20:48 +03:00
youben11
a4db568170 fix(compiler): add missing args in woppbs simulation 2023-11-22 13:49:30 +01:00
Bourgerie Quentin
588075ddc8 test(compiler): Add retry for fixing some flaky tests 2023-11-16 12:48:58 +01:00
Andi Drebes
ed8d1f3857 fix(compiler): Force linking of libomp with libstdc++ in debug builds
Debug builds preserve assertions in libomp, which are implemented
using code form libstdc++. However, the build rules for libomp
explicitly prevent the library from being linked with libstdc++ by
default, resulting in a linker error for debug builds.

This patch sets the option `LIBOMP_USE_STDCPPLIB` for libomp to `ON`
for debug builds in order to force linking with libstdc++.
2023-11-16 11:09:52 +01:00
Bourgerie Quentin
09af803754 feat(compiler): Add support for multi output function up to python bindings 2023-11-13 09:22:54 +01:00
Alexandre Péré
e8ef48ffd8 feat(compiler): introduce concrete-protocol
This commit:
 + Adds support for a protocol which enables inter-op between concrete,
   tfhe-rs and potentially other contributors to the fhe ecosystem.
 + Gets rid of hand-made serialization in the compiler, and
   client/server libs.
 + Refactors client/server libs to allow more pre/post processing of
   circuit inputs/outputs.

The protocol is supported by a definition in the shape of a capnp file,
which defines different types of objects among which:
 + ProgramInfo object, which is a precise description of a set of fhe
   circuit coming from the same compilation (understand function type
   information), and the associated key set.
 + *Key objects, which represent secret/public keys used to
   encrypt/execute fhe circuits.
 + Value object, which represent values that can be transferred between
   client and server to support calls to fhe circuits.

The hand-rolled serialization that was previously used is completely
dropped in favor of capnp in the whole codebase.

The client/server libs, are refactored to introduce a modular design for
pre-post processing. Reading the ProgramInfo file associated with a
compilation, the client and server libs assemble a pipeline of
transformers (functions) for pre and post processing of values coming in
and out of a circuit. This design properly decouples various aspects of
the processing, and allows these capabilities to be safely extended.

In practice this commit includes the following:
 + Defines the specification in a concreteprotocol package
 + Integrate the compilation of this package as a compiler dependency
   via cmake
 + Modify the compiler to use the Encodings objects defined in the
   protocol
 + Modify the compiler to emit ProgramInfo files as compilation
   artifact, and gets rid of the bloated ClientParameters.
 + Introduces a new Common library containing the functionalities shared
   between the compiler and the client/server libs.
 + Introduces a functional pre-post processing pipeline to this common
   library
 + Modify the client/server libs to support loading ProgramInfo objects,
   and calling circuits using Value messages.
 + Drops support of JIT.
 + Drops support of C-api.
 + Drops support of Rust bindings.

Co-authored-by: Nikita Frolov <nf@mkmks.org>
2023-11-09 17:09:04 +01:00
youben11
6a41c31370 fix(compiler): revert to old MANP values
this commit need to be reverted when we start computing the MANP on
linalg.genric ops
2023-10-03 16:17:27 +01:00
youben11
48b2d884c6 fix(compiler): use args to have an init MANP of 1
As the MANP value of zero tensors becomes 0 (instead of 1 previously) we
switch to using argument
2023-10-03 16:17:27 +01:00
youben11
64d0741c1b feat(compiler): support linalg.genric instead of FHELinalg ops in DF parallelization 2023-10-03 16:17:27 +01:00
youben11
88dd13756a feat(compiler): support linalg.generic in the MANP Analysis 2023-10-03 16:17:27 +01:00
youben11
e4835bd002 feat(compiler): support woppbs in simulation 2023-09-28 09:29:48 +01:00
youben11
cd0f02d99c feat(compiler): support crt encoding in scalars during simulation 2023-09-28 09:29:48 +01:00
youben11
4f2b44c9d8 feat(compiler): support compilation of CRT in simulation 2023-09-28 09:29:48 +01:00
rudy
d64c9b0f25 fix(optimizer): multi-parameters, shorter_dump for the compiler --display-optimizer-choice 2023-09-25 15:55:22 +02:00
rudy
1c0a70f911 fix(compiler): conversion to optimizer dag, bad dot before signed lut
this has no effect apart making the shape incorrect
2023-09-25 15:53:19 +02:00
Umut
49dc0578f4 fix(compiler-bindings): don't crash on unknown location formats 2023-09-20 15:01:58 +02:00
rudy
90e62f551b feat(optimizer): multi-params, key sharing 2023-09-19 12:13:25 +02:00
Umut
5d3e4bb2d6 chore(compiler): decrease p_error for a flanky test 2023-09-18 12:42:50 +02:00
Alexandre Péré
d28bf3767b feat(compiler): adds support for dynamic luts in fhelinalg 2023-09-18 12:42:50 +02:00
Nikita Frolov
73a992f0a6 refactor(compiler): generalize noise calculation in FHE/FHELinalg
Instead of having one `getSQManp` implementation per op with a lot of repetition, the noise
calculation is now modular.

- Ops that implements`UnaryEint`/`BinaryInt`/`BinaryEint` interfaces share the operand noise
presence check.
- For many scalar ops no further calculation is needed. If it's not the case, an op can override
`sqMANP`.
- Integer operand types lookups are abstracted into `BinaryInt::operandIntType()`
- Finding largest operand value for a type is abstracted into `BinaryInt::operandMaxConstant`
- Noise calculation for matmul ops is simplified and it's now general enough to work for
`matmul_eint_int`, `matmul_int_eint` and `dot_eint_int` at once.
2023-09-12 14:51:53 +01:00
Bourgerie Quentin
d71201ff8c fix(compiler): Use FHE.zero_tensor instead of bufferization.alloc_tensor as alloc_tensor explictly has a alloc semantic, so it cannot be eliminated by dce 2023-09-11 15:08:49 +02:00
rudy
f4099936e2 fix(compiler): ci, format_cpp.sh, bug + better message 2023-09-11 14:19:08 +02:00
aPere3
1e726a50ed feat(compiler): add support for dynamic luts in compiler. 2023-09-11 10:29:59 +02:00
Alexandre Péré
083ab1103f chore(compiler): moves EncryptMulToDoubleTLU includes 2023-09-11 10:29:59 +02:00
aPere3
b58821b1d1 chore(compiler): refactor e2e tlu tests to use dynamic lut. 2023-09-11 10:29:59 +02:00
rudy
229bbc2327 fix(compiler): do not optimize llvm twice 2023-09-08 15:26:24 +02:00
rudy
74c4a7eae5 fix(compiler): check minimal lwe size 2023-09-08 14:26:32 +02:00
rudy
be77fb26b8 feat(compiler): bench, cifar-16 2023-09-08 10:03:42 +02:00
rudy
4907577b1c feat(compiler): bench, llm, big mat mul 2023-09-08 10:03:42 +02:00
rudy
a28a7bf397 feat(compiler): bench, activate levelled ops 2023-09-08 10:03:42 +02:00