Commit Graph

101 Commits

Author SHA1 Message Date
Andi Drebes
8d71dc2028 feat(compiler): Add verifier to TFHE.encode_expand_lut_for_bootstrap 2024-01-29 15:35:14 +01:00
Andi Drebes
ea07239732 feat(compiler): Add verifiers to TFHE bootstrap operations 2024-01-29 15:35:10 +01:00
Alexandre Péré
f113e8bcfe fix(ci): fix the doc verification 2024-01-26 17:15:03 +01:00
rudy
c298408f26 feat(optimizer): multi-parameters, partitionning using norm2 2024-01-08 20:32:27 +01:00
Bourgerie Quentin
f9c7a79183 refactor(compiler/frontend): Rename option to compress_evaluation_keys 2024-01-08 09:49:06 +01:00
Antoniu Pop
196a3ae4af fix(compiler): distributed execution: fix key transfer to remote nodes. 2024-01-05 19:03:56 +00:00
Antoniu Pop
822316b3d2 feat(compiler): add mixed scheduling of SDFG graphs on CPU and GPU. 2024-01-05 13:15:12 +00:00
Antoniu Pop
acf5780ee3 fix(compiler): handle splice streams (streams originating on device, used both on device and on host) in SDFG for GPU scheduling. 2024-01-05 13:15:12 +00:00
Bourgerie Quentin
f3ec1976ef fix(compiler/optimizer): Returns NotComposable error if not lookup table in dag 2024-01-04 15:00:25 +01:00
Alexandre Péré
60da713312 feat(optimizer): adds support for function composition 2024-01-04 15:00:25 +01:00
Bourgerie Quentin
4d1d7bde7d test(compiler): Add --simulation options to end-to-end-test tool 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
c67fac4082 feat(compiler): Introduce input compression and use tfhe-rs as backend
- added --compress-input compiler option which forces the use of seeded
  bootstrap keys and keyswitch keys
- replaced the concrete-cpu FHE implementation with tfhe-rs

Co-authored-by: Nikita Frolov <nf@mkmks.org>
2023-12-22 15:51:02 +01:00
Bourgerie Quentin
0379f3676c fix(compiler/tests): Recreate the entire testCicuit when retrying a test as the keyset is also used by the clientProgram 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
8ecea9d625 fix(compiler/tests): Print values when a test fail 2023-12-22 15:51:02 +01:00
Bourgerie Quentin
23c721996a refactor(compiler/tests): Refactor internal test utils 2023-12-15 11:00:45 +01:00
rudy
34ffd6f97a feat(compiler): new ops, reinterpret_precision and lsb 2023-12-04 10:20:06 +01:00
youben11
a4db568170 fix(compiler): add missing args in woppbs simulation 2023-11-22 13:49:30 +01:00
Bourgerie Quentin
588075ddc8 test(compiler): Add retry for fixing some flaky tests 2023-11-16 12:48:58 +01:00
Bourgerie Quentin
09af803754 feat(compiler): Add support for multi output function up to python bindings 2023-11-13 09:22:54 +01:00
Alexandre Péré
e8ef48ffd8 feat(compiler): introduce concrete-protocol
This commit:
 + Adds support for a protocol which enables inter-op between concrete,
   tfhe-rs and potentially other contributors to the fhe ecosystem.
 + Gets rid of hand-made serialization in the compiler, and
   client/server libs.
 + Refactors client/server libs to allow more pre/post processing of
   circuit inputs/outputs.

The protocol is supported by a definition in the shape of a capnp file,
which defines different types of objects among which:
 + ProgramInfo object, which is a precise description of a set of fhe
   circuit coming from the same compilation (understand function type
   information), and the associated key set.
 + *Key objects, which represent secret/public keys used to
   encrypt/execute fhe circuits.
 + Value object, which represent values that can be transferred between
   client and server to support calls to fhe circuits.

The hand-rolled serialization that was previously used is completely
dropped in favor of capnp in the whole codebase.

The client/server libs, are refactored to introduce a modular design for
pre-post processing. Reading the ProgramInfo file associated with a
compilation, the client and server libs assemble a pipeline of
transformers (functions) for pre and post processing of values coming in
and out of a circuit. This design properly decouples various aspects of
the processing, and allows these capabilities to be safely extended.

In practice this commit includes the following:
 + Defines the specification in a concreteprotocol package
 + Integrate the compilation of this package as a compiler dependency
   via cmake
 + Modify the compiler to use the Encodings objects defined in the
   protocol
 + Modify the compiler to emit ProgramInfo files as compilation
   artifact, and gets rid of the bloated ClientParameters.
 + Introduces a new Common library containing the functionalities shared
   between the compiler and the client/server libs.
 + Introduces a functional pre-post processing pipeline to this common
   library
 + Modify the client/server libs to support loading ProgramInfo objects,
   and calling circuits using Value messages.
 + Drops support of JIT.
 + Drops support of C-api.
 + Drops support of Rust bindings.

Co-authored-by: Nikita Frolov <nf@mkmks.org>
2023-11-09 17:09:04 +01:00
youben11
88dd13756a feat(compiler): support linalg.generic in the MANP Analysis 2023-10-03 16:17:27 +01:00
youben11
e4835bd002 feat(compiler): support woppbs in simulation 2023-09-28 09:29:48 +01:00
youben11
cd0f02d99c feat(compiler): support crt encoding in scalars during simulation 2023-09-28 09:29:48 +01:00
youben11
4f2b44c9d8 feat(compiler): support compilation of CRT in simulation 2023-09-28 09:29:48 +01:00
rudy
90e62f551b feat(optimizer): multi-params, key sharing 2023-09-19 12:13:25 +02:00
Nikita Frolov
73a992f0a6 refactor(compiler): generalize noise calculation in FHE/FHELinalg
Instead of having one `getSQManp` implementation per op with a lot of repetition, the noise
calculation is now modular.

- Ops that implements`UnaryEint`/`BinaryInt`/`BinaryEint` interfaces share the operand noise
presence check.
- For many scalar ops no further calculation is needed. If it's not the case, an op can override
`sqMANP`.
- Integer operand types lookups are abstracted into `BinaryInt::operandIntType()`
- Finding largest operand value for a type is abstracted into `BinaryInt::operandMaxConstant`
- Noise calculation for matmul ops is simplified and it's now general enough to work for
`matmul_eint_int`, `matmul_int_eint` and `dot_eint_int` at once.
2023-09-12 14:51:53 +01:00
aPere3
1e726a50ed feat(compiler): add support for dynamic luts in compiler. 2023-09-11 10:29:59 +02:00
Alexandre Péré
083ab1103f chore(compiler): moves EncryptMulToDoubleTLU includes 2023-09-11 10:29:59 +02:00
youben11
530bacb2e3 refactor(compiler): clean statistic passes 2023-09-04 09:22:28 +01:00
youben11
54089186ae refactor(compiler): reorganize passes and add memory usage pass 2023-08-29 15:47:25 +01:00
youben11
d88b2c87ac feat(compiler): compute memory usage per location 2023-08-29 15:47:25 +01:00
Umut
ade83d5335 feat(compiler): add more detailed statistics 2023-08-01 18:40:08 +02:00
youben11
f6599a91c6 refactor(compiler): add func to populate RTOps type conversion 2023-07-31 16:57:53 +01:00
Umut
79b38a72ec feat(compiler): provide circuit statistics 2023-07-26 11:08:15 +02:00
youben11
022b1879a1 feat(compiler): support compiling in-memory module 2023-07-21 14:14:55 +01:00
aquint-zama
502fcfdbe6 chore(compiler): fix typos 2023-07-13 14:33:54 +02:00
aquint-zama
9928c0dbd9 docs(compiler): Generate dialect docs for the Tracing Dialect 2023-07-13 14:33:54 +02:00
aquint-zama
60817835db chore(compiler): typos 2023-07-13 14:33:54 +02:00
Andi Drebes
cc6f86957c docs(compiler): Generate dialect docs for the RT Dialect 2023-07-13 14:33:54 +02:00
youben11
27e1835f23 feat(compiler/python): expose simulation to python-bindings 2023-06-27 14:21:42 +01:00
youben11
eb116058e0 feat(compiler): support invoke on simulated circuits 2023-06-27 14:21:42 +01:00
youben11
32ad46f7c5 feat(compiler): disable runtimeCtx pass in simulation 2023-06-27 14:21:42 +01:00
youben11
5e848a6971 feat(compiler/clientlib): support simulation in enc-args 2023-06-27 14:21:42 +01:00
youben11
09b84e0373 refactor(compiler/clientlib): add ValueDecrypter Interface 2023-06-27 14:21:42 +01:00
youben11
ad13602bf3 refactor(compiler/clientlib): add ValueExporter Interface 2023-06-27 14:21:42 +01:00
youben11
7b594c5ecd feat(compiler): add simulation runtime 2023-06-27 14:21:42 +01:00
youben11
b8e462c1cc feat(compiler): add option to compile in simulation mode 2023-06-27 14:21:42 +01:00
youben11
e58b46d86d feat(compiler): add a pass to simulate TFHE ops
lowering is done to CAPI calls that implement simulation as well as
standard MLIR ops
2023-06-27 14:21:42 +01:00
Bourgerie Quentin
5147ac8418 feat(compiler): Add canonicalization of FHE/FHELinalg to_signed to_unsigned ops 2023-06-23 14:34:23 +02:00
Bourgerie Quentin
5a80e22cf3 fix(compiler): Annotate FHELinalg as Pure for canonicalization 2023-06-23 14:34:23 +02:00