Commit Graph

432 Commits

Author SHA1 Message Date
Ayoub Benaissa
d89af27a68 docs(compiler): fix link 2024-08-08 12:31:31 +01:00
Ayoub Benaissa
9c226f1769 docs(compiler): add note for building CP wheel in the compiler 2024-08-08 12:31:31 +01:00
Alexandre Péré
941d78b1d1 feat(optimizer): improve errors for non-composable functions 2024-08-05 16:11:39 +02:00
rudy
89553858dd feat(frontend-python): release GIL on run, encrypt, decrypt 2024-08-02 11:35:08 +02:00
rudy
3adf658a97 fix(compiler): levelledOP conversion to dot like for optimizer
note: this cannot be done as before in the optimizer since the latter had more information.
here we assumed each input contribute equally to the resulting noise.

A better fix could be to provide the linear relation of inputs instead of manp and smanp.
2024-08-01 15:32:52 +02:00
Andi Drebes
9b034d9441 test(compiler): Ensure that ForLoopToParallelPass does not trigger for scf.for with parallel = false
This ensures that the regression fixed by
3cd3dff92a is not reintroduced, which
caused the compiler to abort in `ForLoopToParallelPass` for `scf.for`
loops with iteration arguments, even if these were marked with the
attribute `parallel` set to `false`.
2024-07-30 17:52:36 +02:00
Andi Drebes
3cd3dff92a fix(compiler): Convert scf.for to scf.parallel only if parallel attribute is true
The pattern converting `scf.for` operations to `scf.parallel`
operations from `lib/Transforms/ForLoopToParallel.cpp` contains an
assertion that ensures that the source operation does not have any
iteration arguments in order to keep the conversion as simple as
possible.

However, if the attribute `parallel` of the source operation is
`false`, the operation is replaced with an identical clone and the
conversion could be treated as a no-op.

This change modifies the pattern, such that it simply fails if
`parallel` is `false`, making the check for the absence of iteration
arguments unnecessary and avoiding unnecessary bailouts by the
compiler.
2024-07-29 15:52:02 +02:00
Antoniu Pop
4bb7c21739 fix(compiler): [GPU runtime] enable dependence re-split when dependence is used in multiple subgraphs. 2024-07-23 18:52:01 +01:00
Antoniu Pop
2d5d15f0b0 fix(compiler): [GPU runtime] prevent early deallocation of on-device data when multiple processes use the same input. 2024-07-23 18:52:01 +01:00
Antoniu Pop
cf32638264 fix(compiler): [GPU+DFR runtime] add sanity check to prevent both dataflow parallelization and GPU offload to be activated jointly as this is not yet supported. 2024-07-23 18:52:01 +01:00
Alexandre Péré
2aeec5c82a fix(optimizer): fix leveled noise propagation 2024-07-23 09:40:12 +02:00
Alexandre Péré
bd3f36101f fix(compiler): segfault in tests 2024-07-16 11:55:31 +02:00
Antoniu Pop
0bc933ab20 feat(compiler): [GPU runtime] add timing logs. 2024-07-14 07:27:58 +01:00
Antoniu Pop
2ffe60a108 feat(compiler): [GPU runtime] reduce copies in merge operations by merging SDFG batch outputs in place. 2024-07-14 07:27:58 +01:00
Antoniu Pop
c82cbf0b33 feat(compiler): add a pass to transfer ownership of data buffers to the SDFG runtime on Put operations. This allows to avoid an extra copy of the data for use in asynchronous operations. 2024-07-14 07:27:58 +01:00
Antoniu Pop
fe27cbb212 feat(compiler): Add counter and accumulator timers for time logging. Add environment variable CONCRETELANG_TIMING_ENABLED as prerequisite to activation of timing logs. 2024-07-14 07:27:58 +01:00
Antoniu Pop
71c29d0d02 fix(compiler): [GPU backend] restrict the direct lowering of operators to the GPU backend when already lowering through the SDFG dialect. Otherwise this forces GPU offload even for very fine granularity operations, bypassing SDFG and Batching. 2024-07-14 07:27:58 +01:00
Antoniu Pop
7bac6cb8b9 fix(compiler): [GPU runtime] remove now unnecessary assert restrictions on multiple keys in KS/BS operations for GPU. 2024-07-14 07:27:58 +01:00
Antoniu Pop
f39df55f60 fix(compiler): [GPU runtime] when GPU offload is not available set number of devices to 0 to avoid conflicts. 2024-07-13 19:29:48 +01:00
youben11
8cf8d99aea fix(compiler): move the use of conflicting symbols to the runtime lib
https://github.com/zama-ai/concrete-internal/issues/689
2024-07-05 09:12:29 +02:00
James Marsh
d9c21df157 chore(compiler): Bump version of tfhe-rs and rust compiler and fixed build issues 2024-07-04 17:22:34 +02:00
Alexandre Péré
e5a54ef71d fix(compiler): fix racing the test directories on mac 2024-06-25 16:25:30 +02:00
Antoniu Pop
2f17089eda feat(frontend): add python bindings for functions to check whether the compiler is GPU enabled and whether a GPU is available on the system. 2024-06-24 11:50:56 +01:00
Antoniu Pop
cc9229b5c7 fix(compiler): [GPU backend] add sanity checks when the emitGPUOps option is selected to ensure that the compiler/runtime have GPU capability and that at least one device is available to run on. 2024-06-24 11:50:56 +01:00
Antoniu Pop
f93611e5e1 fix(compiler): [DFR] initialize work function registry and runtime context manager pointers outside of constructors. 2024-06-21 13:36:47 +01:00
Antoniu Pop
1879f9c1e0 fix(compiler): [distributed] remove remote node main cancellation and work function registration. As JIT execution is now deprecated, load work functions from compiled library instead. 2024-06-21 13:36:47 +01:00
Antoniu Pop
50c5d7ff3c test(compiler): remove deprecated dataflow/distributed tests and CI build and test targets. 2024-06-21 13:36:47 +01:00
Antoniu Pop
96ef5cd099 fix(compiler): propagate work function attribute marker during TFHE circuit parametrization rewrite. 2024-06-21 13:36:47 +01:00
Andi Drebes
fe20cda302 chore(compiler): Bump MLIR fork to version including extended canonicalization of tensor.insert_slice
Also, invoke the canonicalizer to include the new canonicalization
pattern after batching in order to eliminate unnecessary copies due to
redundant insertions completely overwriting empty tensors.
2024-06-20 06:34:48 +02:00
Andi Drebes
8a24d1c7b1 fix(compiler): Handle tracing.trace_ciphertext operations in SimulateTFHEPass
The test
`end_to_end_trace.simulate1.trace_ciphertext_without_attributes_16bits.0`
generates `Tracing.trace_ciphertext` operations, which are not handled
by `SimulateTFHEPass`, resulting in a conversion error.

This change causes `Tracing.trace_ciphertext` operations to be
converted into appropriate `Tracing.trace_plaintext` operations.
2024-06-20 06:34:48 +02:00
Alexandre Péré
13253d18f4 fix(frontend): add missing composition serialization 2024-06-18 10:42:00 +02:00
youben11
bce7fe30ba fix(ci): build compiler image with cuda11.8
previous version was not working properly (too recent). So we switched
to 11.8 and thus downgraded to gcc11. The reason for using ?= in the
Makefile is to default to gcc11 in the docker image, as those variables
are set in the env.
2024-06-13 13:41:11 +01:00
Umut
81b04a3562 fix(compiler): use maxpool strides properly when lowering to linalg 2024-06-11 18:32:02 +03:00
Alexandre Péré
5185940807 feat(frontend): add support for wires to concrete-python 2024-06-11 13:23:03 +02:00
Bourgerie Quentin
db2c7550f5 fix(compiler): Allow uncompressed ciphertext in compressed gate to fix composition + compression 2024-06-04 16:12:18 +02:00
Antoniu Pop
0a1934c2ee fix(compiler): fix input parameter deserialization where total data input overflows 32bit accumulator. 2024-05-30 08:11:29 +01:00
Umut
c677f83af7 feat(frontend-python): dynamic indexing 2024-05-29 18:21:46 +03:00
Alexandre Péré
15816354aa feat(optimizer): add support for circuits and composition rules 2024-05-28 09:10:04 +02:00
youben11
974830e40d fix(compiler): control overflow detection for LUT via flag 2024-05-28 07:28:45 +01:00
aquint-zama
08a1871252 chore(compiler): update black dev dependency 2024-05-24 09:48:15 +02:00
youben11
6637b659db feat(compiler/frontend): add flag to enable/disable overflow detection
in simulation
2024-05-24 07:52:03 +01:00
Andi Drebes
2caafc7f3b feat(compiler): Allow dag-multi optimization strategy for data-flow parallelization
With the type inference pass now being able to handle RT operations
and multiple functions, the restriction disallowing data-flow
parallelization when choosing the dag-multi optimization strategy can
be lifted. Remove the check and bail out in `CompilerEngine.cpp`.
2024-05-16 12:07:51 +02:00
Andi Drebes
1703f08e78 feat(compiler): Type Inference: Add support for RT operations
This adds support for the operations `RT.await_future`,
`RT.build_return_ptr_placeholder`, `RT.create_async_task`,
`RT.deref_return_ptr_placeholder`,
`RT.deref_work_function_argument_ptr_placeholder`,
`RT.make_ready_future`, `RT.register_task_work_function`, and
`RT.work_function_return`, the RT types `RT.ptr` and `RT.future`, as
well as the auxiliary operation `arith.select`.
2024-05-16 12:07:51 +02:00
Antoniu Pop
2da8644e57 fix(compiler): fix lowering of tasks generated from loop tiling pass. 2024-05-16 12:07:51 +02:00
Andi Drebes
a5afb1f0a6 feat(compiler): Type Inference: Add support for multiple functions
The type inference pass currently only supports a single function as
it is unable to infer types across function boundaries. This commit
adds support for multiple functions and tracks types across
`func.func`, `func.call` and function `constant` operations.
2024-05-16 12:07:51 +02:00
Andi Drebes
d270886091 refactor(compiler): Type inference: Enable debugging output for entire modules 2024-05-16 12:07:51 +02:00
Andi Drebes
26bed92ee7 fix(compiler): Type Inference: Defer erasure of functions to the end of the rewriting
A function may be referenced at multiple sites, including at locations
that have not been visited by the rewriter when the function itself is
rewritten. By using `IRRewriter::replaceOp` after rewriting a
function, the function is erased immediately, which may cause the
rewriting to fail due to remaining uses of the function.

Defering the erasure of functions to the end of the entire rewriting
process ensures that all uses of the original functions have been
updated to the rewritten functions.
2024-05-16 12:07:51 +02:00
Andi Drebes
c1d742adf0 fix(compiler): Type Inference: Support ops with regions containing multiple blocks
Operations with regions containing multiple blocks are currently not
handled correctly, since the list of successors of an operation is not
updated with rewritten blocks when the operation is
rewritten. Furthermore, functions were assumed to have only a single
block.

This change supports operations with multiple blocks by fixing the
issues above.
2024-05-16 12:07:51 +02:00
Andi Drebes
c371269896 fix(compiler): Type Inference: Allow unresolved types in TypeInferenceUtils::getInferredType 2024-05-16 12:07:51 +02:00
Andi Drebes
ebb08c788f fix(compiler): Type Inference: Register dependencies between op and its related values
Updates of the types inferred for the values related to an operation
only propagate correctly to the operation if there is a direct
producer-consumer relationship or an indirect producer-consumer
relationship via some additional mechanism (e.g., region
successors). However, this is not sufficient to ensure updates of
values and operations that are related otherwise.

This change explicitly models the dependencies between related values
and an operation via `DataFlowAnalysis::addDependency` in order to
guarantee that all updates of the types of values are propagated to
all operations that the type resolver has designated as related
operations.

Furthermore, all operations are visited once initially in order to
guarantee that updates propagate to operations, for which the dataflow
framework does not invoke `visitOperation`.
2024-05-16 12:07:51 +02:00