Commit Graph

600 Commits

Author SHA1 Message Date
Antoniu Pop
c82cbf0b33 feat(compiler): add a pass to transfer ownership of data buffers to the SDFG runtime on Put operations. This allows to avoid an extra copy of the data for use in asynchronous operations. 2024-07-14 07:27:58 +01:00
Antoniu Pop
fe27cbb212 feat(compiler): Add counter and accumulator timers for time logging. Add environment variable CONCRETELANG_TIMING_ENABLED as prerequisite to activation of timing logs. 2024-07-14 07:27:58 +01:00
Antoniu Pop
71c29d0d02 fix(compiler): [GPU backend] restrict the direct lowering of operators to the GPU backend when already lowering through the SDFG dialect. Otherwise this forces GPU offload even for very fine granularity operations, bypassing SDFG and Batching. 2024-07-14 07:27:58 +01:00
Antoniu Pop
7bac6cb8b9 fix(compiler): [GPU runtime] remove now unnecessary assert restrictions on multiple keys in KS/BS operations for GPU. 2024-07-14 07:27:58 +01:00
Antoniu Pop
f39df55f60 fix(compiler): [GPU runtime] when GPU offload is not available set number of devices to 0 to avoid conflicts. 2024-07-13 19:29:48 +01:00
youben11
8cf8d99aea fix(compiler): move the use of conflicting symbols to the runtime lib
https://github.com/zama-ai/concrete-internal/issues/689
2024-07-05 09:12:29 +02:00
James Marsh
d9c21df157 chore(compiler): Bump version of tfhe-rs and rust compiler and fixed build issues 2024-07-04 17:22:34 +02:00
Alexandre Péré
e5a54ef71d fix(compiler): fix racing the test directories on mac 2024-06-25 16:25:30 +02:00
Antoniu Pop
2f17089eda feat(frontend): add python bindings for functions to check whether the compiler is GPU enabled and whether a GPU is available on the system. 2024-06-24 11:50:56 +01:00
Antoniu Pop
cc9229b5c7 fix(compiler): [GPU backend] add sanity checks when the emitGPUOps option is selected to ensure that the compiler/runtime have GPU capability and that at least one device is available to run on. 2024-06-24 11:50:56 +01:00
Antoniu Pop
f93611e5e1 fix(compiler): [DFR] initialize work function registry and runtime context manager pointers outside of constructors. 2024-06-21 13:36:47 +01:00
Antoniu Pop
1879f9c1e0 fix(compiler): [distributed] remove remote node main cancellation and work function registration. As JIT execution is now deprecated, load work functions from compiled library instead. 2024-06-21 13:36:47 +01:00
Antoniu Pop
50c5d7ff3c test(compiler): remove deprecated dataflow/distributed tests and CI build and test targets. 2024-06-21 13:36:47 +01:00
Antoniu Pop
96ef5cd099 fix(compiler): propagate work function attribute marker during TFHE circuit parametrization rewrite. 2024-06-21 13:36:47 +01:00
Andi Drebes
fe20cda302 chore(compiler): Bump MLIR fork to version including extended canonicalization of tensor.insert_slice
Also, invoke the canonicalizer to include the new canonicalization
pattern after batching in order to eliminate unnecessary copies due to
redundant insertions completely overwriting empty tensors.
2024-06-20 06:34:48 +02:00
Andi Drebes
8a24d1c7b1 fix(compiler): Handle tracing.trace_ciphertext operations in SimulateTFHEPass
The test
`end_to_end_trace.simulate1.trace_ciphertext_without_attributes_16bits.0`
generates `Tracing.trace_ciphertext` operations, which are not handled
by `SimulateTFHEPass`, resulting in a conversion error.

This change causes `Tracing.trace_ciphertext` operations to be
converted into appropriate `Tracing.trace_plaintext` operations.
2024-06-20 06:34:48 +02:00
rudy
f299adf461 fix(optimizer): sync with linting update, dead code and deprecated constants 2024-06-18 17:59:36 +02:00
Bourgerie Quentin
b7793aebe8 test(compiler/bindings): Remove test of 80 bits security level while is not more supported 2024-06-18 16:45:09 +02:00
rudy
1da7347f51 fix(optimizer): sync with linting update, dead code and deprecated constants 2024-06-18 16:01:04 +02:00
Alexandre Péré
13253d18f4 fix(frontend): add missing composition serialization 2024-06-18 10:42:00 +02:00
youben11
bce7fe30ba fix(ci): build compiler image with cuda11.8
previous version was not working properly (too recent). So we switched
to 11.8 and thus downgraded to gcc11. The reason for using ?= in the
Makefile is to default to gcc11 in the docker image, as those variables
are set in the env.
2024-06-13 13:41:11 +01:00
Umut
81b04a3562 fix(compiler): use maxpool strides properly when lowering to linalg 2024-06-11 18:32:02 +03:00
Alexandre Péré
5185940807 feat(frontend): add support for wires to concrete-python 2024-06-11 13:23:03 +02:00
Bourgerie Quentin
db2c7550f5 fix(compiler): Allow uncompressed ciphertext in compressed gate to fix composition + compression 2024-06-04 16:12:18 +02:00
Antoniu Pop
0a1934c2ee fix(compiler): fix input parameter deserialization where total data input overflows 32bit accumulator. 2024-05-30 08:11:29 +01:00
Umut
c677f83af7 feat(frontend-python): dynamic indexing 2024-05-29 18:21:46 +03:00
Alexandre Péré
15816354aa feat(optimizer): add support for circuits and composition rules 2024-05-28 09:10:04 +02:00
youben11
974830e40d fix(compiler): control overflow detection for LUT via flag 2024-05-28 07:28:45 +01:00
aquint-zama
08a1871252 chore(compiler): update black dev dependency 2024-05-24 09:48:15 +02:00
youben11
6637b659db feat(compiler/frontend): add flag to enable/disable overflow detection
in simulation
2024-05-24 07:52:03 +01:00
Andi Drebes
2caafc7f3b feat(compiler): Allow dag-multi optimization strategy for data-flow parallelization
With the type inference pass now being able to handle RT operations
and multiple functions, the restriction disallowing data-flow
parallelization when choosing the dag-multi optimization strategy can
be lifted. Remove the check and bail out in `CompilerEngine.cpp`.
2024-05-16 12:07:51 +02:00
Andi Drebes
1703f08e78 feat(compiler): Type Inference: Add support for RT operations
This adds support for the operations `RT.await_future`,
`RT.build_return_ptr_placeholder`, `RT.create_async_task`,
`RT.deref_return_ptr_placeholder`,
`RT.deref_work_function_argument_ptr_placeholder`,
`RT.make_ready_future`, `RT.register_task_work_function`, and
`RT.work_function_return`, the RT types `RT.ptr` and `RT.future`, as
well as the auxiliary operation `arith.select`.
2024-05-16 12:07:51 +02:00
Antoniu Pop
2da8644e57 fix(compiler): fix lowering of tasks generated from loop tiling pass. 2024-05-16 12:07:51 +02:00
Andi Drebes
a5afb1f0a6 feat(compiler): Type Inference: Add support for multiple functions
The type inference pass currently only supports a single function as
it is unable to infer types across function boundaries. This commit
adds support for multiple functions and tracks types across
`func.func`, `func.call` and function `constant` operations.
2024-05-16 12:07:51 +02:00
Andi Drebes
d270886091 refactor(compiler): Type inference: Enable debugging output for entire modules 2024-05-16 12:07:51 +02:00
Andi Drebes
26bed92ee7 fix(compiler): Type Inference: Defer erasure of functions to the end of the rewriting
A function may be referenced at multiple sites, including at locations
that have not been visited by the rewriter when the function itself is
rewritten. By using `IRRewriter::replaceOp` after rewriting a
function, the function is erased immediately, which may cause the
rewriting to fail due to remaining uses of the function.

Defering the erasure of functions to the end of the entire rewriting
process ensures that all uses of the original functions have been
updated to the rewritten functions.
2024-05-16 12:07:51 +02:00
Andi Drebes
c1d742adf0 fix(compiler): Type Inference: Support ops with regions containing multiple blocks
Operations with regions containing multiple blocks are currently not
handled correctly, since the list of successors of an operation is not
updated with rewritten blocks when the operation is
rewritten. Furthermore, functions were assumed to have only a single
block.

This change supports operations with multiple blocks by fixing the
issues above.
2024-05-16 12:07:51 +02:00
Andi Drebes
c371269896 fix(compiler): Type Inference: Allow unresolved types in TypeInferenceUtils::getInferredType 2024-05-16 12:07:51 +02:00
Andi Drebes
ebb08c788f fix(compiler): Type Inference: Register dependencies between op and its related values
Updates of the types inferred for the values related to an operation
only propagate correctly to the operation if there is a direct
producer-consumer relationship or an indirect producer-consumer
relationship via some additional mechanism (e.g., region
successors). However, this is not sufficient to ensure updates of
values and operations that are related otherwise.

This change explicitly models the dependencies between related values
and an operation via `DataFlowAnalysis::addDependency` in order to
guarantee that all updates of the types of values are propagated to
all operations that the type resolver has designated as related
operations.

Furthermore, all operations are visited once initially in order to
guarantee that updates propagate to operations, for which the dataflow
framework does not invoke `visitOperation`.
2024-05-16 12:07:51 +02:00
Andi Drebes
ea41400b90 feat(compiler): Type inference: Delegate enumeration of related values to TypeResolver
This delegates the enumeration of values related to an operation to
the class `TypeResolver`. This allows for the customization of this
process via a class inheriting `TypeResolver`.
2024-05-16 12:07:51 +02:00
Andi Drebes
c3366d9eef feat(compiler): Type Inference: Add support for tensor.dim 2024-05-16 12:07:51 +02:00
Andi Drebes
b297de93c4 refactor(compiler): Type inference: Generalize constraint for equal element types to nested types 2024-05-16 12:07:51 +02:00
youben11
b6a43cfc5c feat(compiler/simu): support signed integers 2024-05-15 10:53:42 +01:00
youben11
c655856b1c test(compiler): overflow in simulation 2024-05-15 10:53:42 +01:00
youben11
cae77e89bf feat(compiler/simu): add loc in overflow warnings 2024-05-15 10:53:42 +01:00
youben11
b953d57fe4 feat(compiler): warn when there is overflow in sim (native encoding) 2024-05-15 10:53:42 +01:00
youben11
001aba6809 refactor(compiler, simu): rewrite add/mul to CAPI calls 2024-05-15 10:53:42 +01:00
Umut
d954b279e8 fix(compiler): use exact passes on check tests to avoid reordering when new passes are introduced 2024-05-15 11:21:46 +03:00
Umut
8e4d237364 feat(compiler): fancy assignment 2024-05-15 11:21:46 +03:00
Umut
6e36d0f801 feat(compiler): tensor.generate to scf.forall 2024-05-15 11:21:46 +03:00