Commit Graph

3498 Commits

Author SHA1 Message Date
Benoit Chevallier-Mames
5e8acf4f70 fix(compiler): this file had been removed from GitBook. 2024-04-25 11:31:50 +02:00
Benoit Chevallier-Mames
7d75adf66e fix(compiler): fixing a few typos in the doc 2024-04-25 11:30:59 +02:00
Andi Drebes
51b5b42c05 fix(compiler): Support indirect references to IVs in indexes when hoisting RT.await_future ops
Until now, the pass hoisting `RT.await_future` operations only
supports `tensor.parallel_insert_slice` operations that use loop
induction variables directly as indexes. Any more complex indexing
expressions produce a domination error, since a
`tensor.parallel_insert_slice` cloned by the pass into an additional
parallel for loop is left with references to values from the original
loop.

This change properly clones operations producing intermediate values
within the original parallel for loop and thus adds support for
indexing expressions that reference loops IVs only indirectly.
nightly-2024.04.26
2024-04-25 11:05:54 +02:00
Luis Montero
5c5f573e5b fix(frontend): remove ResourceWarning from using TemporaryDirectory
We currently have something like `var = TemporaryDirectory`
without explicitly calling the `close` method on `var` thus
raising a `ResourceWarning` when the resource is implicitely cleaned-up.

This could be avoided by using `mkdtemp` instead.
2024-04-25 10:02:47 +03:00
Andi Drebes
16f00416fc fix(compiler): Fix RT type parser: Remove unconditional parsing of prefix "future"
The type parser of the RT dialect unconditionally attempts to parse
the string "future". This breaks the parsing of any RT type. Remove
unconditional parsing of this prefix.
nightly-2024.04.25
2024-04-23 13:26:35 +02:00
Andi Drebes
e238067cd8 fix(compiler): Type inference rewriter: Handle return-like operations correctly
Return-like operations require special treatment by the type inference
rewriter, since their operand types are both tied to the result types
of their producers and to result types of their parent operations.

The inference scheme for ordinary operations, in which the initial
local inference state is composed of the operand types of the
rewritten producers and the old types of related operations before
rewriting is insufficient, since this may result in a mismatch between
the inferred types and the actual types of the already rewritten
parent operation.

Until now, precedence of the new result types of the parent operation
has been implemented by simply designating these types as the operand
types of a return-like operation. However, while this works as
intended for return-like operations, which simply forward values
(e.g., `func.return`), this creates invalid IR for other return-like
operations (e.g., `tensor.yield`).

This change implements precedence of the result types of the parent
operation of a return-like operation by adding the return types of the
already rewritten parent operation to the initial local inference
state before final invocation of type inference.
nightly-2024.04.23 nightly-2024.04.20 nightly-2024.04.24
2024-04-19 09:10:57 +02:00
Benoit Chevallier-Mames
46f92ec6ec chore(frontend): clarify XOR-distance example. nightly-2024.04.19 2024-04-18 19:14:38 +02:00
Bourgerie Quentin
676a787ff2 fix(compiler): Use concrete-cpu function to compute key size in compilation feedbacks 2024-04-18 14:55:43 +02:00
Benoit Chevallier-Mames
705fd02d97 docs(compiler): fix a typo 2024-04-18 14:51:33 +02:00
Benoit Chevallier-Mames
c9767ce990 docs(compiler): Windows pip version will not exist
use Windows WSL instead
2024-04-18 14:50:20 +02:00
Alexandre Péré
ba1d049d95 fix(compiler): increases deserialization limit for values 2024-04-18 14:22:51 +02:00
Bourgerie Quentin
f66dedaf29 docs(compiler): Add warnings about IND-CPA security model 2024-04-18 14:10:00 +02:00
Andi Drebes
a88968d8df fix(compiler): Type inference rewriter: Fix use-after-free in function renaming
The type inference rewriter changes the name of the rewritten function
to the name of the original function when the rewriting process is
complete. However, the name is retrieved from the original function
operation after the operation has already been replaced and thus
destroyed, resulting in a null pointer dereference.

This change retrieves the name of the original function before it is
replaced and saves it in a copy, which is then used to safely assign
the new name to the rewritten function.
2024-04-17 09:55:10 +02:00
Umut
3d0727b845 feat(frontend-python): add option to configure tlu on original bit width nightly-2024.04.17 nightly-2024.04.18 2024-04-16 16:45:05 +03:00
Alexandre Péré
34de883dd9 chore(frontends): add tests to increase coverage of fhe modules 2024-04-16 10:35:06 +02:00
Benoit Chevallier-Mames
a3762ab2cd docs(compiler): fix broken links
closes #https://github.com/zama-ai/concrete-internal/issues/682
nightly-2024.04.16
2024-04-15 14:09:43 +02:00
Bourgerie Quentin
2d36bb1688 docs(frontend-python): Simplify and unify README and quick start example 2024-04-15 09:44:12 +02:00
Benoit Chevallier-Mames
6de9a388d2 docs(compiler): apply a fix by hand since bad link 2024-04-15 09:37:48 +02:00
Benoit Chevallier-Mames
2424352dbf docs(compiler): update apidocs 2024-04-15 09:37:48 +02:00
Benoit Chevallier-Mames
689f2d3cca docs(compiler): applying more minor fixes to the doc, for coherence 2024-04-15 09:37:48 +02:00
Antoniu Pop
7c59c64564 fix(ci): [distributed CI] fix workflow and move build/test outside docker (needed to run on multiple nodes). nightly-2024.04.13 2024-04-12 09:56:30 +02:00
Antoniu Pop
2eb9c19de2 test(compiler): use mono parameters for dataflow/distributed tests until multi-parameter support is available. 2024-04-12 09:56:30 +02:00
Antoniu Pop
e16487e2c9 fix(compiler): create a copy of return memref for task work functions where the return is not allocated within the function. This is necessary to ensure deallocation cannot occur before the use. 2024-04-12 09:56:30 +02:00
Andi Drebes
204810580d feat(compiler): Add support for tiling of optimizer.partition_frontier 2024-04-12 09:05:41 +02:00
Andi Drebes
fcfaaee2ef feat(compiler): Add support for tiling of fhelinalg.transpose 2024-04-12 09:05:41 +02:00
Andi Drebes
12ab53ba92 refactor(compiler): Remove attribute "tile-sizes" from operations after tiling
The tiling infrastructure preserves attributes of tiled
`linalg.generic` operations, such that the attribute for the tile
sizes specified for the `linalg.generic` operation before tiling is
copied to the `linalg.generic` operation that is part of the generated
IR for a single tile.

This change causes the attribute to be removed after tiling, since it
does not make sense to preserve the attribute for per-tile operations.
2024-04-12 09:05:41 +02:00
Andi Drebes
a8231ce19f feat(compiler): Add support for tiling of fhelinalg.sum 2024-04-12 09:05:41 +02:00
Andi Drebes
c6d2dfc916 feat(compiler): Add support for tiling of fhelinalg.apply_multi_lookup_table 2024-04-12 09:05:41 +02:00
Andi Drebes
46e28c648f feat(compiler): Add support for tiling of fhelinalg.apply_mapped_lookup_table 2024-04-12 09:05:41 +02:00
Andi Drebes
58c1315205 feat(compiler): Copy tile sizes for all matmul ops when lowering to linalg.generic
For now, the attribute "tile-sizes" is copied from the FHELinalg
operation to the corresponding `linalg.generic` operation only for
multiplications of encrypted matrices with plaintext matrices.

The changes of this commit also cause the attribute to be copied for
multiplications between plaintext ad ciphertext matrices, as well as
for multiplications between two ciphertext matrices.
2024-04-12 09:05:41 +02:00
Andi Drebes
09a6702301 feat(compiler): Add support for tiling of element-wise FHELinalg operations 2024-04-12 09:05:41 +02:00
Miles
5d68424531 fix typos nightly-2024.04.12 2024-04-11 14:50:16 +02:00
Alexandre Péré
5537eb53de fix(frontends): add catch-all for exceptions 2024-04-11 14:49:24 +02:00
Bourgerie Quentin
f6308394b6 test(compiler): Test compress_input_ciphertext + simulation 2024-04-11 14:42:28 +02:00
Bourgerie Quentin
f6aa484ee2 fix(compiler): Do not use input ciphertext compression in simulation
close zama-ai/concrete-internal#656
2024-04-11 14:42:28 +02:00
Benoit Chevallier-Mames
ca7fb67d66 fix(compiler): fix the compilation doc 2024-04-11 14:38:52 +02:00
Andi Drebes
f506f5f7e3 test(compiler): Add check tests for pass hoisting RT.await_future operations nightly-2024.04.11 nightly-2024.04.10 2024-04-08 16:16:07 +02:00
Andi Drebes
7cf5483425 test(compiler): Add tests for tiling generating partial tiles 2024-04-08 16:16:07 +02:00
Andi Drebes
d620fa9a44 feat(compiler): Add pass hoisting RT.await_future out of scf.forall loops
The new pass hoists `RT.await_future` operations whose results are
yielded by scf.forall operations out of the loops in order to avoid
over-synchronization of data-flow tasks.

E.g., the following IR:

```
scf.forall (%arg) in (16)
  shared_outs(%o1 = %sometensor, %o2 = %someothertensor)
  -> (tensor<...>, tensor<...>)
{
  ...
  %rph = "RT.build_return_ptr_placeholder"() :
    () -> !RT.rtptr<!RT.future<tensor<...>>>
  "RT.create_async_task"(..., %rph, ...) { ... } : ...
  %future = "RT.deref_return_ptr_placeholder"(%rph) :
    (!RT.rtptr<!RT.future<...>>) -> !RT.future<tensor<...>>
  %res = "RT.await_future"(%future) : (!RT.future<tensor<...>>) -> tensor<...>
  ...
  scf.forall.in_parallel {
    ...
    tensor.parallel_insert_slice %res into %o1[..., %arg2, ...] [...] [...] :
      tensor<...> into tensor<...>
    ...
  }
}
```

is transformed into:

```
%tensoroffutures = tensor.empty() : tensor<16x!RT.future<tensor<...>>>

scf.forall (%arg) in (16)
  shared_outs(%otfut = %tensoroffutures, %o2 = %someothertensor)
  -> (tensor<...>, tensor<...>)
{
  ...
  %rph = "RT.build_return_ptr_placeholder"() :
    () -> !RT.rtptr<!RT.future<tensor<...>>>
  "RT.create_async_task"(..., %rph, ...) { ... } : ...
  %future = "RT.deref_return_ptr_placeholder"(%rph) :
    (!RT.rtptr<!RT.future<...>>) -> !RT.future<tensor<...>>
  %wrappedfuture = tensor.from_elements %future :
    tensor<1x!RT.future<tensor<...>>>
  ...
  scf.forall.in_parallel {
    ...
    tensor.parallel_insert_slice %wrappedfuture into %otfut[%arg] [1] [1] :
      tensor<1xRT.future<tensor<...>>> into tensor<16x!RT.future<tensor<...>>>
    ...
  }
}

scf.forall (%arg) in (16) shared_outs(%o = %sometensor) -> (tensor<...>) {
  %future = tensor.extract %tensoroffutures[%arg] :
    tensor<4x!RT.future<tensor<...>>>
  %res = "RT.await_future"(%future) : (!RT.future<tensor<...>>) -> tensor<...>
  scf.forall.in_parallel {
    tensor.parallel_insert_slice %res into %o[..., %arg, ...] [...] [...] :
      tensor<...> into tensor<...>
  }
}
```
2024-04-08 16:16:07 +02:00
Andi Drebes
fd513f1e6e feat(compiler): Add support for various memref operations for RT task bufferization
This adds support for `memref.alloc`, `memref.load`, `memref.store`,
`memref.copy` and `memref.subview` to the RT task bufferization pass.
2024-04-08 15:50:48 +02:00
Andi Drebes
999c9a9add feat(compiler): Add support for dynamically-sized memrefs in lowering patterns for RT tasks 2024-04-08 15:50:48 +02:00
Andi Drebes
3ad3dcb08f refactor(compiler): Use signature conversion for conversion of ops with nested blocks
The current scheme used by reinstantiating conversion patterns in
`lib/Conversion/Utils/Dialects` for operations with blocks is to
create a new operation with empty blocks, to move the operations from
the old blocks and then to replace any references to block
arguments. However, such in-place updates of the types of block
arguments leave conversion patterns for operations nested in the
blocks without the ability to determine the original types of values
from before the update.

This change uses proper signature conversion for block arguments, such
that the original types of block arguments with converted types is
preserved, while the new types are made available through the dialect
conversion infrastructure via the respective adaptors.
2024-04-08 15:50:48 +02:00
Andi Drebes
48d919bd25 feat(compiler): Add support for tensor.{from_elements,dim} operations in TFHE passes 2024-04-08 12:02:49 +02:00
Andi Drebes
0c7e3a3518 feat(compiler): Add support for nested Memrefs in memory usage estimator 2024-04-08 12:02:49 +02:00
Andi Drebes
f668e82f20 refactor(compiler): Make type conversion in RT task bufferization recursive 2024-04-08 12:02:49 +02:00
Andi Drebes
68d0014218 feat(compiler): Support non-ciphertext types in TFHE to Concrete conversion patterns
Some of the TFHE to Concrete conversion patterns implicitly assume
that operands are ciphertexts and thus that the converted types have a
higher number of dimensions than the original types. However, for
non-ciphertext types, the number of dimensions before and after the
conversion must be the same.

This commit adds a check to the respective conversion patterns
triggering a simple type conversion that preserves the number of
dimensions for non-ciphertext types.
2024-04-08 12:02:49 +02:00
Andi Drebes
9216c617e4 refactor(compiler): Make type conversion in TFHE global parametrization recursive 2024-04-08 12:02:49 +02:00
Andi Drebes
a855e2bef6 refactor(compiler): Make type conversion in scalar FHE to TFHE conversion recursive 2024-04-08 12:02:49 +02:00
Andi Drebes
74efe8be0a feat(compiler): Declare RT futures usable as element types for memrefs 2024-04-08 12:02:49 +02:00
Andi Drebes
93bb849233 refactor(compiler): Use reinstantiating conversion patterns for RT operations 2024-04-08 12:02:49 +02:00