- no more Concrete ciphertext/plaintext types: they are represented using standard MLIR types (int/tensor)
- Technically BConcrete was renamed to Concrete, and old Concrete was
removed
- TFHE -> Concrete now takes into account the conversion of tensor of
ciphertext into tensors of an additional dimension (LWE dim)
- Bufferization now works in Concrete
- Old Concrete optimization were moved to TFHE
- Concrete is now the dialect that lowers to CAPI calls
- TFHE -> Concrete now uses OpConversionPattern and is much cleaner in
terms of type conversion
- Disabled tests for batching, as there was something weird about it:
batchable operations implemented in Concrete but pass run in FHELinalg
The switch to reinstantiating conversion patterns for the conversion
from FHE to TFHE in commit 73fd6c5fe7
caused all attributes of `scf.for` operations to be dropped during the
conversion. This included the custom attribute `parallel`, which is
exploited further down the compilation pipeline to generate parallel
code. As a result, the performance of end-to-end benchmarks dropped
significantly.
This patch copies all attributes of `scf.for` operations upon
reinstantiation, which solves the performance regression.
Use `OpConversionPattern` instead of `OpRewritePattern` for operation
conversion during dialect conversion. This makes explicit and in-place
type conversions unnecessary, since `OpConversionPattern` already
properly converts operand types and provides them to the rewrite rule
through an operation adaptor.
The main contributions of this commit are the two class templates
`TypeConvertingReinstantiationPattern` and
`GenericOneToOneOpConversionPattern`.
The former allows for the definition of a simple replacement rule that
re-instantiates an operation after the types of its operands have been
converted. This is especially useful for type-polymorphic operations
during dialect conversion.
The latter allows for the definition of patterns, where one operation
needs to be replaced with a different operation after conversion of
its operands.
The default implementations for the class templates provide
conversions rules for operations that have a generic builder method
that takes the desired return type(s), the operands and (optionally) a
set of attributes. How attributes are discarded during a conversion
(either by omitting the builder argument or by passing an empty set of
attributes) can be defined through specialization of
`ReinstantiationAttributeDismissalStrategy`.
Custom replacement rules that deviate from the scheme above should be
implemented by specializing
`TypeConvertingReinstantiationPattern::matchAndRewrite()` and
`GenericOneToOneOpConversionPattern::matchAndRewrite()`.
This adds a new option `--unroll-loops-with-sdfg-convertible-ops`,
which causes loops containing SDFG-convertible operations to be fully
unrolled upon the extraction of SDFG-operations using the
`--emit-sdfg-ops` switch. This avoids constant roundtrips between an
SDFG-capable accelerator and the host during execution of a loop.
The option is limited to `scf.for` loops with static bounds and a
static step size. Since full unrolling of loops with large bounds
results in a large number of operations, the option is disabled by
default.
This adds a new pass `ExtractSDGOps`, which scans a function for
operations that implement `SDFGConvertibleOpInterface`, replaces them
with SDFG processes and constructs an SDFG graph around the processes.
Initialization and teardown of the SDFG graph are embedded into the
function and take place at the beginning of the function and before
the function's terminator, respectively.
The pass can be invoked using concretecompiler by specifying the new
compilation option `--emit-sdfg-ops` or programmatically on a
`CompilerEngine` using the new compilation option `extractSDFGOps`.
- unify CPU and GPU bootstrapping operations
- remove operations to build GLWE from table: this is now done in
wrapper functions
- remove GPU memory management operations: done in wrappers now, but we
will have to think about how to deal with it later in MLIR
For now what it works are only levelled ops with user parameters. (take a look to the tests)
Done:
- Add parameters to the fhe parameters to support CRT-based large integers
- Add command line options and tests options to allows the user to give those new parameters
- Update the dialects and pipeline to handle new fhe parameters for CRT-based large integers
- Update the client parameters and the client library to handle the CRT-based large integers
Todo:
- Plug the optimizer to compute the CRT-based large interger parameters
- Plug the pbs for the CRT-based large integer
when lowering the linalg conv operation into scf, then bufferized, there
is a cast operation on an allocated buffer to the type of a
memref with different strides (casued by the slice). This incompatible
cast makes the lowering fails.
Rebase to llvm-project at 3f81841474fe with a pending upstream patch
for arbitrary element types in linalg named operations.
Co-authored-by: Ayoub Benaissa <ayoub.benaissa@zama.ai>
This commit is introduced because python bindings for `tensor.from_elements` are not generated automatically. Previously, we overcame this with string manipulation, but with the latest version of the compiler, it became a problem. This commit should be reverted eventually. See https://discourse.llvm.org/t/cannot-create-tensor-from-elements-operation-from-python-bindings/4768 for the discussion in LLVM forums.
