Commit Graph

219 Commits

Author SHA1 Message Date
Quentin Bourgerie
be92b4580d Merge branch 'master' into hlfhelinalg-binary-op-lowering 2021-10-29 13:47:35 +02:00
Quentin Bourgerie
e66fb20c4f test(compiler): Re introduce tests that was removed by a prior refacto 2021-10-29 13:44:34 +02:00
Quentin Bourgerie
85d102c9b2 refactor(compiler): Simplify the compiler flow and re enable --passes compiler option
No more need to compute the fhe context at high level
2021-10-29 13:44:34 +02:00
Quentin Bourgerie
41cba63113 refactor(compiler): Move the keyset generation from CompilerEngine to JitCompilerEngine 2021-10-29 13:44:34 +02:00
Andi Drebes
1187cfbd62 refactor(compiler): Refactor CompilerEngine and related classes
This commit contains several incremental improvements towards a clear
interface for lambdas:

  - Unification of static and JIT compilation by using the static
    compilation path of `CompilerEngine` within a new subclass
    `JitCompilerEngine`.

  - Clear ownership for compilation artefacts through
    `CompilationContext`, making it impossible to destroy objects used
    directly or indirectly before destruction of their users.

  - Clear interface for lambdas generated by the compiler through
    `JitCompilerEngine::Lambda` with a templated call operator,
    encapsulating otherwise manual orchestration of `CompilerEngine`,
    `JITLambda`, and `CompilerEngine::Argument`.

  - Improved error handling through `llvm::Expected<T>` and proper
    error checking following the conventions for `llvm::Expected<T>`
    and `llvm::Error`.

Co-authored-by: youben11 <ayoub.benaissa@zama.ai>
2021-10-29 13:44:34 +02:00
youben11
d738104c4b fix: use std::string for JIT entrypoint funcname
As we store the funcname for the entrypoint for later use, a pointer
might point to some random memory when used as there is no special
management for that name at the higher levels.
2021-10-29 13:44:34 +02:00
Andi Drebes
3ae924e174 enhance(compiler): Disable RTTI for unit tests
LLVM and MLIR are compiled without runtime type information
(RTTI). Use the same restrictions for unit tests to avoid linker
errors related to typeinfo when building the test executable.
2021-10-29 13:44:34 +02:00
Andi Drebes
e76aee7e10 feat(compiler): Add class StreamStringError with a stream interface for llvm::Error
Composing error messages for `llvm::Error` is either done by using
`llvm::createStringError()` with an appropriate format string and
arguments or by writing to a `std::string`-backed
`llvm::raw_string_ostream` and passing the result to
`llvm::make_error<llvm::StringError>()` verbatim.

The new class `StreamStringError` encapsulates the latter solution
into a class with an appropriate stream operator and implicit cast
operators to `llvm::Error` and `llvm::Expected`.

Example usage:

   llvm::Error foo(int i, size_t s, ...) {
      ...
      if(...) {
        return StreamStringError()
               << "Some error message with an integer: "
               << i << " and a size_t: " << s;
      }
      ...
   }
2021-10-29 13:44:34 +02:00
Andi Drebes
b12be45143 feat(compiler): Add method getResultVectorSize to JITLambda::Argument
Add method `JITLambda::Argument::getResultVectorSize` that returns the
number of elements of the result if the result is a vector.
2021-10-29 13:44:34 +02:00
Andi Drebes
d4b4839d6e fix(compiler): Take into account function parameters in MaxMANPPass
When determining the maximum MANP and precision, the `MaxMANPPass`
only takes into account results generated by an operation, but ignores
function parameters. However, encrypted function parameters are
assumed to have a MANP value of 1 and can have an arbitrary
precision. This patch takes into account function arguments by using
their default MANP values and the extracting their precision.
2021-10-29 13:44:34 +02:00
Andi Drebes
0423a05db8 feat(compiler): Add support for tensor.extract operations in MANP pass
Add support for `tensor.extract` operations in the MANP pass. This
currently only supports extract operations on tensors of encrypted
integers, which are passed as function arguments, e.g.:

 func @extract_ith(%t: tensor<10x!HLFHE.eint<5>>, %i: index) -> !HLFHE.eint<5>{
   %c = tensor.extract %t[%i] : tensor<10x!HLFHE.eint<5>>
   return %c : !HLFHE.eint<5>
 }
2021-10-29 13:44:34 +02:00
Quentin Bourgerie
31d9f568fa refactor(test): Move HLFHELinalg e2e test 2021-10-29 12:18:30 +02:00
Quentin Bourgerie
a135d05e4d feat(compiler): Lowering of HLFHELinalg.mul_eint_int 2021-10-29 12:18:30 +02:00
Quentin Bourgerie
0b5ee3497a feat(compiler): Lowering of HLFHELinalg.sub_int_eint 2021-10-29 12:18:22 +02:00
Quentin Bourgerie
b81cb978d0 feat(compiler): Lowering of HLFHELinalg.add_eint_int 2021-10-29 12:17:38 +02:00
Quentin Bourgerie
a804e90291 fix(compiler): Fix lowering MidLFHE.add_glwe_int, the operand type can be already lowered 2021-10-29 12:17:38 +02:00
Quentin Bourgerie
ba54560680 feat(compiler): Lower HLFHELinalg binary operators 2021-10-29 12:17:38 +02:00
Quentin Bourgerie
dea1be9d52 feat(compiler): HLFHELinalg.apply_lookup_table definition 2021-10-29 12:17:38 +02:00
Quentin Bourgerie
f72d51d98d feat(compiler): HLFHELinalg.mul_eint_int 2021-10-29 12:17:38 +02:00
Quentin Bourgerie
64e327209f feat(compiler): HLFHELinalg.sub_int_eint 2021-10-29 12:17:32 +02:00
Quentin Bourgerie
3b02a16f7b feat(compiler): HLFHELinalg.add_eint definition 2021-10-29 12:16:41 +02:00
Quentin Bourgerie
0d4e10169b feat(compiler): Introduce the HLFHELinalg dialect and a first operator HLFHELinalg.add_eint_int 2021-10-29 12:16:41 +02:00
youben11
941465060e build: setup target and tools for building tarballs 2021-10-28 15:59:31 +01:00
youben11
dc2d6a362e chore: stop using "build" target in CI 2021-10-28 15:59:31 +01:00
Andi Drebes
2c63018ed2 fix(compiler): Makefile: Make targets without file dependencies PHONY
Most of the targets in `Makefile` do not deped on files produced by
other targets and use target names solely for dependency
management. Make all such targets PHONY in order to avoid that they
are skipped accidentially when a file with the same name is present.
2021-10-27 13:39:35 +02:00
Andi Drebes
e7b2582639 fix(compiler): Makefile: Invoke CMake for each Python target separately
The Makefile target `python-bindings` invokes CMake with multiple
targets specified after the `--target` commandline option. However, as
per the CMake manpage, only one target may be specified at once.

This changes the single invocation of CMake to separate invocations
for each target.

Tested with CMake version 3.13.4.
2021-10-27 13:39:35 +02:00
Andi Drebes
527887bbf9 fix(compiler): Makefile: Do not let target 'build' depend on directory
The target `build` creates a build directory with the same name and
initializes through an invocation of CMake. Regardless of the success
or failure of the CMake invocation, all subsequent invocations of the
target do not invoke CMake anymore, as the target's prerequisites are
satisfied through the existence of the build directory created upon
the first invocation.

This patch changes the dependencies to the build directory with an
intermediate target that depends on a stamp file that is only created
when the first CMake invocation in the build directory succeeds.
2021-10-27 13:39:35 +02:00
youben11
ab7a208112 fix: store OpPassManager& before call to addPass
weird bug with c++ 6.3 on the manylinux_2_24 image (Debian9) generating
erroneous asm instructions for call to nest on PassManager
2021-10-26 16:31:04 +02:00
youben11
5a2e9460fb build: setup build tools for python package
- Docker image to build wheels for linux_x86_64 CPython 3.[8,9,10] with
  GLIBC >= 2.24
- Specify which Python to use in Makefile
- Fix cmake build to handle when libpython isn't available (cmake>3.18)
2021-10-26 16:31:04 +02:00
youben11
2e3560654f chore: bump LLVM to 55e76c70 2021-10-26 16:31:04 +02:00
youben11
7b29600721 refactor: don't use designated initializers 2021-10-26 16:31:04 +02:00
Quentin Bourgerie
247cc489c5 refactor(compiler): Refactor JITLambda::Argument::setArg 2021-10-21 14:40:07 +02:00
Quentin Bourgerie
b5f68c20c7 fix(compiler): Remove the default constraint since tensor operators are supported 2021-10-20 16:21:52 +02:00
Quentin Bourgerie
1c202ebaff enhance(compiler): Support of tensor operators in MANP pass (close #169) 2021-10-20 16:21:52 +02:00
Quentin Bourgerie
fcc992db2b chore(compiler): Rename std tensor test to clear tensor 2021-10-15 16:48:03 +02:00
Quentin Bourgerie
b6be5b5743 test(compiler): Test tensor.extract_slice and tensor.insert_slice with encrypted tensor #168 2021-10-15 16:47:53 +02:00
Quentin Bourgerie
d93da5bdc0 enhance(compiler): Handle tensor.extract_slice and tensor.insert_slice in the FHE lowering #168 2021-10-15 16:47:35 +02:00
Quentin Bourgerie
3e27592cd1 test(compiler): Add test on 2D clear tensor, tensor.extract, tensor.extract_slice, tensor.insert_slice #168 2021-10-15 16:47:35 +02:00
Quentin Bourgerie
3b2f21a715 enhance(compiler): Handle multi-dimensional tensor #167 2021-10-15 16:44:19 +02:00
Quentin Bourgerie
8aaeff525c test(compiler): Retry when an apply_lookup_table fail 2021-10-13 10:28:35 +02:00
Quentin Bourgerie
966160bc2c fix(compiler): apply_lookup_table operators take tensor of i64
We decide to make this choice as they are issue to crate tensor of custom integer type in python.
+ we don't do the integer extension before convert to the concrete CAPI that requires i64
2021-10-13 10:28:35 +02:00
Quentin Bourgerie
b7e9f12799 test(compiler): Enhance HLFHE.apply_lookup_table test, still need to make some distribution test 2021-10-13 10:28:35 +02:00
Quentin Bourgerie
8f4da14bdb fix(compiler): Pad constant tabulated lambda when the input precision of apply_lookup_table has been changed 2021-10-13 10:28:35 +02:00
youben11
6204f93878 fix: call getChecked to better handle failure 2021-10-12 16:57:18 +02:00
youben11
33d75a92f4 ci: remove logging of LLVM commit
the loggign is actually incorrect, as this would be the version of the
repo, not the one in the env, so it doesn't even make sense
2021-10-12 11:50:15 +01:00
youben11
01757fa6d5 fix: forward errors instead of creating new ones
LLVM errors should be handled/consumed. Creating a new one and leaving
the previous one alive will crash the compiler. Whenever we don't want a
crash (e.g. logging the error is enough), but still wanna continue the
execution, we can just consume it.
2021-10-12 11:50:15 +01:00
github-actions
1c030ac10b chore: update LLVM to f1e9ecea4 2021-10-11 01:45:15 +00:00
github-actions
e682394a6d chore: update LLVM to 02ac5e5cf 2021-10-09 11:44:46 +01:00
youben11
952968567b ci: update msg of LLVM auto-update commit 2021-10-09 11:44:46 +01:00
youben11
b30e09546b test: use 6bits only for LUT tests 2021-10-08 16:37:43 +01:00