Commit Graph

93 Commits

Author SHA1 Message Date
Quentin Bourgerie
3330b2f855 fix(compiler): Take care of program with no input when creating client parameters 2022-02-17 15:53:23 +01:00
Quentin Bourgerie
1f3944047f feature(compiler): Introduce the BConcrete Dialect 2022-02-17 15:53:23 +01:00
Antoniu Pop
dddad849c7 enhance(compiler): add --parallelize-loops and --parallelize-dataflow compile flags in addition to --parallelize which enables both. 2022-02-03 10:59:12 +00:00
Antoniu Pop
ec633d57f6 fix(compiler): predicate OpenMP loop parallelization of Linalg on the --parallelize flag. 2022-02-03 10:59:12 +00:00
Quentin Bourgerie
f52965e22f enhance(compiler): Lower linalg to parallel loops and instantiare parallel loops with openmp 2022-02-03 10:59:12 +00:00
youben11
71cccc6a89 fix(mac): add options for required libraries 2022-01-24 17:02:25 +00:00
youben11
faa0abea3e fix: use appropriate cmd for dynamic lib generation
macos and linux require a different cmd
2022-01-24 17:02:25 +00:00
youben11
2926ea6a5e fix: use appropriate type for optimizingTransformer func
the call does return an `std::function` and was being referenced using
an `llvm::function_ref`, which apparently with some optim on Mac was
referncing bad memory location
2022-01-24 17:02:25 +00:00
youben11
2009ee1c94 chore: changing ref to repo after its renaming
also formatting and update the check_license script to match that
2022-01-06 12:40:10 +01:00
Antoniu Pop
93835e0106 fix(runtime): move deallocation of key clones in context destructor. 2022-01-04 12:43:33 +01:00
Antoniu Pop
066a3d4e17 fix(runtime): remove strucutred binding as incompatible with C++ standard version. 2022-01-04 12:43:33 +01:00
Antoniu Pop
84f01f78d6 fix(runtime/DFR): create only one BSK clone per thread and deallocate. 2022-01-04 12:43:33 +01:00
rudy
b8bd38dd6c feat(compiler): Output client parameters when compile to a library
close #198
2022-01-03 17:57:16 +01:00
youben11
f1161f7f6d chore: formatting
Quick fix due to ordering of includes, had to add #include
<mlir/Transforms/DialectConversion.h> to include/concretelang/Conversion/Utils/GenericOpTypeConversionPattern.h
2022-01-03 09:29:19 +01:00
youben11
940cb96be4 chore: rename dialects
HLFHE to FHE
MidLFHE to TFHE
LowLFHE to Concrete
2021-12-29 15:13:34 +01:00
youben11
e73291abdc chore: rename compiler to concrete-compiler
zamalang => concretelang
zamacompiler => concretecompiler
2021-12-29 15:13:34 +01:00
youben11
971cb56182 chore: licensing 2021-12-29 15:13:34 +01:00
rudy
d8fee32cea feat(HLFHELinalg): add apply_mapped_table_lookup
Resolves #182
2021-12-27 15:34:33 +01:00
Antoniu Pop
cdca7ca6f7 feat(compiler): add Dataflow/RT dialect and code generation for dataflow auto parallelization. 2021-12-23 15:57:53 +00:00
Quentin Bourgerie
db683f4a0e feat(compiler/runtime): Introduce a RT dialect 2021-12-23 15:57:53 +00:00
rudy
7a286ba9f9 fix(KeySetCache): handle concurrent generation
Resolves #305
2021-12-20 10:07:48 +01:00
Andi Drebes
f319ba37d2 feat(compiler): Integrate HLFHELinalg tiling passes into compilation pipeline 2021-12-17 16:09:59 +01:00
Andi Drebes
16d0502f56 fix(compiler): Initialize strides of memref parameters when JIT-invoking a function
Upon invocation of a function with memref arguments, the strides for
all dimensions are currently set to 0. This causes dynamic offsets to
be calculated incorrectly in the function body.

This patch replaces the placeholder values with the actual strides for
each dimension and adds a test with parametric slice extraction from a
tensor that triggers dynamic indexing.
2021-12-13 15:20:54 +01:00
youben11
0a7ac76b10 fix: move unique_ptr<T> to be converted to llvm::Expected
gcc 6.3 was complaining about not being able to wrap it in an
llvm::Expected
2021-12-08 17:59:53 +01:00
rudy
74d52cb649 fix(clang): remove an error reported by clang 2021-12-08 11:31:23 +01:00
youben11
58f385b11b fix: move CompilationResult in return
old toolchain (gcc 6.x) was complaining about not being able to convert
CompilationResult to Expected<CompilationResult>
2021-12-07 16:14:44 +01:00
rudy
cc58608589 chore(Lambda): simplify, extract, enhance message for bit width rounding
bit with rounding: 5bit element is widen to a standard 8bit word
2021-12-07 15:29:05 +01:00
rudy
209463be22 chore(Lambda): simplify and extract null parameter detection 2021-12-07 15:29:05 +01:00
rudy
2c56a26c75 fix(Lambda): missing superfluous check in setArg
[----------] Global test environment tear-down
[==========] 7 tests from 1 test suite ran. (1513 ms total)
[  PASSED  ] 7 tests.

  YOU HAVE 2 DISABLED TESTS
2021-12-07 15:29:05 +01:00
rudy
47b4b667bb fix(Lambda): #253 fix the bug about Lambda parameter number verification
[----------] Global test environment tear-down
[==========] 6 tests from 1 test suite ran. (1513 ms total)
[  PASSED  ] 6 tests.

  YOU HAVE 3 DISABLED TESTS

Compared to previous commit, a fatal test is disabled
2021-12-07 15:29:05 +01:00
rudy
9a09afaa80 bug(Lambda): #253 reactivate the bug about Lambda parameter number verification
[----------] Global test environment tear-down
[==========] 6 tests from 1 test suite ran. (1327 ms total)
[  PASSED  ] 5 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Lambda_check_param.scalar_tensor_to_tensor_good_number_param

 1 FAILED TEST
  YOU HAVE 3 DISABLED TESTS
2021-12-07 15:29:05 +01:00
Mayeul@Zama
dad4390518 feat(compiler): add a key cache 2021-12-07 12:11:56 +01:00
Mayeul@Zama
f193fd71a2 fix(compiler): fix mixed-up (g)lwe dimension/size
add dimension -> size conversion
rename k -> glweDimension, polynomialSize -> logPolynomialSize
remove (in/out)putLweSize from KeySwitchLweOp
remove GlweSizeType from LowLFHE
2021-12-07 12:11:56 +01:00
Quentin Bourgerie
fb58dcc59d enhance(compiler/lowlfhe): Give the runtime context as function argument instead of a global variable (close #195) 2021-12-02 10:56:47 +01:00
rudy
975ee86a5e feat(compiler): add --compile option
Resolves #196
2021-11-30 17:42:28 +01:00
rudy
ddca6840f1 fix(CompilerEngine): include mlir diagnostic in error message
Resolves #288

example
before:
Failed to lower to LLVM dialect

after:
Failed to lower to LLVM dialect
test.mlir:3:10: error: unexpected error: 'linalg.copy' op expected indexing_map #1 to have 2 dim(s) to match the number of loops
    %0 = tensor.extract_slice %arg0[0, 0] [3, 1] [1, 1] : tensor<3x2x!HLFHE.eint<3>> to tensor<3x!HLFHE.eint<3>>
         ^
2021-11-29 17:15:46 +01:00
Quentin Bourgerie
b598f0ffca refactor(compiler): Move the conversion from scf to std on the pipeline intead of embedded in a pass 2021-11-12 16:50:29 +01:00
youben11
99cce18c6a feat: get RT lib path from py and use as sharedlib
Try to find the runtime library automatically (should only work on
proper installation of the package), and fail silently by not passing
any RT lib. The RT lib can also be specified manually. The RT lib will
be used as a shared library by the JIT compiler.
2021-11-12 15:06:49 +01:00
Andi Drebes
3118983287 enhance(compiler): Add function JITLambda::Arguments::getResultWidth
Add a new method `JITLambda::Arguments::getResultWidth` returning the
width of a scalar result or the element type of a tensor result at a
given position.
2021-11-12 12:00:39 +01:00
Andi Drebes
e4cd340e36 enhance(compiler): JIT: Support result tensors with arbitrary element types
Currently, `JITLambda::Arguments` assumes result tensors are always
composed of `uint64_t` elements. This change adds support for
arbitrary scalar element types.
2021-11-12 12:00:39 +01:00
youben11
56e261d140 fix(compiler): pass dimensions at TensorLambdaArg creation 2021-11-08 11:55:02 +01:00
Andi Drebes
a670ee3f85 enhance(compiler): Use const pointers in JITLambda::Arguments::setArg
All results in code compiled by zamacompiler are passed as return
values, which means that all tensors passed as function arguments are
constant inputs that are never written.

This patch changes the arguments used as data pointers for input
tensors in `JITLambda::Arguments::setArg()` from `void*` to `const
void*` to emphasize their use as inputs and to allow for constant
arrays to be passed as function inputs.
2021-11-04 19:07:54 +01:00
Andi Drebes
1ad3d57f66 feat(compiler): Add method JITLambda::Argument::getResultType(size_t)
Add a method `JITLambda::Argument::getResultType(size_t pos)` that
returns the type of the result with the index `pos` of a
`JITLambda::Argument`.
2021-11-03 18:32:06 +01:00
Andi Drebes
5999d402ec enhance(compiler): CompilerEngine: Forward parse errors instead of printing them
By default, `mlir::SourceMgrDiagnosticVerifierHandler` used by
`CompilerEngine::compile()` prints parse errors to
`llvm::errs()`. This makes it impossible for a caller of
`CompilerEngine::compile()` to process parse errors or to suppress the
emission of error messages to the standard error stream altogether.

This change captures parse errors in a string via a string-backed
output stream and forwards the error message in the `llvm::Error`
instance of the return value.
2021-11-03 12:05:12 +01:00
Quentin Bourgerie
be92b4580d Merge branch 'master' into hlfhelinalg-binary-op-lowering 2021-10-29 13:47:35 +02:00
Quentin Bourgerie
85d102c9b2 refactor(compiler): Simplify the compiler flow and re enable --passes compiler option
No more need to compute the fhe context at high level
2021-10-29 13:44:34 +02:00
Quentin Bourgerie
41cba63113 refactor(compiler): Move the keyset generation from CompilerEngine to JitCompilerEngine 2021-10-29 13:44:34 +02:00
Andi Drebes
1187cfbd62 refactor(compiler): Refactor CompilerEngine and related classes
This commit contains several incremental improvements towards a clear
interface for lambdas:

  - Unification of static and JIT compilation by using the static
    compilation path of `CompilerEngine` within a new subclass
    `JitCompilerEngine`.

  - Clear ownership for compilation artefacts through
    `CompilationContext`, making it impossible to destroy objects used
    directly or indirectly before destruction of their users.

  - Clear interface for lambdas generated by the compiler through
    `JitCompilerEngine::Lambda` with a templated call operator,
    encapsulating otherwise manual orchestration of `CompilerEngine`,
    `JITLambda`, and `CompilerEngine::Argument`.

  - Improved error handling through `llvm::Expected<T>` and proper
    error checking following the conventions for `llvm::Expected<T>`
    and `llvm::Error`.

Co-authored-by: youben11 <ayoub.benaissa@zama.ai>
2021-10-29 13:44:34 +02:00
Andi Drebes
e76aee7e10 feat(compiler): Add class StreamStringError with a stream interface for llvm::Error
Composing error messages for `llvm::Error` is either done by using
`llvm::createStringError()` with an appropriate format string and
arguments or by writing to a `std::string`-backed
`llvm::raw_string_ostream` and passing the result to
`llvm::make_error<llvm::StringError>()` verbatim.

The new class `StreamStringError` encapsulates the latter solution
into a class with an appropriate stream operator and implicit cast
operators to `llvm::Error` and `llvm::Expected`.

Example usage:

   llvm::Error foo(int i, size_t s, ...) {
      ...
      if(...) {
        return StreamStringError()
               << "Some error message with an integer: "
               << i << " and a size_t: " << s;
      }
      ...
   }
2021-10-29 13:44:34 +02:00
Andi Drebes
b12be45143 feat(compiler): Add method getResultVectorSize to JITLambda::Argument
Add method `JITLambda::Argument::getResultVectorSize` that returns the
number of elements of the result if the result is a vector.
2021-10-29 13:44:34 +02:00