Commit Graph

273 Commits

Author SHA1 Message Date
youben11
36413235c5 feat(compiler): lower HLFHELinalg.apply_multi_lut
Support broadcasting
2021-11-15 11:25:44 +01:00
youben11
ac7f1f5a6b feat(compiler): add HLFHELinalg.apply_multi_lut
Apply a lookup table on the different elements of the encrypted tensors,
but using different lookup tables.

MANP support as well as tests
2021-11-15 11:25:44 +01:00
Ayoub Benaissa
85954f0373 fix: correct PYTHONPATH in docker image 2021-11-15 10:30:43 +01:00
Quentin Bourgerie
f2b5d3316f fix(compiler): don't try to broadcast when dims are equals with value 1 (7c8399)
Removed due of miss conflict resolution
2021-11-15 09:59:52 +01:00
Quentin Bourgerie
5df775a51b feat(compiler): MANP Analysis of HLFHELinalg.matmul (closes #178) 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
3fad9870a8 cleanup(compiler): Fix warning about comparison of expressions with different signedness in TensorOpsToLinalg 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
af0819c403 feat(compiler): Lower HLFHELinalg.matmul_eint_int to linalg.generic (close #177) 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
b598f0ffca refactor(compiler): Move the conversion from scf to std on the pipeline intead of embedded in a pass 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
12b4771405 feat(compiler/linalg): Definition of HLFHELinalg.matmul_eint_int (closes #176) 2021-11-12 16:50:29 +01:00
youben11
7c83994c7f fix(compiler): don't try to broadcast when dims are equals with value 1 2021-11-12 15:07:08 +01:00
youben11
99cce18c6a feat: get RT lib path from py and use as sharedlib
Try to find the runtime library automatically (should only work on
proper installation of the package), and fail silently by not passing
any RT lib. The RT lib can also be specified manually. The RT lib will
be used as a shared library by the JIT compiler.
2021-11-12 15:06:49 +01:00
Andi Drebes
e356b8f797 fix(ci): Conformance test: Abort on failure in any command in the run section
By default, all commands in the `run` section of
`.github/workflows/conformance.yml` are executed, regardless of their
return values and the return value of the last command is used as the
result for the entire flow. Failing test followed by passing tests
thus remain undetected.

This change forces the use of the bash shell and causes it to exit
immediately with a non-zero return value when one of the commands
fails using `set -e`.
2021-11-12 15:05:55 +01:00
youben11
3246f8b035 chore: remove build files from docker image
- Reduce the size of the docker image
- Make sure ccache is always used => faster CI tests
2021-11-12 15:05:55 +01:00
Ayoub Benaissa
2088e86edd ci: run concretelib compatibility-check once a week 2021-11-12 12:12:07 +01:00
Andi Drebes
d1960a2a7f test(compiler): Add test for lambdas returning tensors of different integer types
Add tests with lambdas returning constant n-dimensional tensors
composed of different integer types (uint8_t, uint16_t, uint32_t, and
uint64_t).
2021-11-12 12:00:39 +01:00
Andi Drebes
f54c0dd8d8 enhance(compiler): JitCompilerEngine: Add support for arbitrary int result tensors
Add support for result tensors composed of uint8_t, uint16_t, uint32_t
and uint64_t elements, replacing the current implementation, which
only supports uint64_t tensors.
2021-11-12 12:00:39 +01:00
Andi Drebes
3118983287 enhance(compiler): Add function JITLambda::Arguments::getResultWidth
Add a new method `JITLambda::Arguments::getResultWidth` returning the
width of a scalar result or the element type of a tensor result at a
given position.
2021-11-12 12:00:39 +01:00
Andi Drebes
e4cd340e36 enhance(compiler): JIT: Support result tensors with arbitrary element types
Currently, `JITLambda::Arguments` assumes result tensors are always
composed of `uint64_t` elements. This change adds support for
arbitrary scalar element types.
2021-11-12 12:00:39 +01:00
Andi Drebes
3bf493e83c ci: Force use of ccache in conformance.yml
Set the make variable `CCACHE` to `ON` in order to force the use of
`ccache` during compilation of the code. For unknown reasons, the
build directory must also be removed. Otherwise, a complete rebuild
circumventing the cache is triggered and `ccache` does not have any
effect on compilation time.

Debug information printing `ccache` statistics before and after the
build has also been added.
2021-11-10 15:46:48 +01:00
youben11
a888a6b329 typo: bad function name 2021-11-10 15:24:31 +01:00
youben11
6df9f09e48 feat(compiler): lower HLFHELinalg.neg_eint 2021-11-10 15:24:31 +01:00
youben11
99d6d11616 feat(compiler): support HLFHELinalg.neg_eint in MANP 2021-11-10 15:24:31 +01:00
youben11
d9adbab036 feat(compiler): add HLFHELinalg.neg_eint op 2021-11-10 15:24:31 +01:00
youben11
ba40f5582a test(compiler): test HLFHE.neg_eint execution 2021-11-10 15:24:31 +01:00
youben11
efacd7d8a1 feat(compiler): support HLFHE.neg_eint in MANP 2021-11-10 15:24:31 +01:00
youben11
08869bc998 chore: separate unittests into different executables
it's easier to run a specific set of tests if they are into separate
executable. We shouldn't have to run them all for testing a specific set
of tests
2021-11-10 15:24:21 +01:00
youben11
aee493392b feat(compiler): lower neg_eint from HLFHE to LowLFHE 2021-11-10 15:24:21 +01:00
youben11
c1cd6bb380 feat(compiler): add MidLFHE.neg_glwe op 2021-11-10 15:24:06 +01:00
youben11
cc4e403e7a feat(compiler): add HLFHE.neg_eint op 2021-11-10 14:52:35 +01:00
Andi Drebes
47cb4b0e70 fix(compiler): HLFHELinalg JIT tests: Use immutable ArrayRefs for tensor arguments
Fix compiler warnings related to the conversion from const pointers to
mutable pointers in `tests/unittest/end_to_end_jit_hlfhelinalg.cc`.
2021-11-10 11:25:35 +01:00
Andi Drebes
8cd893a7fd fix(compiler): JIT unit tests: Remove unused variables
Remove unused variable `pos` in `End2EndJit_ClearTensor_2D::extract`
and `End2EndJit_EncryptedTensor_2D::extract` to avoid warnings on
compilation.
2021-11-10 11:25:35 +01:00
Andi Drebes
5cc551dd4f fix(compiler): JIT unit tests: Don't compare expressions of different signedness
Fix compiler warnings in `end_to_end_jit_clear_tensor.cc` and
`end_to_end_jit_encrypted_tensor.cc` due to the comparison of integer
expressions of different signedness.
2021-11-10 11:25:35 +01:00
rudy
97389a3b06 Move HLFHE.dot_eint_int to HLFHELinalg.
Resolves #183
2021-11-09 11:39:08 +01:00
youben11
0670d4efc1 fix(python): add mlir[.dialects] as namespace packages
I initially thought there were missing init files, but it was intended
that mlir and mlir.dialects are namespace packages
2021-11-08 15:29:57 +01:00
Andi Drebes
c401b5ae44 test(compiler): Add JIT test returning multi-dimensional TensorLambdaArgument 2021-11-08 12:00:20 +01:00
youben11
59e859177c refactor: replace ExecutionArg by TensorLambdaArg
This add support for tensor inputs from the python bindings
2021-11-08 11:55:02 +01:00
youben11
56e261d140 fix(compiler): pass dimensions at TensorLambdaArg creation 2021-11-08 11:55:02 +01:00
youben11
b501e3d6c0 feat(python): support functions returning tensors 2021-11-08 11:55:02 +01:00
youben11
badc8e44bf test(comiler): add tests for result tensor when calling Lambda::operator()
Lambda::operator()<unique_ptr<LambdaArgument>> was giving bad results
when using tensors due to bad references. Now it's fixed and we want to
spot that in the future.
2021-11-04 19:07:54 +01:00
Andi Drebes
c92f047721 fix(compiler): TensorLambdaArgument: Copy input data instead of using a reference
`TensorLambdaArgument` uses an `llvm::MutableArrayRef` to reference
the tensor values. This prevents temporary tensors from being used as
an argument, due to the data of the `TensorLambdaArgument` being
accessed after the destruction of the temporary.

This patch changes the type of the data field of
`TensorLambdaArgument` from `llvm::MutableArrayRef` to `std::vector`
and causes input data to be copied in order to guarantee that all data
remains available until invocation of the destructor.
2021-11-04 19:07:54 +01:00
Andi Drebes
a670ee3f85 enhance(compiler): Use const pointers in JITLambda::Arguments::setArg
All results in code compiled by zamacompiler are passed as return
values, which means that all tensors passed as function arguments are
constant inputs that are never written.

This patch changes the arguments used as data pointers for input
tensors in `JITLambda::Arguments::setArg()` from `void*` to `const
void*` to emphasize their use as inputs and to allow for constant
arrays to be passed as function inputs.
2021-11-04 19:07:54 +01:00
Andi Drebes
2033a70ad2 ci: Use ccache when building the project
In some cases, even small changes in zamacompiler lead to the
recompilation of the LLVM / MLIR sources, causing significant delay in
the CI. This change forces the use of ccache to speed up recompilation
after the project has been built at least once.
2021-11-04 16:01:27 +01:00
Andi Drebes
d6dc9874d9 enhance(compiler): Add support for ccache in Makefile
This adds a new variable `CCACHE` to `Makefile` that specifies whether
CMake is instructed upon configuration of the project to use the
default, plain C++ compiler or ccache as a wrapper.

By default, ccache is not used. To override the default value and
force the use of ccache, the variable `CCACHE` must be set to `ON`,
e.g., using `make CCACHE=ON`.
2021-11-04 16:01:27 +01:00
Andi Drebes
3d6ad06f46 test(compiler): Add tests for JitCompilerEngine::Lambda with a generic result 2021-11-03 18:32:06 +01:00
Andi Drebes
9040e5ab00 feat(compiler): Add generic overload for result of JitCompilerEngine::Lambda
This adds a new overload for `JitCompilerEngine::Lambda::operator()`,
returning a result wrapped in a `std::unique_ptr<LambdaArgument>` with
meta information about the result. This allows for generic invocations
of JitCompilerEngine::Lambda::operator(), where the result type is
unknown before the invocation.
2021-11-03 18:32:06 +01:00
Andi Drebes
1ad3d57f66 feat(compiler): Add method JITLambda::Argument::getResultType(size_t)
Add a method `JITLambda::Argument::getResultType(size_t pos)` that
returns the type of the result with the index `pos` of a
`JITLambda::Argument`.
2021-11-03 18:32:06 +01:00
youben11
97ee8134ed fix(python): register hlfhelinalg dialect 2021-11-03 15:37:43 +01:00
Andi Drebes
5999d402ec enhance(compiler): CompilerEngine: Forward parse errors instead of printing them
By default, `mlir::SourceMgrDiagnosticVerifierHandler` used by
`CompilerEngine::compile()` prints parse errors to
`llvm::errs()`. This makes it impossible for a caller of
`CompilerEngine::compile()` to process parse errors or to suppress the
emission of error messages to the standard error stream altogether.

This change captures parse errors in a string via a string-backed
output stream and forwards the error message in the `llvm::Error`
instance of the return value.
2021-11-03 12:05:12 +01:00
Andi Drebes
0cb5007c3c enhance(compiler): Python bindings: Forward llvm::Expected error messages in exceptions
The code in `lib/CAPI/Support/CompilerEngine.cpp` invokes several
functions returning an `llvm::Expected<T>`. When those fail, the error
message retrieved from the error object the `llvm::Expected<T>`
instance is written to the standard error stream via
`mlir::zamalang::log_error()` and an exception with a more generic
error message is thrown.

This causes errors to show up on the standard error stream in tests
generating errors on purpose and checking them, e.g.:

  ```
  tests/python/test_compiler_engine.py::test_compile_invalid[not
  @main] Compilation failed: cannot find the function for generate
  client parameters PASSED
  ```

This patch forwards the error message from an `llvm::Expected<T>`
instance in a runtime exception rather than writing it to the standard
error stream. Since exceptions are properly caught by the tests, no
errors show up during testing.
2021-11-02 16:34:42 +01:00
youben11
766b42fb49 feat(python): generate the hlfhelinalg dialect module 2021-11-02 11:45:18 +01:00