Commit Graph

162 Commits

Author SHA1 Message Date
Andi Drebes
2c63018ed2 fix(compiler): Makefile: Make targets without file dependencies PHONY
Most of the targets in `Makefile` do not deped on files produced by
other targets and use target names solely for dependency
management. Make all such targets PHONY in order to avoid that they
are skipped accidentially when a file with the same name is present.
2021-10-27 13:39:35 +02:00
Andi Drebes
e7b2582639 fix(compiler): Makefile: Invoke CMake for each Python target separately
The Makefile target `python-bindings` invokes CMake with multiple
targets specified after the `--target` commandline option. However, as
per the CMake manpage, only one target may be specified at once.

This changes the single invocation of CMake to separate invocations
for each target.

Tested with CMake version 3.13.4.
2021-10-27 13:39:35 +02:00
Andi Drebes
527887bbf9 fix(compiler): Makefile: Do not let target 'build' depend on directory
The target `build` creates a build directory with the same name and
initializes through an invocation of CMake. Regardless of the success
or failure of the CMake invocation, all subsequent invocations of the
target do not invoke CMake anymore, as the target's prerequisites are
satisfied through the existence of the build directory created upon
the first invocation.

This patch changes the dependencies to the build directory with an
intermediate target that depends on a stamp file that is only created
when the first CMake invocation in the build directory succeeds.
2021-10-27 13:39:35 +02:00
youben11
ab7a208112 fix: store OpPassManager& before call to addPass
weird bug with c++ 6.3 on the manylinux_2_24 image (Debian9) generating
erroneous asm instructions for call to nest on PassManager
2021-10-26 16:31:04 +02:00
youben11
5a2e9460fb build: setup build tools for python package
- Docker image to build wheels for linux_x86_64 CPython 3.[8,9,10] with
  GLIBC >= 2.24
- Specify which Python to use in Makefile
- Fix cmake build to handle when libpython isn't available (cmake>3.18)
2021-10-26 16:31:04 +02:00
youben11
7b29600721 refactor: don't use designated initializers 2021-10-26 16:31:04 +02:00
Quentin Bourgerie
8aaeff525c test(compiler): Retry when an apply_lookup_table fail 2021-10-13 10:28:35 +02:00
Quentin Bourgerie
966160bc2c fix(compiler): apply_lookup_table operators take tensor of i64
We decide to make this choice as they are issue to crate tensor of custom integer type in python.
+ we don't do the integer extension before convert to the concrete CAPI that requires i64
2021-10-13 10:28:35 +02:00
Quentin Bourgerie
b7e9f12799 test(compiler): Enhance HLFHE.apply_lookup_table test, still need to make some distribution test 2021-10-13 10:28:35 +02:00
Quentin Bourgerie
8f4da14bdb fix(compiler): Pad constant tabulated lambda when the input precision of apply_lookup_table has been changed 2021-10-13 10:28:35 +02:00
youben11
6204f93878 fix: call getChecked to better handle failure 2021-10-12 16:57:18 +02:00
youben11
01757fa6d5 fix: forward errors instead of creating new ones
LLVM errors should be handled/consumed. Creating a new one and leaving
the previous one alive will crash the compiler. Whenever we don't want a
crash (e.g. logging the error is enough), but still wanna continue the
execution, we can just consume it.
2021-10-12 11:50:15 +01:00
youben11
b30e09546b test: use 6bits only for LUT tests 2021-10-08 16:37:43 +01:00
youben11
5a308ddc75 fix(python): add missing import
Also added a test to catch future similar failure. Mainly, we didn't
have tests that imported and used the HLFHE dialect from the python
bindings
2021-10-08 14:21:45 +01:00
youben11
ebdc95f827 chore: update llvm and fix API breaks 2021-10-08 12:11:34 +01:00
youben11
bfc763144c ci: new unified build system 2021-10-07 14:38:50 +01:00
youben11
2972fa4403 refactor(python): rework the bindings with latest MLIR version
- Go through CAPI for python bindings
- Consuming LLVM errors in CAPI: fixes previous issue which made this
  impossible in the python bindings
2021-10-07 14:38:50 +01:00
youben11
3406b322d5 fix: prefix builtin ops w/ builtin. 2021-10-07 14:38:50 +01:00
youben11
09ad814e95 fix: use ref to PatternRewriter in NativeCodeCall 2021-10-07 14:38:50 +01:00
youben11
7a48900fae chore: build zamacompiler as an LLVM external proj 2021-10-07 14:38:50 +01:00
youben11
450925b7bf chore(python test): clean test case by adding ids 2021-10-07 14:38:50 +01:00
Mayeul@Zama
cb0e8b0e98 doc(compiler): update the instructions to build and test the compiler in the README 2021-09-29 11:16:05 +02:00
Andi Drebes
2acfa63eb7 feat(compiler): Determine FHE circuit constraints instead of using default values
This replaces the default FHE circuit constrains (maximum encrypted
integer width of 7 bits and a Minimal Arithmetic Noise Padding of 10
with the results of the `MaxMANP` pass, which determines these values
automatically from the input program.

Since the maximum encrypted integer width and the maximum value for
the Minimal Arithmetic Noise Padding can only be derived from HLFHE
operations, the circuit constraints are determined automatically by
`zamacompiler` only if the option `--entry-dialect=hlfhe` was
specified.

For lower-level dialects, `zamacompiler` has been provided with the
options `--assume-max-eint-precision=...` and `--assume-max-manp=...`
that allow a user to specify the values for the maximum required
precision and maximum values for the Minimal Arithmetic Noise Padding.
2021-09-28 11:35:58 +02:00
Andi Drebes
2ed1720234 bugfix(compiler): Do not apply offset of -1 to 2-norm in getV0Parameter
A value of 1 is perfectly valid for the 2-norm of the constant vector
of a dot operation. Hence, the log2 of that value of 0 is also
perfectly valid. However, the offset of -1 in `getV0Parameter` applied
to that value yields an offset into a static lookup table of -1, which
triggers a segmentation fault.

This patch removes the offset of -1 for the log2 value of the 2-norm
in `getV0Parameter`.
2021-09-28 11:35:58 +02:00
Andi Drebes
5ce3fd5f43 refactor(compiler): Move ceilLog2 to separate header file 2021-09-28 11:35:58 +02:00
Andi Drebes
1200a46e49 feat(compiler): Add HLFHE pass selecting maximum MANP and encrypted integer width
This pass calculates the squared Minimal Arithmetic Noise Padding
(MANP) for each operation using the MANP pass and extracts the maximum
(non-squared) Minimal Arithmetic Noise Padding and the maximum
ecrypted integer width from.
2021-09-28 11:35:58 +02:00
Andi Drebes
6a6fae96f6 test(compiler): Add tests for the Minimal Arithmetic Noise Padding analysis pass
This adds tests for the Minimal Arithmetic Noise Padding analysis pass
that is invoked with --debug-manp, testing both elementary operations
and DAGs of computations on encrypted integers.
2021-09-28 11:35:58 +02:00
Andi Drebes
54661528a8 feat(compiler): Add action dump-hlfhe-manp
The new option --acion=dump-hlfhe-manp invokes the Minimal Arithmetic
Noise Padding Analysis pass based on the squared 2-norm metric from
`lib/Dialect/HLFHE/Analysis/MANP.cpp` and dumps the module afterwards
with an extra attribute `MANP` for each HLFHE operation.
2021-09-28 11:35:58 +02:00
Andi Drebes
ed762942c1 feat(compiler): Add pass for Minimal Arithmetic Noise Padding
This pass calculates the squared Minimal Arithmetic Noise Padding
(MANP) for each operation of a function and stores the result in an
integer attribute named "sqMANP". This metric is identical to the
squared 2-norm of the constant vector of an equivalent dot product
between a vector of encrypted integers resulting directly from an
encryption and a vector of plaintext constants.

The pass supports the following operations:

 - HLFHE.dot_eint_int
 - HLFHE.zero
 - HLFHE.add_eint_int
 - HLFHE.add_eint
 - HLFHE.sub_int_eint
 - HLFHE.mul_eint_int
 - HLFHE.apply_lookup_table

If any other operation is encountered, the pass conservatively
fails. The pass further makes the optimistic assumption that all
values passed to a function are either the direct result of an
encryption of a noise-refreshing operation.
2021-09-28 11:35:58 +02:00
Andi Drebes
30374ebb2c refactor(compiler): Introduce compilation pipeline with multiple entries / exits
This refactoring commit restructures the compilation pipeline of
`zamacompiler`, such that it is possible to enter and exit the
pipeline at different points, effectively defining the level of
abstraction at the input and the required level of abstraction for the
output.

The entry point is specified using the `--entry-dialect`
argument. Valid choices are:

  `--entry-dialect=hlfhe`:   Source contains HLFHE operations
  `--entry-dialect=midlfhe`: Source contains MidLFHE operations
  `--entry-dialect=lowlfhe`: Source contains LowLFHE operations
  `--entry-dialect=std`:     Source does not contain any FHE Operations
  `--entry-dialect=llvm`:    Source is in LLVM dialect

The exit point is defined by an action, specified using --action.

  `--action=roundtrip`:
     Parse the source file to in-memory representation and immediately
     dump as text without any processing

  `--action=dump-midlfhe`:
     Lower source to MidLFHE and dump result as text

  `--action=dump-lowlfhe`:
     Lower source to LowLFHE and dump result as text

  `--action=dump-std`:
     Lower source to only standard MLIR dialects (i.e., all FHE
     operations have already been lowered)

  `--action=dump-llvm-dialect`:
     Lower source to MLIR's LLVM dialect (i.e., the LLVM dialect, not
     LLVM IR)

  `--action=dump-llvm-ir`:
     Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
     actual LLVM IR)

  `--action=dump-optimized-llvm-ir`:
     Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
     actual LLVM IR), pass the result through the LLVM optimizer and
     print the result.

  `--action=dump-jit-invoke`:
     Execute the full lowering pipeline to optimized LLVM IR, JIT
     compile the result, invoke the function specified in
     `--jit-funcname` with the parameters from `--jit-args` and print
     the functions return value.
2021-09-28 11:35:58 +02:00
Andi Drebes
ddebedd1d6 refactor(compiler): Make function getV0Parameter return a pointer to a constant
The function `getV0Parameter()` returns a pointer to a static array,
which is not intended to be mutable. Reflect this in the return type
of the function.
2021-09-28 11:35:58 +02:00
Andi Drebes
6a76177a47 refactor(compiler): Move JIT functionality to separate source file 2021-09-28 11:35:58 +02:00
Andi Drebes
b9e2690823 refactor(compiler): Replace LOG_{VERBOSE,ERROR} macros with C++-style streams
Replace the macros `LOG_VERBOSE` and `LOG_ERROR` with C++-style
streams retrieved through `log_verbose()` and `log_error()`. This
aligns with the `MLIR` infrastructure and avoids pollution of the
global namespace through a common header file in subsequent
refactoring commits splitting the functionality of `src/main.cpp` into
multiple files.
2021-09-28 11:35:58 +02:00
Quentin Bourgerie
b4776a619d test(compiler): Reenable 7 bits test, we should change how the test are done to take care of the probability of exactness 2021-09-27 13:37:11 +02:00
Quentin Bourgerie
0f93fa1dcc fix(compiler/parameters): new parameters for v0 (p=0.99993 for 1-6 and 0.95 for 7) 2021-09-27 13:37:11 +02:00
Quentin Bourgerie
2cc72dd721 test(compiler): Fix end to end test we loss the 7 bits 2021-09-27 13:37:11 +02:00
Quentin Bourgerie
7454c30485 fix(compiler/parameters): Update parameters 2021-09-27 13:37:11 +02:00
Quentin Bourgerie
d9be22c1b1 fix(compiler/runtime): decode with integers 2021-09-27 13:37:11 +02:00
youben11
3feab09c88 test(LUT): random function 2021-09-27 13:37:11 +02:00
Quentin Bourgerie
3a254bcb87 fix(compiler): Integrate the security estimator to compute variances 2021-09-20 13:46:08 +02:00
youben11
967fda07a0 feat(execution): run on both int and tensor args 2021-09-10 10:00:04 +01:00
Quentin Bourgerie
c37ac41c1a test(compiler): LowLFHEUnpametrize 2021-09-09 20:57:18 +02:00
Quentin Bourgerie
8cc0af1220 fix(compiler): Add a pass to unparametrize LowLFHE ciphertext to remove unrelized_convesrion_cast for linalg bufferization 2021-09-09 20:35:28 +02:00
youben11
d0e71dd4f1 fix: use IndexType for size of LWE allocation 2021-09-09 20:35:28 +02:00
youben11
746d991af6 refactor: ins forward decl w generic ty @pass-init
Insert forward declarations with generic types at pass initialization.
More docs for all the pass for lowering LUT
2021-09-09 20:35:28 +02:00
youben11
d97512f507 tests: remove RunJit tests
It's better to have execution tests as unittest (that we have now),
better control of the execution than just MLIR.
2021-09-09 20:35:28 +02:00
youben11
3893ad39bb fix: divide and round to decode decrypted ct
Doing a right shift will ommit the bits that can be used to round to the
nearest integer
2021-09-09 20:35:28 +02:00
youben11
272a725e9a feat(lower_LUT): add precision attribute to encode LUT at runtime 2021-09-09 20:35:28 +02:00
youben11
c4dd639ea7 tests(LUT): use relative error 2021-09-09 20:35:28 +02:00
youben11
1e07733257 fix(lower-LUT): keep output size of KS during lowering
output size of keyswiting wasn't set properly. As this information must
come from the selected parameters, it should goes down from the MidLFHE
to the appropriate call to ciphertext allocation
2021-09-09 20:35:28 +02:00