This commit contains several incremental improvements towards a clear
interface for lambdas:
- Unification of static and JIT compilation by using the static
compilation path of `CompilerEngine` within a new subclass
`JitCompilerEngine`.
- Clear ownership for compilation artefacts through
`CompilationContext`, making it impossible to destroy objects used
directly or indirectly before destruction of their users.
- Clear interface for lambdas generated by the compiler through
`JitCompilerEngine::Lambda` with a templated call operator,
encapsulating otherwise manual orchestration of `CompilerEngine`,
`JITLambda`, and `CompilerEngine::Argument`.
- Improved error handling through `llvm::Expected<T>` and proper
error checking following the conventions for `llvm::Expected<T>`
and `llvm::Error`.
Co-authored-by: youben11 <ayoub.benaissa@zama.ai>
Composing error messages for `llvm::Error` is either done by using
`llvm::createStringError()` with an appropriate format string and
arguments or by writing to a `std::string`-backed
`llvm::raw_string_ostream` and passing the result to
`llvm::make_error<llvm::StringError>()` verbatim.
The new class `StreamStringError` encapsulates the latter solution
into a class with an appropriate stream operator and implicit cast
operators to `llvm::Error` and `llvm::Expected`.
Example usage:
llvm::Error foo(int i, size_t s, ...) {
...
if(...) {
return StreamStringError()
<< "Some error message with an integer: "
<< i << " and a size_t: " << s;
}
...
}
When determining the maximum MANP and precision, the `MaxMANPPass`
only takes into account results generated by an operation, but ignores
function parameters. However, encrypted function parameters are
assumed to have a MANP value of 1 and can have an arbitrary
precision. This patch takes into account function arguments by using
their default MANP values and the extracting their precision.
Add support for `tensor.extract` operations in the MANP pass. This
currently only supports extract operations on tensors of encrypted
integers, which are passed as function arguments, e.g.:
func @extract_ith(%t: tensor<10x!HLFHE.eint<5>>, %i: index) -> !HLFHE.eint<5>{
%c = tensor.extract %t[%i] : tensor<10x!HLFHE.eint<5>>
return %c : !HLFHE.eint<5>
}
- Docker image to build wheels for linux_x86_64 CPython 3.[8,9,10] with
GLIBC >= 2.24
- Specify which Python to use in Makefile
- Fix cmake build to handle when libpython isn't available (cmake>3.18)
We decide to make this choice as they are issue to crate tensor of custom integer type in python.
+ we don't do the integer extension before convert to the concrete CAPI that requires i64
LLVM errors should be handled/consumed. Creating a new one and leaving
the previous one alive will crash the compiler. Whenever we don't want a
crash (e.g. logging the error is enough), but still wanna continue the
execution, we can just consume it.
This replaces the default FHE circuit constrains (maximum encrypted
integer width of 7 bits and a Minimal Arithmetic Noise Padding of 10
with the results of the `MaxMANP` pass, which determines these values
automatically from the input program.
Since the maximum encrypted integer width and the maximum value for
the Minimal Arithmetic Noise Padding can only be derived from HLFHE
operations, the circuit constraints are determined automatically by
`zamacompiler` only if the option `--entry-dialect=hlfhe` was
specified.
For lower-level dialects, `zamacompiler` has been provided with the
options `--assume-max-eint-precision=...` and `--assume-max-manp=...`
that allow a user to specify the values for the maximum required
precision and maximum values for the Minimal Arithmetic Noise Padding.
A value of 1 is perfectly valid for the 2-norm of the constant vector
of a dot operation. Hence, the log2 of that value of 0 is also
perfectly valid. However, the offset of -1 in `getV0Parameter` applied
to that value yields an offset into a static lookup table of -1, which
triggers a segmentation fault.
This patch removes the offset of -1 for the log2 value of the 2-norm
in `getV0Parameter`.
This pass calculates the squared Minimal Arithmetic Noise Padding
(MANP) for each operation using the MANP pass and extracts the maximum
(non-squared) Minimal Arithmetic Noise Padding and the maximum
ecrypted integer width from.
The new option --acion=dump-hlfhe-manp invokes the Minimal Arithmetic
Noise Padding Analysis pass based on the squared 2-norm metric from
`lib/Dialect/HLFHE/Analysis/MANP.cpp` and dumps the module afterwards
with an extra attribute `MANP` for each HLFHE operation.
This pass calculates the squared Minimal Arithmetic Noise Padding
(MANP) for each operation of a function and stores the result in an
integer attribute named "sqMANP". This metric is identical to the
squared 2-norm of the constant vector of an equivalent dot product
between a vector of encrypted integers resulting directly from an
encryption and a vector of plaintext constants.
The pass supports the following operations:
- HLFHE.dot_eint_int
- HLFHE.zero
- HLFHE.add_eint_int
- HLFHE.add_eint
- HLFHE.sub_int_eint
- HLFHE.mul_eint_int
- HLFHE.apply_lookup_table
If any other operation is encountered, the pass conservatively
fails. The pass further makes the optimistic assumption that all
values passed to a function are either the direct result of an
encryption of a noise-refreshing operation.
This refactoring commit restructures the compilation pipeline of
`zamacompiler`, such that it is possible to enter and exit the
pipeline at different points, effectively defining the level of
abstraction at the input and the required level of abstraction for the
output.
The entry point is specified using the `--entry-dialect`
argument. Valid choices are:
`--entry-dialect=hlfhe`: Source contains HLFHE operations
`--entry-dialect=midlfhe`: Source contains MidLFHE operations
`--entry-dialect=lowlfhe`: Source contains LowLFHE operations
`--entry-dialect=std`: Source does not contain any FHE Operations
`--entry-dialect=llvm`: Source is in LLVM dialect
The exit point is defined by an action, specified using --action.
`--action=roundtrip`:
Parse the source file to in-memory representation and immediately
dump as text without any processing
`--action=dump-midlfhe`:
Lower source to MidLFHE and dump result as text
`--action=dump-lowlfhe`:
Lower source to LowLFHE and dump result as text
`--action=dump-std`:
Lower source to only standard MLIR dialects (i.e., all FHE
operations have already been lowered)
`--action=dump-llvm-dialect`:
Lower source to MLIR's LLVM dialect (i.e., the LLVM dialect, not
LLVM IR)
`--action=dump-llvm-ir`:
Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
actual LLVM IR)
`--action=dump-optimized-llvm-ir`:
Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
actual LLVM IR), pass the result through the LLVM optimizer and
print the result.
`--action=dump-jit-invoke`:
Execute the full lowering pipeline to optimized LLVM IR, JIT
compile the result, invoke the function specified in
`--jit-funcname` with the parameters from `--jit-args` and print
the functions return value.
The function `getV0Parameter()` returns a pointer to a static array,
which is not intended to be mutable. Reflect this in the return type
of the function.
Replace the macros `LOG_VERBOSE` and `LOG_ERROR` with C++-style
streams retrieved through `log_verbose()` and `log_error()`. This
aligns with the `MLIR` infrastructure and avoids pollution of the
global namespace through a common header file in subsequent
refactoring commits splitting the functionality of `src/main.cpp` into
multiple files.
output size of keyswiting wasn't set properly. As this information must
come from the selected parameters, it should goes down from the MidLFHE
to the appropriate call to ciphertext allocation
We currently use LD_PRELOAD with the python extension to make the JIT
execution find the appropriate symbols, however, not linking with some
libraries caused other tools such as make to complain of not finding
symbols from libpthread and others
The polynomialSize is currently holding its log2 instead of the actual
value. This should be fixed later, but in the meantime, we need to
compute it from log2
