This commit contains several incremental improvements towards a clear
interface for lambdas:
- Unification of static and JIT compilation by using the static
compilation path of `CompilerEngine` within a new subclass
`JitCompilerEngine`.
- Clear ownership for compilation artefacts through
`CompilationContext`, making it impossible to destroy objects used
directly or indirectly before destruction of their users.
- Clear interface for lambdas generated by the compiler through
`JitCompilerEngine::Lambda` with a templated call operator,
encapsulating otherwise manual orchestration of `CompilerEngine`,
`JITLambda`, and `CompilerEngine::Argument`.
- Improved error handling through `llvm::Expected<T>` and proper
error checking following the conventions for `llvm::Expected<T>`
and `llvm::Error`.
Co-authored-by: youben11 <ayoub.benaissa@zama.ai>
This replaces the default FHE circuit constrains (maximum encrypted
integer width of 7 bits and a Minimal Arithmetic Noise Padding of 10
with the results of the `MaxMANP` pass, which determines these values
automatically from the input program.
Since the maximum encrypted integer width and the maximum value for
the Minimal Arithmetic Noise Padding can only be derived from HLFHE
operations, the circuit constraints are determined automatically by
`zamacompiler` only if the option `--entry-dialect=hlfhe` was
specified.
For lower-level dialects, `zamacompiler` has been provided with the
options `--assume-max-eint-precision=...` and `--assume-max-manp=...`
that allow a user to specify the values for the maximum required
precision and maximum values for the Minimal Arithmetic Noise Padding.
The new option --acion=dump-hlfhe-manp invokes the Minimal Arithmetic
Noise Padding Analysis pass based on the squared 2-norm metric from
`lib/Dialect/HLFHE/Analysis/MANP.cpp` and dumps the module afterwards
with an extra attribute `MANP` for each HLFHE operation.
This pass calculates the squared Minimal Arithmetic Noise Padding
(MANP) for each operation of a function and stores the result in an
integer attribute named "sqMANP". This metric is identical to the
squared 2-norm of the constant vector of an equivalent dot product
between a vector of encrypted integers resulting directly from an
encryption and a vector of plaintext constants.
The pass supports the following operations:
- HLFHE.dot_eint_int
- HLFHE.zero
- HLFHE.add_eint_int
- HLFHE.add_eint
- HLFHE.sub_int_eint
- HLFHE.mul_eint_int
- HLFHE.apply_lookup_table
If any other operation is encountered, the pass conservatively
fails. The pass further makes the optimistic assumption that all
values passed to a function are either the direct result of an
encryption of a noise-refreshing operation.
This refactoring commit restructures the compilation pipeline of
`zamacompiler`, such that it is possible to enter and exit the
pipeline at different points, effectively defining the level of
abstraction at the input and the required level of abstraction for the
output.
The entry point is specified using the `--entry-dialect`
argument. Valid choices are:
`--entry-dialect=hlfhe`: Source contains HLFHE operations
`--entry-dialect=midlfhe`: Source contains MidLFHE operations
`--entry-dialect=lowlfhe`: Source contains LowLFHE operations
`--entry-dialect=std`: Source does not contain any FHE Operations
`--entry-dialect=llvm`: Source is in LLVM dialect
The exit point is defined by an action, specified using --action.
`--action=roundtrip`:
Parse the source file to in-memory representation and immediately
dump as text without any processing
`--action=dump-midlfhe`:
Lower source to MidLFHE and dump result as text
`--action=dump-lowlfhe`:
Lower source to LowLFHE and dump result as text
`--action=dump-std`:
Lower source to only standard MLIR dialects (i.e., all FHE
operations have already been lowered)
`--action=dump-llvm-dialect`:
Lower source to MLIR's LLVM dialect (i.e., the LLVM dialect, not
LLVM IR)
`--action=dump-llvm-ir`:
Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
actual LLVM IR)
`--action=dump-optimized-llvm-ir`:
Lower source to plain LLVM IR (i.e., not the LLVM dialect, but
actual LLVM IR), pass the result through the LLVM optimizer and
print the result.
`--action=dump-jit-invoke`:
Execute the full lowering pipeline to optimized LLVM IR, JIT
compile the result, invoke the function specified in
`--jit-funcname` with the parameters from `--jit-args` and print
the functions return value.
Replace the macros `LOG_VERBOSE` and `LOG_ERROR` with C++-style
streams retrieved through `log_verbose()` and `log_error()`. This
aligns with the `MLIR` infrastructure and avoids pollution of the
global namespace through a common header file in subsequent
refactoring commits splitting the functionality of `src/main.cpp` into
multiple files.
This changes the semantics of `HLFHE.dot_eint_int` from memref-based
reference semantics to tensor-based value semantics. The former:
"HLFHE.dot_eint_int"(%arg0, %arg1, %arg2) :
(memref<Nx!HLFHE.eint<0>>, memref<Nxi32>, memref<!HLFHE.eint<0>>) -> ()
becomes:
"HLFHE.dot_eint_int"(%arg0, %arg1) :
(tensor<Nx!HLFHE.eint<0>>, tensor<Nxi32>) -> !HLFHE.eint<0>
As a side effect, data-flow analyses become much easier. With the
previous memref type of the plaintext argument it is difficult to
check whether the plaintext values are statically defined constants or
originate from a memory region changed at execution time (e.g., for
analyses evaluating the impact on noise). Changing the plaintext type
from `memref` to `vector` makes such analyses significantly easier.
* feat(compiler): low level fhe dialect
* feat(compiler): using generated printer/parser in LowLFHE
* feat(compiler): new types and ops for LowLFHE
* tests(compiler): LowLFHE types and ops
* feat(compiler): fill ops
* cleanup
* summary + description
* tests(compiler): use new CLI args
* formatting
This adds a new command line option
`--convert-hlfhe-tensor-ops-to-linalg` that invokes a conversion pass
replacing any HLFHE tensor operation with an appropriate instance of
`linalg.generic`.
The option `--verify-diagnostics` allows the user to interleave
invalid source code with diagnostic messages that should be produced
during parsing and checks if these messages are actually produced. The
expected diagnostic messages are specified using `// expected-error`
markers.
The new option `--split-input-file` allows the user to add `// -----`
markers to a source file, causing the file to be split into multiple
chunks that are treated as if they were specified in different files.
Use a more idiomatic approach for I/O using `llvm::SourceManager` to
manage input files and `mlir::openOutputFile` to write to the
output. This commit prepares management of diagnostics in subsequent
changes.
LLVM already provides a command line parser, covering the use of tclap
in the sources. This patch uses the LLVM command line parser instead
of tclap and thus reduces the number of external dependencies.
