In some cases, even small changes in zamacompiler lead to the
recompilation of the LLVM / MLIR sources, causing significant delay in
the CI. This change forces the use of ccache to speed up recompilation
after the project has been built at least once.
This adds a new variable `CCACHE` to `Makefile` that specifies whether
CMake is instructed upon configuration of the project to use the
default, plain C++ compiler or ccache as a wrapper.
By default, ccache is not used. To override the default value and
force the use of ccache, the variable `CCACHE` must be set to `ON`,
e.g., using `make CCACHE=ON`.
This adds a new overload for `JitCompilerEngine::Lambda::operator()`,
returning a result wrapped in a `std::unique_ptr<LambdaArgument>` with
meta information about the result. This allows for generic invocations
of JitCompilerEngine::Lambda::operator(), where the result type is
unknown before the invocation.
By default, `mlir::SourceMgrDiagnosticVerifierHandler` used by
`CompilerEngine::compile()` prints parse errors to
`llvm::errs()`. This makes it impossible for a caller of
`CompilerEngine::compile()` to process parse errors or to suppress the
emission of error messages to the standard error stream altogether.
This change captures parse errors in a string via a string-backed
output stream and forwards the error message in the `llvm::Error`
instance of the return value.
The code in `lib/CAPI/Support/CompilerEngine.cpp` invokes several
functions returning an `llvm::Expected<T>`. When those fail, the error
message retrieved from the error object the `llvm::Expected<T>`
instance is written to the standard error stream via
`mlir::zamalang::log_error()` and an exception with a more generic
error message is thrown.
This causes errors to show up on the standard error stream in tests
generating errors on purpose and checking them, e.g.:
```
tests/python/test_compiler_engine.py::test_compile_invalid[not
@main] Compilation failed: cannot find the function for generate
client parameters PASSED
```
This patch forwards the error message from an `llvm::Expected<T>`
instance in a runtime exception rather than writing it to the standard
error stream. Since exceptions are properly caught by the tests, no
errors show up during testing.
This commit contains several incremental improvements towards a clear
interface for lambdas:
- Unification of static and JIT compilation by using the static
compilation path of `CompilerEngine` within a new subclass
`JitCompilerEngine`.
- Clear ownership for compilation artefacts through
`CompilationContext`, making it impossible to destroy objects used
directly or indirectly before destruction of their users.
- Clear interface for lambdas generated by the compiler through
`JitCompilerEngine::Lambda` with a templated call operator,
encapsulating otherwise manual orchestration of `CompilerEngine`,
`JITLambda`, and `CompilerEngine::Argument`.
- Improved error handling through `llvm::Expected<T>` and proper
error checking following the conventions for `llvm::Expected<T>`
and `llvm::Error`.
Co-authored-by: youben11 <ayoub.benaissa@zama.ai>
As we store the funcname for the entrypoint for later use, a pointer
might point to some random memory when used as there is no special
management for that name at the higher levels.
LLVM and MLIR are compiled without runtime type information
(RTTI). Use the same restrictions for unit tests to avoid linker
errors related to typeinfo when building the test executable.
Composing error messages for `llvm::Error` is either done by using
`llvm::createStringError()` with an appropriate format string and
arguments or by writing to a `std::string`-backed
`llvm::raw_string_ostream` and passing the result to
`llvm::make_error<llvm::StringError>()` verbatim.
The new class `StreamStringError` encapsulates the latter solution
into a class with an appropriate stream operator and implicit cast
operators to `llvm::Error` and `llvm::Expected`.
Example usage:
llvm::Error foo(int i, size_t s, ...) {
...
if(...) {
return StreamStringError()
<< "Some error message with an integer: "
<< i << " and a size_t: " << s;
}
...
}
When determining the maximum MANP and precision, the `MaxMANPPass`
only takes into account results generated by an operation, but ignores
function parameters. However, encrypted function parameters are
assumed to have a MANP value of 1 and can have an arbitrary
precision. This patch takes into account function arguments by using
their default MANP values and the extracting their precision.
Add support for `tensor.extract` operations in the MANP pass. This
currently only supports extract operations on tensors of encrypted
integers, which are passed as function arguments, e.g.:
func @extract_ith(%t: tensor<10x!HLFHE.eint<5>>, %i: index) -> !HLFHE.eint<5>{
%c = tensor.extract %t[%i] : tensor<10x!HLFHE.eint<5>>
return %c : !HLFHE.eint<5>
}
Most of the targets in `Makefile` do not deped on files produced by
other targets and use target names solely for dependency
management. Make all such targets PHONY in order to avoid that they
are skipped accidentially when a file with the same name is present.
The Makefile target `python-bindings` invokes CMake with multiple
targets specified after the `--target` commandline option. However, as
per the CMake manpage, only one target may be specified at once.
This changes the single invocation of CMake to separate invocations
for each target.
Tested with CMake version 3.13.4.
The target `build` creates a build directory with the same name and
initializes through an invocation of CMake. Regardless of the success
or failure of the CMake invocation, all subsequent invocations of the
target do not invoke CMake anymore, as the target's prerequisites are
satisfied through the existence of the build directory created upon
the first invocation.
This patch changes the dependencies to the build directory with an
intermediate target that depends on a stamp file that is only created
when the first CMake invocation in the build directory succeeds.
- Docker image to build wheels for linux_x86_64 CPython 3.[8,9,10] with
GLIBC >= 2.24
- Specify which Python to use in Makefile
- Fix cmake build to handle when libpython isn't available (cmake>3.18)
