Commit Graph

669 Commits

Author SHA1 Message Date
youben11
0987fade80 fix: parameterize DATAFLOW_EXECUTION in python wheel 2022-10-07 16:59:34 +01:00
David Testé
89d7f065ae ci(compiler): schedule end-to-end benchmarks
Run end-to-end benchmarks on a weekly basis. Results are parsed and
then sent to Slab CI bot to be later plotted into Grafana.
2022-10-07 14:45:56 +02:00
Arthur Meyre
9c744f7edd chore: fix repo setup instructions, update .gitignore 2022-10-07 09:49:00 +02:00
Quentin Bourgerie
090e0dbbca fix(compiler): Fix maps with symbolic offsets in memref casts for wrapper functions
This reverts commit d9363ffb27.
2022-10-07 09:16:19 +02:00
Quentin Bourgerie
cf9a36c197 feat(compiler/runtime): Support the pbs for crt encoding (enable apply_lookup_table up to 16bits) 2022-10-07 09:16:19 +02:00
Ayoub Benaissa
6a5e65631c chore: update concrete-ffi version
no major breaking changes, just enhancements
check https://github.com/zama-ai/concrete-core/releases
2022-10-06 12:48:17 +01:00
Mayeul@Zama
819da92e3a dep(optimizer): use options api from optimizer 2022-10-05 14:20:51 +02:00
Andi Drebes
e35c5bddcf fix(test): Add missing dependency from run-mlbench to generate-mlbench
The target `run-mlbench` indirectly depends on the contents of
`tests.ml/bench.zip` which are extracted by `generate-mlbench`. If
`generate-mlbench` has not been built pefore, `run-mlbench`
fails. This adds a the missing dependency from `run-mlbench` to
`generate-mlbench`.
2022-10-04 16:40:27 +02:00
Andi Drebes
55f843cd07 test(compiler): Fix comparison of ints with different signedness in unit test lib 2022-10-04 14:40:40 +02:00
Andi Drebes
85442f425d test(compiler): Move clear text tests to YAML test specification
This moves all tests from
`tests/end_to_end_tests/end_to_end_jit_clear_tensor.cc` to the test
specification in YAML format in
`tests/end_to_end_fixture/end_to_end_clear_tensor.yaml`. Parametric
tests and tests invoking lambdas in loops have been fully unrolled.
2022-10-04 14:40:40 +02:00
Andi Drebes
a7051c2c9c enhance(client/server): Add support for scalar results
This patch adds support for scalar results to the client/server
protocol and tests. In addition to `TensorData`, a new type
`ScalarData` is added. Previous representations of scalar values using
one-dimensional `TensorData` instances have been replaced with proper
instantiations of `ScalarData`.

The generic use of `TensorData` for scalar and tensor values has been
replaced with uses of a new variant `ScalarOrTensorData`, which can
either hold an instance of `TensorData` or `ScalarData`.
2022-10-04 14:40:40 +02:00
Andi Drebes
710dd7a88c enhance(compiler): Add function returning LambdaArgument type as string
This adds a new function `getLambdaArgumentTypeAsString(const
LambdaArgument&)` returning the name of a lambda argument type as a
string, e.g., `"uint8_t"` for an `IntLambdaArgument<uint8_t>` or
`"tensor<uint8_t>"` for a
`TensorLambdaArgument<IntLambdaArgument<uint8_t>>`.

Note that, due to the static inheritance scheme for Lambda Arguments
and explicit instantiation, this is only implemented for the common
backing integer types `uint8_t`, `int8_t`, `uint16_t`, `int16_t`,
`uint32_t`, `int32_t`, `uint64_t`, and `int64_t`.
2022-10-04 14:40:40 +02:00
Andi Drebes
7a3cf64171 enhance(compiler): Add constructor for TensorLambdaArgument moving vector values 2022-10-04 14:40:40 +02:00
Andi Drebes
9e24eccc9d enhance(compiler): Add operators == and != for Integer / Tensor Lambda Arguments 2022-10-04 14:40:40 +02:00
Andi Drebes
ade86d935b enhance(jit): Add support for clear text results in ServerLambda
This removes the restriction of `ServerLambda` that the source program
must return encrypted values.
2022-10-04 14:40:40 +02:00
Andi Drebes
a5289a3145 enhance(tests): Add option use_default_fhe_constraints to YAML test descriptions
This adds a new optional boolean parameter
`use_default_fhe_constraints` to the YAML description of tests that
causes the test backend to use the default FHE constraints if set to
true. This is necessary for clear text tests, since automatic
determination of FHE parameters by the `CompilerEngine` fails for
programs not using encrypted integers and causes the program to abort.
2022-10-04 14:40:40 +02:00
Andi Drebes
847e8607f3 enhance(compiler): Add support for debug builds in Makefile
This adds a new variable `BUILD_TYPE` to the Makefile, controlling
whether the build should be a debug or a release build (values `Debug`
and `Release`, respectively). The default mode is `Release`. Depending
on the build type, the build directory is set to `build-Debug` or
`build-Release`. This enables debug and release builds to co-exist and
to switch easily between the two.
2022-10-04 14:40:40 +02:00
Andi Drebes
7427a4506d test(compiler): Abort test if a stream for a YAML file is in a bad state
The code in `loadEndToEndDesc` attempts to load the contents of a YAML
file without ensuring that the stream from which the file is read was
set up correctly. As a result, streams in a bad state, e.g., if the
file to be opened does not exist, simply result in an empty string for
the file contents. The test driver cannot distinguish this result from
a valid YAML file that is simply empty.

This patch causes `loadEndToEndDesc` to abort execution whenever a
file input stream could not be set up correctly.
2022-10-04 14:40:40 +02:00
Andi Drebes
8255d3e190 fix(compiler): Add support for clear result tensors with element width != 64 bits
Returning tensors with elements whose width is not equal to 64 results
in garbled data. This commit extends the `TensorData` class used to
represent tensors in JIT compilation with support for signed /
unsigned elements of 8/16/32 and 64 bits, such that all clear text
tensors with up to 64 bits can be represented accurately.
2022-10-04 14:40:40 +02:00
Mayeul@Zama
f1833f06f2 feat(compiler): clarify input encryption noise variance 2022-10-03 14:00:16 +02:00
rudy
cb2c9ef6bf feat: accept no evaluation keys 2022-09-26 14:43:25 +02:00
rudy
08ed2fc49b fix: when both sol.p_error and sol.global_p_error are near 0 2022-09-21 14:23:16 +02:00
rudy
e1fb417c54 fix(optimizer): no ceiling for MANP value given to the optimizer 2022-09-19 17:59:22 +02:00
youben11
628df8849c refactor: unify Makefile target naming style 2022-09-19 13:24:20 +01:00
youben11
8f05f9ea72 ci: change package release process
it should be now faster using the same compilation cache as the test
build.
2022-09-19 13:24:20 +01:00
Antoniu Pop
55b719f100 feat(tests): add instantiation of end-to-end tests for each option sets. 2022-09-19 13:02:20 +01:00
Antoniu Pop
4fbb05e18c feat(compiler): add an asynchronous interface for bootstrap and keyswitch using std::promise/future. 2022-09-19 13:02:20 +01:00
Antoniu Pop
09184077d9 benchmarking: add YAML generator and ML benchmarks (see issue https://github.com/zama-ai/concrete-ml-internal/issues/1543). 2022-09-15 14:38:56 +01:00
Antoniu Pop
2cf80e76eb feat(compiler): move the lowering of dataflow tasks to RT dialect before bufferization. 2022-09-15 11:55:37 +01:00
Andi Drebes
26901a32da test(compiler): Test the bufferization of BConcrete with memrefs with non-zero offsets 2022-09-14 16:47:41 +02:00
Andi Drebes
d9363ffb27 fix(compiler): Use maps with symbolic offsets in memref casts for wrapper functions
The bufferization of the BConcrete dialect emits calls to Concrete
wrapper functions and casts all memrefs to ranked memrefs with dynamic
strides and an implicit identity layout map. The implicit identity map
does not allow for casts of memrefs with non-zero offsets, e.g.,
resulting from folding of memrefs related to intermediate results
passed as operands to the operation implemented by a wrapper.

Casting to memrefs symbolic offsets in the layout map (e.g.,
`[d0, d1, ...](s0, s1, ...) -> (d0 + s0, d1 + s1, ...)`) allows
for more flexibility, in particular this adds support for memrefs
with non-zero, constant offsets returned by operations generating
intermediate results.
2022-09-14 16:47:41 +02:00
Quentin Bourgerie
8c605a6267 test(keyset): Set a proper variance for encryp_decrypt test 2022-09-14 13:50:26 +02:00
Umut
465b4b95bb chore: remove seeder selection messages 2022-09-14 12:37:42 +02:00
Quentin Bourgerie
dbfde466bc feat(python): Add compilation feedback to the python bindings 2022-09-14 10:03:25 +02:00
Quentin Bourgerie
f4673e8276 feat(compiler): First draft or compilation feedback 2022-09-14 10:03:25 +02:00
youben11
0cd9a7984d feat: support dataflow execution in python packages
use latest image manylinux_2_28_x86_64 using libc 2.28
2022-09-13 15:18:15 +01:00
rudy
32437af31b feat(test): check test error rate for fast cases 2022-09-12 17:29:03 +02:00
rudy
48bf6e2696 feat(optimizer): report or warn using global p-error 2022-09-12 17:22:38 +02:00
Umut
41c9f86803 feat: create encrypted signed integer type 2022-09-09 17:38:21 +03:00
Antoniu Pop
39e7313348 feat(tests): add function for increasing the stack limit and use for benchmark overflow. 2022-09-08 11:35:56 +01:00
rudy
e5d9cb1af3 fix: display the reason when the linker can't be called and fix the root cause 2022-09-08 12:09:06 +02:00
rudy
deef4486ba fix: negative value have full complement 2 noise
Revert ae9a04cd56
2022-09-06 15:27:20 +02:00
youben11
4b584ca64e ci: create new keysetcache
last commit updated which keys we serialize
2022-09-06 07:18:34 +01:00
youben11
661d33c2b6 feat: keep std bsk and conv to fourier when needed 2022-09-06 07:18:34 +01:00
youben11
942b41d07c fix: destroy glwe secret key after use 2022-09-06 07:18:34 +01:00
Quentin Bourgerie
b0743a9924 Revert "fix(optimizer): Temporary fallback to the v0 strategy while the dag one is not fixed"
This reverts commit 98a799f807.
2022-09-01 14:13:38 +02:00
Quentin Bourgerie
30df69366d enhance(python-bindings): Expose the option to set or not the v0 strategy of the optimizer. 2022-09-01 10:37:00 +02:00
Quentin Bourgerie
98a799f807 fix(optimizer): Temporary fallback to the v0 strategy while the dag one is not fixed 2022-09-01 10:37:00 +02:00
Mayeul@Zama
eb4b0753dc enhance(runtime): Move to the official concrete-core-ffi release 0.2.0-rc2
Co-authored-by: Mayeul@Zama <mayeul.debellabre@zama.ai>
Co-authored-by: Quentin Bourgerie <bourgerie.quentin@gmail.com>
2022-08-30 18:08:47 +02:00
Ayoub Benaissa
393d1617eb fix: add linking options for MacOS
Starting from Mac 11 (Big Sur), it appears we need to add -L
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -lSystem for the
sharedlib to link properly.i
2022-08-26 10:45:15 +01:00