github/concrete
1
1
Fork 0
mirror of https://github.com/zama-ai/concrete.git synced 2026-02-10 04:35:03 -05:00
Code Issues Packages Projects Releases Wiki Activity
349 Commits 185 Branches 425 Tags
e73291abdc094da5e882814d0530fcb405165cdf
Commit Graph

199 Commits

Author SHA1 Message Date
youben11
e73291abdc chore: rename compiler to concrete-compiler 
zamalang => concretelang
zamacompiler => concretecompiler
 2021-12-29 15:13:34 +01:00
youben11
971cb56182 chore: licensing 2021-12-29 15:13:34 +01:00
rudy
d8fee32cea feat(HLFHELinalg): add apply_mapped_table_lookup 
Resolves #182
 2021-12-27 15:34:33 +01:00
Antoniu Pop
cdca7ca6f7 feat(compiler): add Dataflow/RT dialect and code generation for dataflow auto parallelization. 2021-12-23 15:57:53 +00:00
Quentin Bourgerie
db683f4a0e feat(compiler/runtime): Introduce a RT dialect 2021-12-23 15:57:53 +00:00
Antoniu Pop
bc3d647453 fix(dfr): make HPX runtime environment persistent and deallocate temporaries. 2021-12-23 15:57:53 +00:00
Antoniu Pop
2cba686c66 fix(runtime): enable concurrent key usage by cloning it on get operation. 2021-12-23 15:57:53 +00:00
rudy
5ed2dcb3c1 fix(compiler): MANP segfault bug in case of missing MANP value 2021-12-20 10:40:58 +01:00
rudy
7a286ba9f9 fix(KeySetCache): handle concurrent generation 
Resolves #305
 2021-12-20 10:07:48 +01:00
Andi Drebes
f319ba37d2 feat(compiler): Integrate HLFHELinalg tiling passes into compilation pipeline 2021-12-17 16:09:59 +01:00
Andi Drebes
bc75831c86 feat(compiler): Add passes for tiling of HLFHELinalg.matmul_eint_int 
Add two passes related to the tiling of `HLFHELinalg.matmul_eint_int`
operations.

The `HLFHELinalgTilingMarker` pass takes a vector of tile sizes and
adds an integer array attribute "tile-sizes" to each instance of
`HLFHELinalg.matmul_eint_int`, e.g.,

  "HLFHELinalg.matmul_eint_int"(%arg0, %arg1) {"tile-sizes" = [2, 2, 2]} :
    (tensor<4x2x!HLFHE.eint<6>>, tensor<2x2xi7>) -> tensor<4x2x!HLFHE.eint<6>>

The `HLFHELinalgTiling` performs the actual tiling of each
`HLFHELinalg.matmul_eint_int` operation marked with a "tile-sizes"
attribute. The tiling preserves the level of abstraction of
HLFHELinalg and is implemented as a perfect loop nest of SCF for loops
with a `HLFHELinalg.matmul_eint_int` in the body.

For example,

  func @main(%arg0: tensor<4x2x!HLFHE.eint<6>>, %arg1: tensor<2x2xi7>)
    -> tensor<4x2x!HLFHE.eint<6>>
  {
    %0 = "HLFHELinalg.matmul_eint_int"(%arg0, %arg1) {"tile-sizes" = [2, 2, 2]} :
           (tensor<4x2x!HLFHE.eint<6>>, tensor<2x2xi7>) -> tensor<4x2x!HLFHE.eint<6>>
    return %0 : tensor<4x2x!HLFHE.eint<6>>
  }

becomes:

  func @main(%arg0: tensor<4x2x!HLFHE.eint<6>>, %arg1: tensor<2x2xi7>)
    -> tensor<4x2x!HLFHE.eint<6>>
  {
    %c2 = arith.constant 2 : index
    %c0 = arith.constant 0 : index
    %c4 = arith.constant 4 : index

    %0 = "HLFHELinalg.zero"() : () -> tensor<4x2x!HLFHE.eint<6>>
    %1 = scf.for %arg2 = %c0 to %c4 step %c2 iter_args(%arg3 = %0)
           -> (tensor<4x2x!HLFHE.eint<6>>) {
      %2 = scf.for %arg4 = %c0 to %c2 step %c2 iter_args(%arg5 = %arg3)
             -> (tensor<4x2x!HLFHE.eint<6>>) {
        %3 = scf.for %arg6 = %c0 to %c2 step %c2 iter_args(%arg7 = %arg5)
	       -> (tensor<4x2x!HLFHE.eint<6>>) {
          %4 = tensor.extract_slice %arg0[%arg2, %arg4] [2, 2] [1, 1] :
	         tensor<4x2x!HLFHE.eint<6>> to tensor<2x2x!HLFHE.eint<6>>
          %5 = tensor.extract_slice %arg1[%arg4, %arg6] [2, 2] [1, 1] :
	         tensor<2x2xi7> to tensor<2x2xi7>
          %6 = tensor.extract_slice %arg7[%arg2, %arg6] [2, 2] [1, 1] :
	         tensor<4x2x!HLFHE.eint<6>> to tensor<2x2x!HLFHE.eint<6>>

          %7 = "HLFHELinalg.matmul_eint_int"(%4, %5) :
	         (tensor<2x2x!HLFHE.eint<6>>, tensor<2x2xi7>)
		 -> tensor<2x2x!HLFHE.eint<6>>

          %8 = "HLFHELinalg.add_eint"(%6, %7) :
	         (tensor<2x2x!HLFHE.eint<6>>, tensor<2x2x!HLFHE.eint<6>>)
		 -> tensor<2x2x!HLFHE.eint<6>>

          %9 = tensor.insert_slice %8 into %arg7[%arg2, %arg6] [2, 2] [1, 1] :
	         tensor<2x2x!HLFHE.eint<6>> into tensor<4x2x!HLFHE.eint<6>>

          scf.yield %9 : tensor<4x2x!HLFHE.eint<6>>
        }
        scf.yield %3 : tensor<4x2x!HLFHE.eint<6>>
      }
      scf.yield %2 : tensor<4x2x!HLFHE.eint<6>>
    }
    return %1 : tensor<4x2x!HLFHE.eint<6>>
  }

Only full tiles are supported, i.e., the size of the dimensions of the
operands must be a multiple of the respective tile sizes.
 2021-12-17 16:09:59 +01:00
Andi Drebes
27ca5122bc enhance(compiler): Use named constant for the default pattern rewriting benefit 
This introduces a new header file `zamalang/Support/Constants.h` for
constants, currently only populated with a constant for the default
pattern rewriting benefit of 1.
 2021-12-17 15:28:21 +01:00
youben11
b812f6f7f2 feat(python): version python package from git tag 
automatic detection of version from git tag and update of the python
version file
 2021-12-16 09:15:25 +01:00
youben11
60b2cfd9b7 feat: support more dtype for scalars/tensors 
dtype supported now: uint8, uint16, uint32, uint64
 2021-12-13 17:40:28 +01:00
youben11
550318f67e feat: support different numpy object types as input 
- Scalar types: int, np.uint8 (to extend with other types later), and np.ndarray with shape == ()
- Tensor types: np.ndarray
 2021-12-13 17:40:28 +01:00
rudy
d34792c977 fix(compiler): remove 1 warning (usgined vs signed) 2021-12-13 16:43:35 +01:00
Andi Drebes
16d0502f56 fix(compiler): Initialize strides of memref parameters when JIT-invoking a function 
Upon invocation of a function with memref arguments, the strides for
all dimensions are currently set to 0. This causes dynamic offsets to
be calculated incorrectly in the function body.

This patch replaces the placeholder values with the actual strides for
each dimension and adds a test with parametric slice extraction from a
tensor that triggers dynamic indexing.
 2021-12-13 15:20:54 +01:00
Antoniu Pop
bb44124999 feat(dfr): add the DFR (DataFlow Runtime). 2021-12-08 18:35:29 +00:00
youben11
0a7ac76b10 fix: move unique_ptr<T> to be converted to llvm::Expected 
gcc 6.3 was complaining about not being able to wrap it in an
llvm::Expected
 2021-12-08 17:59:53 +01:00
rudy
74d52cb649 fix(clang): remove an error reported by clang 2021-12-08 11:31:23 +01:00
youben11
58f385b11b fix: move CompilationResult in return 
old toolchain (gcc 6.x) was complaining about not being able to convert
CompilationResult to Expected<CompilationResult>
 2021-12-07 16:14:44 +01:00
rudy
cc58608589 chore(Lambda): simplify, extract, enhance message for bit width rounding 
bit with rounding: 5bit element is widen to a standard 8bit word
 2021-12-07 15:29:05 +01:00
rudy
209463be22 chore(Lambda): simplify and extract null parameter detection 2021-12-07 15:29:05 +01:00
rudy
2c56a26c75 fix(Lambda): missing superfluous check in setArg 
[----------] Global test environment tear-down
[==========] 7 tests from 1 test suite ran. (1513 ms total)
[  PASSED  ] 7 tests.

  YOU HAVE 2 DISABLED TESTS
 2021-12-07 15:29:05 +01:00
rudy
47b4b667bb fix(Lambda): #253 fix the bug about Lambda parameter number verification 
[----------] Global test environment tear-down
[==========] 6 tests from 1 test suite ran. (1513 ms total)
[  PASSED  ] 6 tests.

  YOU HAVE 3 DISABLED TESTS

Compared to previous commit, a fatal test is disabled
 2021-12-07 15:29:05 +01:00
rudy
9a09afaa80 bug(Lambda): #253 reactivate the bug about Lambda parameter number verification 
[----------] Global test environment tear-down
[==========] 6 tests from 1 test suite ran. (1327 ms total)
[  PASSED  ] 5 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Lambda_check_param.scalar_tensor_to_tensor_good_number_param

 1 FAILED TEST
  YOU HAVE 3 DISABLED TESTS
 2021-12-07 15:29:05 +01:00
Mayeul@Zama
dad4390518 feat(compiler): add a key cache 2021-12-07 12:11:56 +01:00
Mayeul@Zama
f193fd71a2 fix(compiler): fix mixed-up (g)lwe dimension/size 
add dimension -> size conversion
rename k -> glweDimension, polynomialSize -> logPolynomialSize
remove (in/out)putLweSize from KeySwitchLweOp
remove GlweSizeType from LowLFHE
 2021-12-07 12:11:56 +01:00
Quentin Bourgerie
bb1add2a6f enhance(compiler): Remove the restriction of analysis on the HLFHELinalg.dot 
While the manp analysis wasn't handle tensor the dot operation restrict the operands to come from a block argument. Since the tensor are handled in the manp pass this restriction has no more meaning.
 2021-12-07 11:17:33 +01:00
Quentin Bourgerie
fb58dcc59d enhance(compiler/lowlfhe): Give the runtime context as function argument instead of a global variable (close #195) 2021-12-02 10:56:47 +01:00
Andi Drebes
6fb907295d feat(compiler): Add lowering for HLFHELinalg.zero to linalg.generate 
Add a rewrite pattern that transforms an instance of
`HLFHELinalg.zero` into an instance of `linalg.generate` with an
appropriate region yielding a zero value.

Example:

  %out = "HLFHELinalg.zero"() : () -> tensor<MxNx!HLFHE.eint<p>>

becomes:

  %0 = tensor.generate   {
    ^bb0(%arg2: index, %arg3: index):
       %zero = "HLFHE.zero"() : () -> !HLFHE.eint<p>
       tensor.yield %zero : !HLFHE.eint<p>
  } : tensor<MxNx!HLFHE.eint<p>>
 2021-12-01 11:24:27 +01:00
Andi Drebes
4883eebfa3 feat(compiler): Add HLFHELinalg.zero operation 
Add a new operation `HLFHELinalg.zero`, broadcasting an encrypted,
zero-valued integer into a tensor of encrypted integers with static
shape.

Example creating a one-dimensional tensor with five elements all
initialized to an encrypted zero:

  %tensor = "HLFHELinalg.zero"() : () -> tensor<5x!HLFHE.eint<4>>
 2021-12-01 11:24:27 +01:00
Andi Drebes
0b151724b8 enhance(compiler): Add support for tensor.generate in lowering pipeline 
Operations with regions currently need explicit patterns for type
conversion throughout lowering. This change adds the required patterns
for `tensor.generate` to the lowering passes, such that the operation
can be used starting from the lowering from HLFHE to MidLFHE.
 2021-12-01 11:24:24 +01:00
rudy
975ee86a5e feat(compiler): add --compile option 
Resolves #196
 2021-11-30 17:42:28 +01:00
Quentin Bourgerie
582328a35d cleanup(compiler/passes): Remove some warnings 
Remove unused variables in LowLFHEToConcreteCAPI
 2021-11-30 16:11:33 +01:00
rudy
ddca6840f1 fix(CompilerEngine): include mlir diagnostic in error message 
Resolves #288

example
before:
Failed to lower to LLVM dialect

after:
Failed to lower to LLVM dialect
test.mlir:3:10: error: unexpected error: 'linalg.copy' op expected indexing_map #1 to have 2 dim(s) to match the number of loops
    %0 = tensor.extract_slice %arg0[0, 0] [3, 1] [1, 1] : tensor<3x2x!HLFHE.eint<3>> to tensor<3x!HLFHE.eint<3>>
         ^
 2021-11-29 17:15:46 +01:00
Andi Drebes
8c06a5a1b2 enhance(compiler): Replace linalg type converter pattern with generic pattern 
Replace `LinalgGenericTypeConverterPattern`, specialized for
`linalg.generic` with a generic type converter pattern
`RegionOpTypeConverterPattern` that can be instantiated for any
operation that has one or more regions.

Further enhancements:
- Supports multiple regions
- Uses more idiomatic instantiations of `llvm::for_each` instead of
  manual iterations using for loops
 2021-11-26 17:05:21 +01:00
Quentin Bourgerie
fe6de8f0c5 style(compiler): Formatting 2021-11-24 09:19:25 +01:00
Quentin Bourgerie
ddbafd713d feat(compiler): Add the HLFHELinalg.matmul_int_eint operator 2021-11-24 09:19:25 +01:00
Andi Drebes
ffe32f5e90 fix(compiler): Suppress various warnings about expressions with different signedness 2021-11-19 16:23:15 +01:00
Quentin Bourgerie
100862e484 feat(compiler): Add the support of linalg.tensor_expand_shape and linalg.tensor_collapse_shape on encrypted tensors 2021-11-17 14:19:15 +01:00
youben11
c5e3d9add8 fix(compiler): reserve the right amount of memory 2021-11-15 11:25:44 +01:00
youben11
36413235c5 feat(compiler): lower HLFHELinalg.apply_multi_lut 
Support broadcasting
 2021-11-15 11:25:44 +01:00
youben11
ac7f1f5a6b feat(compiler): add HLFHELinalg.apply_multi_lut 
Apply a lookup table on the different elements of the encrypted tensors,
but using different lookup tables.

MANP support as well as tests
 2021-11-15 11:25:44 +01:00
Quentin Bourgerie
f2b5d3316f fix(compiler): don't try to broadcast when dims are equals with value 1 (7c8399) 
Removed due of miss conflict resolution
 2021-11-15 09:59:52 +01:00
Quentin Bourgerie
5df775a51b feat(compiler): MANP Analysis of HLFHELinalg.matmul (closes #178) 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
3fad9870a8 cleanup(compiler): Fix warning about comparison of expressions with different signedness in TensorOpsToLinalg 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
af0819c403 feat(compiler): Lower HLFHELinalg.matmul_eint_int to linalg.generic (close #177) 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
b598f0ffca refactor(compiler): Move the conversion from scf to std on the pipeline intead of embedded in a pass 2021-11-12 16:50:29 +01:00
Quentin Bourgerie
12b4771405 feat(compiler/linalg): Definition of HLFHELinalg.matmul_eint_int (closes #176) 2021-11-12 16:50:29 +01:00