github/concrete
1
1
Fork 0
mirror of https://github.com/zama-ai/concrete.git synced 2026-02-08 19:44:57 -05:00
Code Issues Packages Projects Releases Wiki Activity

docs: write some missing sections and update some outdated ones

Browse Source
This commit is contained in:
Umut
2021-12-08 11:05:57 +03:00
parent 4a7d939bd0
commit b55cf2958a
17 changed files with 365 additions and 377 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/_static/tutorials/artifacts/auto/1.initial.graph.png vendored
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 7.6 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

BIN
docs/_static/tutorials/artifacts/auto/2.final.graph.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.9 KiB

BIN
docs/_static/tutorials/artifacts/manual/1.initial.graph.png vendored
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 33 KiB

BIN
docs/_static/tutorials/artifacts/manual/2.after-float-fuse-0.graph.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

BIN
docs/_static/tutorials/artifacts/manual/2.final.graph.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 23 KiB

BIN
docs/_static/tutorials/artifacts/manual/3.final.graph.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

25
docs/dev/explanation/COMPILATION.md
View File

@@ -1,9 +1,3 @@
```{warning}
FIXME(umut): update with the new API
```
# Compilation Pipeline In Depth
## What is **concretefhe**?
@@ -23,15 +17,12 @@ import concrete.numpy as hnp
def f(x, y):
return (2 * x) + y
# Define the inputs of homomorphized function
x = hnp.EncryptedScalar(hnp.UnsignedInteger(2))
y = hnp.EncryptedScalar(hnp.UnsignedInteger(1))
# Create a NumPy FHE Compiler
compiler = hnp.NPFHECompiler(f, {"x": "encrypted", "y": "encrypted"})
# Compile the function to its homomorphic equivalent
circuit = hnp.compile_numpy_function(
f, {"x": x, "y": y},
[(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1), (3, 0), (3, 1)],
)
# Compile an FHE Circuit using an inputset
compiler.eval_on_inputset([(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1), (3, 0), (3, 1)])
circuit = compiler.get_compiled_fhe_circuit()
# Make homomorphic inference
circuit.run(1, 0)
@@ -223,7 +214,7 @@ def f(x):
### Parameters
```
x = EncryptedScalar(UnsignedInteger(2))
x = "encrypted"
```
#### Corresponding operation graph
@@ -272,8 +263,8 @@ def f(x, y):
### Parameters
```
x = EncryptedScalar(UnsignedInteger(3))
y = EncryptedScalar(UnsignedInteger(1))
x = "encrypted"
y = "encrypted"
```
#### Corresponding operation graph

21
docs/user/benchmarks.md
View File

@@ -1,22 +1,21 @@
# Benchmarks
```{warning}
FIXME(Umut): update a bit
```
In order to track our progress over time, we have set a [public benchmark](https://progress.zama.ai) containing:
- a list of functions that we want to compile
To track our progress over time, we have created a [progress tracker](https://progress.zama.ai) that have
- list of targets that we want to compile
- status on the compilation of these functions
- compilation time
- evaluation time on different hardware's
- compilation and evaluation times on different hardware
- accuracy of the functions for which it makes sense
- loss of the functions for which it makes sense
Remark that we are not limited to these, and we'll certainly add more information later, as key generation time, encryption and decryption time, and more evaluation time once the explicit inference API is available.
Note that we are not limited to these, and we'll certainly add more information (e.g., key generation time, encryption time, inference time, decryption time, etc.) once the explicit inference API is available.
The benchmark can be used by competitive frameworks or technologies, in order to compare fairly with the **Concrete Framework**. Notably, one can see:
```{warning}
FIXME(all): update the sentence above when the encrypt, decrypt, run_inference, keygen API's are available
```
Our public benchmarks can be used by competing frameworks or technologies for comparison with **Concrete Framework**. Notably, one can see:
- if the same functions can be compiled
- what are discrepancies in the exactness of the evaluations
- how do evaluation times compare
If one wants to see more functions in the benchmark or if there is more information you would like the benchmark to track, don't hesitate to drop an email to <hello@zama.ai>.
If you want to see more functions in the progress tracker or if there is another metric you would like to track, don't hesitate to drop an email to <hello@zama.ai>.

1
docs/user/howto/COMPILING_AND_EXECUTING.md
View File

@@ -85,7 +85,6 @@ FIXME(benoit): explain the API to encrypt, run_inference, decrypt, keygen etc wh
## Further reading
- [Arithmetic Operations Tutorial](../tutorial/ARITHMETIC_OPERATIONS.md)
- [Working With Floating Points Tutorial](../tutorial/WORKING_WITH_FLOATING_POINTS.md)
- [Table Lookup Tutorial](../tutorial/TABLE_LOOKUP.md)
- [Compiling a torch model](../tutorial/COMPILING_TORCH_MODEL.md)

41
docs/user/howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md
View File

@@ -43,31 +43,34 @@ Hopefully, it is just a misunderstanding or a small mistake on your side, that o
When things are more complicated, or if you want to have a look by yourself, you may want to have a look to the compilation reports, which are called artifacts. This is as simple as described in [here](../tutorial/COMPILATION_ARTIFACTS.md)
This function will create a directory, containing notably:
```{warning}
FIXME(Umut): check it is still accurate
```
- bounds.txt: a file describing the expected ranges of data in the different steps of the computation
- cryptographic_parameters.txt: a file describing the different keys
- ir_nodes.txt: a file describing the different nodes in the intermediate representation (IR)
- optimizations_applied.txt: a file describing the different optimizations which were applied
- target_nodes.txt: a file describing the different nodes in the VM graph
The artifact system will create a directory, containing:
- **environment.txt:** information about your system
- **requirements.txt:** information about your python dependencies
- **function.txt:** source code of the function you are compiling
- **parameters.txt:** parameters you specified for compilation
- **1.initial.graph.txt:** textual representation of the initial computation graph right after tracing
- **1.initial.graph.png:** visual representation of the initial computation graph right after tracing
- ...
- **X.description.graph.txt:** textual representation of the Xth computation graph after topological transforms
- **X.description.graph.png:** visual representation of the Xth computation graph after topological transforms
- ...
- **N.final.graph.txt:** textual representation of the final computation graph right before MLIR conversion
- **N.final.graph.png:** visual representation of the final computation graph right before MLIR conversion
- **bounds.txt:** ranges of data in the different steps of the computation for the final graph that is being compiled
- **mlir.txt**: resulting MLIR code that is sent to the compiler (if compilation succeeded)
- **traceback.txt**: information about the error you encountered (if compilation failed)
Attaching the artifact with your issue or Slack message may help people to have a look at the core of the problem.
The more precise your bug, the more likely we can reproduce and fix
```{warning}
FIXME(Umut): is it still needed or do we already have some of those information in artifacts?
```
In order to simplify our work and let us reproduce your bug easily, any information is useful. Notably, in addition to the python script, some information like:
- the OS version
- the python version
- the python packages you use
- the reproducibility rate you see on your side
To simplify our work and let us reproduce your bug easily, we need all the information we can get. So, in addition to your python script, the following information would be very useful.
- compilation artifacts
- reproducibility rate you see on your side
- any insight you might have on the bug
- any workaround you have been able to find
may be useful to us. Don't remember, **Concrete** is a project where we are open to contribution, more information at Contributing (TODO: add a link).
Remember, **Concrete Framework** is a project where we are open to contributions, more information at [Contributing](../../dev/howto/CONTRIBUTING.md).
## Submitting an issue

1
docs/user/howto/PRINTING_AND_DRAWING.md
View File

@@ -1,5 +1,6 @@
```{warning}
FIXME(all): should we update to the new API or have it for both?
FIXME(all): we should add an example drawing and printing with fusing (so subgraphs)
```
# Printing and Drawing

52
docs/user/howto/REDUCE_NEEDED_PRECISION.md
View File

@@ -1,8 +1,54 @@
```{warning}
FIXME(Umut): put somewhere an example of an error dump if the user asks for too much precision and explain
# Having a Function Which Requires Less Precision
With our current technology, we cannot represent integers with more than 7 bits.
We are actively working on supporting larger integers, so it should get better in the future.
## What happens when you have larger values?
You get a compilation error. Here is an example:
<!--python-test:skip-->
```python
import concrete.numpy as hnp
def f(x):
return 42 * x
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"})
compiler.eval_on_inputset(range(2 ** 3))
compiler.get_compiled_fhe_circuit()
```
# Having a Function Which Requires Less Precision
results in
```
Traceback (most recent call last):
File "/home/default/Documents/Projects/Zama/hdk/dist/demo.py", line 9, in <module>
circuit = compiler.get_compiled_fhe_circuit()
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/np_fhe_compiler.py", line 274, in get_compiled_fhe_circuit
return compile_op_graph_to_fhe_circuit(
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 676, in compile_op_graph_to_fhe_circuit
result = run_compilation_function_with_error_management(
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 141, in run_compilation_function_with_error_management
return compilation_function()
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 674, in compilation_function
return _compile_op_graph_to_fhe_circuit_internal(op_graph, show_mlir, compilation_artifacts)
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 626, in _compile_op_graph_to_fhe_circuit_internal
prepare_op_graph_for_mlir(op_graph)
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 603, in prepare_op_graph_for_mlir
update_bit_width_for_mlir(op_graph)
File "/home/default/Documents/Projects/Zama/hdk/concrete/common/mlir/utils.py", line 204, in update_bit_width_for_mlir
raise RuntimeError(
RuntimeError: max_bit_width of some nodes is too high for the current version of the compiler (maximum must be 7) which is not compatible with:
%0 = x # EncryptedScalar<uint3>
%1 = 42 # ClearScalar<uint6>
%2 = mul(%0, %1) # EncryptedScalar<uint9>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 9 bits is not supported for the time being
return %2
```
when you try to run.
## Why can some computation work with less precision?

1
docs/user/index.rst
View File

@@ -15,7 +15,6 @@ Getting Started
:caption: Tutorial
tutorial/COMPILING_TORCH_MODEL.md
tutorial/ARITHMETIC_OPERATIONS.md
tutorial/TABLE_LOOKUP.md
tutorial/WORKING_WITH_FLOATING_POINTS.md
tutorial/NUMPY_SUPPORT.md

245
docs/user/tutorial/ARITHMETIC_OPERATIONS.md
View File

@@ -1,245 +0,0 @@
```{warning}
FIXME(Umut): update a bit, with the new API
FIXME(Umut/Arthur): update a bit to explain things with the tensors and new operations. At the same time, I think we can exhaustively give examples of every supported functions, since we start to have a lot, so maybe, we would just explain a bit?
FIXME(all): actually, I am not even sure we should keep this .md, it can't be exhaustive enough, and looks pretty trivial. What do you think
```
# Arithmetic Operations
In this tutorial, we are going to go over all arithmetic operations available in **Concrete**. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile the functions below.
## Addition
### Static ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x):
return x + 42
```
or
<!--python-test:skip-->
```python
def f(x):
return 42 + x
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(3) == 45
circuit.run(0) == 42
```
### Dynamic ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x, y):
return x + y
```
or
<!--python-test:skip-->
```python
def f(x, y):
return y + x
```
results in
<!--python-test:skip-->
```python
circuit.run(6, 4) == 10
circuit.run(1, 1) == 2
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
- `y = ClearScalar(UnsignedInteger(bits))`
### EncryptedScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x, y):
return x + y
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
- `y = EncryptedScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(7, 7) == 14
circuit.run(3, 4) == 7
```
## Subtraction
### Static ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x):
return 3 - x
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(2) == 1
circuit.run(3) == 0
```
### Dynamic ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x, y):
return y - x
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
- `y = ClearScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(2, 4) == 2
circuit.run(1, 7) == 6
```
## Multiplication
### Static ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x):
return x * 2
```
or
<!--python-test:skip-->
```python
def f(x):
return 2 * x
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(2) == 4
circuit.run(5) == 10
```
### Dynamic ClearScalar and EncryptedScalar
<!--python-test:skip-->
```python
def f(x, y):
return x * y
```
or
<!--python-test:skip-->
```python
def f(x, y):
return y * x
```
where
- `x = EncryptedScalar(UnsignedInteger(bits))`
- `y = ClearScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run(2, 3) == 6
circuit.run(1, 7) == 7
```
## Dot Product
### Dynamic ClearTensor and EncryptedTensor
<!--python-test:skip-->
```python
def f(x, y):
return np.dot(x, y)
```
or
<!--python-test:skip-->
```python
def f(x, y):
return np.dot(y, x)
```
where
- `x = EncryptedTensor(UnsignedInteger(bits), shape=(2,))`
- `y = ClearTensor(UnsignedInteger(bits), shape=(2,))`
results in
<!--python-test:skip-->
```python
circuit.run([1, 1], [2, 3]) == 5
circuit.run([2, 3], [2, 3]) == 13
```
## Combining all together
<!--python-test:skip-->
```python
def f(x, y, z):
return 100 - (2 * (np.dot(x, y) + z))
```
where
- `x = EncryptedTensor(UnsignedInteger(bits), shape=(2,))`
- `y = ClearTensor(UnsignedInteger(bits), shape=(2,))`
- `z = EncryptedScalar(UnsignedInteger(bits))`
results in
<!--python-test:skip-->
```python
circuit.run([1, 2], [4, 3], 10) == 60
circuit.run([2, 3], [3, 2], 5) == 66
```

183
docs/user/tutorial/COMPILATION_ARTIFACTS.md
View File

@@ -1,7 +1,3 @@
```{warning}
FIXME(Umut): check it is still valid. I guess yes, but some files may have gone or renamed.
```
# Compilation Artifacts
In this tutorial, we are going to go over the artifact system, which is designed to inspect/debug the compilation process easily.
@@ -16,7 +12,7 @@ def f(x):
return np.sin(x)
```
This function fails (for now) to compile because `Concrete` doesn't support floating point outputs. When you try to compile it (you might want to check [this](../howto/COMPILING_AND_EXECUTING.md) to see how you can do that), an exception will be raised and the artifacts will be exported automatically.
This function fails to compile because **Concrete Framework** doesn't support floating point outputs. When you try to compile it (you might want to check [this](../howto/COMPILING_AND_EXECUTING.md) to see how you can do that), an exception will be raised and the artifacts will be exported automatically.
### environment.txt
@@ -60,34 +56,58 @@ x :: EncryptedScalar<Integer<unsigned, 3 bits>>
### 1.initial.graph.txt
This file contains information about the initial computation graph of the function you are trying to compile.
This file contains textual representation of the initial computation graph right after tracing.
```
%0 = x # EncryptedScalar<Integer<unsigned, 3 bits>>
%1 = np.sin(0) # EncryptedScalar<Float<64 bits>>
return(%1)
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
### 1.initial.graph.png
This file contains the visualization of the initial computation graph of the function you are trying to compile.
This file contains the visual representation of the initial computation graph right after tracing.
![](../../_static/tutorials/artifacts/auto/1.initial.graph.png)
### 2.final.graph.txt
This file contains textual representation of the final computation graph right before MLIR conversion.
```
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
### 2.final.graph.png
This file contains the visual representation of the final computation graph right before MLIR conversion.
![](../../_static/tutorials/artifacts/auto/2.final.graph.png)
### traceback.txt
This file contains information about the error you got.
```
Traceback (most recent call last):
File "/src/concrete/numpy/compile.py", line 301, in compile_numpy_function
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 141, in run_compilation_function_with_error_management
return compilation_function()
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 769, in compilation_function
return _compile_numpy_function_internal(
File "/src/concrete/numpy/compile.py", line 234, in _compile_numpy_function_internal
op_graph = _compile_numpy_function_into_op_graph_internal(
File "/src/concrete/numpy/compile.py", line 103, in _compile_numpy_function_into_op_graph_internal
raise ValueError(
ValueError: <lambda> cannot be compiled as it has nodes with either float inputs or outputs.
Offending nodes : <concrete.common.representation.intermediate.GenericFunction object at 0x7f6689fd37f0>
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 722, in _compile_numpy_function_internal
fhe_circuit = _compile_op_graph_to_fhe_circuit_internal(
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 626, in _compile_op_graph_to_fhe_circuit_internal
prepare_op_graph_for_mlir(op_graph)
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 597, in prepare_op_graph_for_mlir
raise RuntimeError(
RuntimeError: function you are trying to compile isn't supported for MLIR lowering
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer outputs are supported
return %1
```
## Manual export
@@ -96,74 +116,114 @@ Manual exports are mostly used for visualization. Nonetheless, they can be very
```python
import concrete.numpy as hnp
import numpy as np
import pathlib
def f(x):
return 127 - (50 * (np.sin(x) + 1)).astype(np.uint32)
artifacts = hnp.CompilationArtifacts(pathlib.Path("/tmp/custom/export/path"))
hnp.compile_numpy_function(
lambda x: 100 - (3 * (x + 2)),
{"x": hnp.EncryptedScalar(hnp.UnsignedInteger(3))},
inputset=range(2 ** 3),
compilation_artifacts=artifacts,
)
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"}, compilation_artifacts=artifacts)
compiler.eval_on_inputset(range(2 ** 3))
artifacts.export()
```
Since this example compiles, we can see some new artifacts.
### 1.initial.graph.txt
This file contains information about the initial computation graph of the function you are trying to compile.
This file contains textual representation of the initial computation graph right after tracing.
```
%0 = Constant(100) # ClearScalar<Integer<unsigned, 7 bits>>
%1 = Constant(3) # ClearScalar<Integer<unsigned, 2 bits>>
%2 = x # EncryptedScalar<Integer<unsigned, 3 bits>>
%3 = Constant(2) # ClearScalar<Integer<unsigned, 2 bits>>
%4 = Add(2, 3) # EncryptedScalar<Integer<unsigned, 3 bits>>
%5 = Mul(4, 1) # EncryptedScalar<Integer<unsigned, 3 bits>>
%6 = Sub(0, 5) # EncryptedScalar<Integer<unsigned, 7 bits>>
return(%6)
%0 = 127 # ClearScalar<uint7>
%1 = 50 # ClearScalar<uint6>
%2 = 1 # ClearScalar<uint1>
%3 = x # EncryptedScalar<uint3>
%4 = sin(%3) # EncryptedScalar<float64>
%5 = add(%4, %2) # EncryptedScalar<float64>
%6 = mul(%5, %1) # EncryptedScalar<float64>
%7 = astype(%6, dtype=uint32) # EncryptedScalar<uint32>
%8 = sub(%0, %7) # EncryptedScalar<uint32>
return %8
```
### 1.initial.graph.png
This file contains the visualization of the initial computation graph of the function you are trying to compile.
This file contains the visual representation of the initial computation graph right after tracing.
![](../../_static/tutorials/artifacts/manual/1.initial.graph.png)
### 2.final.graph.txt
### 2.after-float-fuse-0.graph.txt
This file contains information about the final computation graph of the function you are trying to compile.
This file contains textual representation of the intermediate computation graph after fusing.
```
%0 = Constant(100) # ClearScalar<Integer<unsigned, 7 bits>>
%1 = Constant(3) # ClearScalar<Integer<unsigned, 2 bits>>
%2 = x # EncryptedScalar<Integer<unsigned, 3 bits>>
%3 = Constant(2) # ClearScalar<Integer<unsigned, 2 bits>>
%4 = Add(2, 3) # EncryptedScalar<Integer<unsigned, 4 bits>>
%5 = Mul(4, 1) # EncryptedScalar<Integer<unsigned, 5 bits>>
%6 = Sub(0, 5) # EncryptedScalar<Integer<unsigned, 7 bits>>
return(%6)
%0 = 127 # ClearScalar<uint7>
%1 = x # EncryptedScalar<uint3>
%2 = subgraph(%1) # EncryptedScalar<uint32>
%3 = sub(%0, %2) # EncryptedScalar<uint32>
return %3
Subgraphs:
%2 = subgraph(%1):
%0 = 50 # ClearScalar<uint6>
%1 = 1 # ClearScalar<uint1>
%2 = float_subgraph_input # EncryptedScalar<uint3>
%3 = sin(%2) # EncryptedScalar<float64>
%4 = add(%3, %1) # EncryptedScalar<float64>
%5 = mul(%4, %0) # EncryptedScalar<float64>
%6 = astype(%5, dtype=uint32) # EncryptedScalar<uint32>
return %6
```
### 2.final.graph.png
### 2.after-float-fuse-0.graph.png
This file contains the visualization of the final computation graph of the function you are trying to compile.
This file contains the visual representation of the intermediate computation graph after fusing.
![](../../_static/tutorials/artifacts/manual/2.final.graph.png)
![](../../_static/tutorials/artifacts/manual/2.after-float-fuse-0.graph.png)
### 3.final.graph.txt
This file contains textual representation of the final computation graph right before MLIR conversion.
```
%0 = 127 # ClearScalar<uint7>
%1 = x # EncryptedScalar<uint3>
%2 = subgraph(%1) # EncryptedScalar<uint7>
%3 = sub(%0, %2) # EncryptedScalar<uint7>
return %3
Subgraphs:
%2 = subgraph(%1):
%0 = 50 # ClearScalar<uint6>
%1 = 1 # ClearScalar<uint1>
%2 = float_subgraph_input # EncryptedScalar<uint3>
%3 = sin(%2) # EncryptedScalar<float64>
%4 = add(%3, %1) # EncryptedScalar<float64>
%5 = mul(%4, %0) # EncryptedScalar<float64>
%6 = astype(%5, dtype=uint32) # EncryptedScalar<uint7>
return %6
```
### 3.final.graph.png
This file contains the visual representation of the final computation graph right before MLIR conversion.
![](../../_static/tutorials/artifacts/manual/3.final.graph.png)
### bounds.txt
This file contains information about the bounds of the final computation graph of the function you are trying to compile using the input set you provide.
```
%0 :: [100, 100]
%1 :: [3, 3]
%2 :: [0, 7]
%3 :: [2, 2]
%4 :: [2, 9]
%5 :: [6, 27]
%6 :: [73, 94]
%0 :: [127, 127]
%1 :: [0, 7]
%2 :: [2, 95]
%3 :: [32, 125]
```
You can learn what bounds are [here](../../dev/explanation/TERMINOLOGY_AND_STRUCTURE.md).
@@ -175,15 +235,14 @@ This file contains information about the MLIR of the function you are trying to
```
module {
func @main(%arg0: !HLFHE.eint<7>) -> !HLFHE.eint<7> {
%c100_i8 = constant 100 : i8
%c3_i8 = constant 3 : i8
%c2_i8 = constant 2 : i8
%0 = "HLFHE.add_eint_int"(%arg0, %c2_i8) : (!HLFHE.eint<7>, i8) -> !HLFHE.eint<7>
%1 = "HLFHE.mul_eint_int"(%0, %c3_i8) : (!HLFHE.eint<7>, i8) -> !HLFHE.eint<7>
%2 = "HLFHE.sub_int_eint"(%c100_i8, %1) : (i8, !HLFHE.eint<7>) -> !HLFHE.eint<7>
return %2 : !HLFHE.eint<7>
%c127_i8 = arith.constant 127 : i8
%cst = arith.constant dense<"..."> : tensor<128xi64>
%0 = "HLFHE.apply_lookup_table"(%arg0, %cst) : (!HLFHE.eint<7>, tensor<128xi64>) -> !HLFHE.eint<7>
%1 = "HLFHE.sub_int_eint"(%c127_i8, %0) : (i8, !HLFHE.eint<7>) -> !HLFHE.eint<7>
return %1 : !HLFHE.eint<7>
}
}
```
You can learn more about MLIR [here](../../dev/explanation/MLIR.md).

104
docs/user/tutorial/INDEXING.md
View File

@@ -1,9 +1,103 @@
```{warning}
FIXME(Umut): to be done
```
# Indexing
# Slicing
## Constant Indexing
Constant indexing refers to the index being static (i.e., known during compilation).
Here are some examples of constant indexing:
### Extracting a single element
```python
import concrete.numpy as hnp
import numpy as np
def f(x):
return x[1]
inputset = [np.random.randint(0, 2 ** 3, size=(3,), dtype=np.uint8) for _ in range(10)]
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"})
compiler.eval_on_inputset(inputset)
circuit = compiler.get_compiled_fhe_circuit()
test_input = np.array([4, 2, 6], dtype=np.uint8)
expected_output = 2
assert np.array_equal(circuit.run(test_input), expected_output)
```
You can use negative indexing.
```python
import concrete.numpy as hnp
import numpy as np
def f(x):
return x[-1]
inputset = [np.random.randint(0, 2 ** 3, size=(3,), dtype=np.uint8) for _ in range(10)]
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"})
compiler.eval_on_inputset(inputset)
circuit = compiler.get_compiled_fhe_circuit()
test_input = np.array([4, 2, 6], dtype=np.uint8)
expected_output = 6
assert np.array_equal(circuit.run(test_input), expected_output)
```
You can use multidimensional indexing as well.
```python
import concrete.numpy as hnp
import numpy as np
def f(x):
return x[-1, 1]
inputset = [np.random.randint(0, 2 ** 3, size=(3, 2), dtype=np.uint8) for _ in range(10)]
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"})
compiler.eval_on_inputset(inputset)
circuit = compiler.get_compiled_fhe_circuit()
test_input = np.array([[4, 2], [1, 5], [7, 6]], dtype=np.uint8)
expected_output = 6
assert np.array_equal(circuit.run(test_input), expected_output)
```
### Extracting a slice
```python
import concrete.numpy as hnp
import numpy as np
def f(x):
return x[1:4]
inputset = [np.random.randint(0, 2 ** 3, size=(5,), dtype=np.uint8) for _ in range(10)]
compiler = hnp.NPFHECompiler(f, {"x": "encrypted"})
compiler.eval_on_inputset(inputset)
circuit = compiler.get_compiled_fhe_circuit()
test_input = np.array([4, 2, 6, 1, 7], dtype=np.uint8)
expected_output = np.array([2, 6, 1], dtype=np.uint8)
assert np.array_equal(circuit.run(test_input), expected_output)
```
You can use multidimensional slicing as well.
#### Note
There are certain limitations of slicing due to MLIR. So if you stumple into `RuntimeError: Compilation failed: Failed to lower to LLVM dialect`, know that we are aware of it, and we are trying to make such cases compilable.
## Dynamic Indexing
Dynamic indexing refers to the index being dynamic (i.e., can change during runtime).
Such indexing is especially useful for things like decision trees.
Unfortunately, we don't support dynamic indexing for the time being.

68
docs/user/tutorial/TABLE_LOOKUP.md
View File

@@ -1,20 +1,15 @@
```{warning}
FIXME(Umut): update a bit, with the new API
FIXME(Umut): add the multiTLU case
```
# Table Lookup
In this tutorial, we are going to go over the ways to perform table lookups in **Concrete**. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile the functions below.
In this tutorial, we are going to go over the ways to perform direct table lookups in **Concrete Framework**. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile the functions below.
## Direct table lookup
**Concrete** provides a special class to allow direct table lookups. Here is how to import and use it:
**Concrete Framework** provides a special class to allow direct table lookups. Here is how to use it:
```python
from concrete.common.extensions.table import LookupTable
import concrete.numpy as hnp
table = LookupTable([2, 1, 3, 0])
table = hnp.LookupTable([2, 1, 3, 0])
def f(x):
return table[x]
@@ -22,7 +17,7 @@ def f(x):
where
- `x = EncryptedScalar(UnsignedInteger(2))`
- `x = "encrypted"` scalar
results in
@@ -34,9 +29,56 @@ circuit.run(2) == 3
circuit.run(3) == 0
```
Moreover, direct lookup tables can be used with tensors where the same table lookup is applied to each value in the tensor, so
- `x = "encrypted"` tensor of shape `(2, 3)`
results in
<!--python-test:skip-->
```python
input = np.array([[0, 1, 3], [2, 3, 1]], dtype=np.uint8)
circuit.run(input) == [[2, 1, 0], [3, 0, 1]]
```
## Direct Multi Table Lookup
Sometimes you may want to apply a different lookup table to each value in a tensor. That's where direct multi lookup table becomes handy. Here is how to use it:
<!--python-test:skip-->
```python
import concrete.numpy as hnp
squared = hnp.LookupTable([i ** 2 for i in range(4)])
cubed = hnp.LookupTable([i ** 3 for i in range(4)])
table = hnp.MultiLookupTable([
[squared, cubed],
[squared, cubed],
[squared, cubed],
])
def f(x):
return table[x]
```
where
- `x = "encrypted"` tensor of shape `(3, 2)`
results in
<!--python-test:skip-->
```python
input = np.array([[2, 3], [1, 2], [3, 0]], dtype=np.uint8)
circuit.run(input) == [[4, 27], [1, 8], [9, 0]]
```
Basically, we applied `squared` table to the first column and `cubed` to the second one.
## Fused table lookup
Direct tables are tedious to prepare by hand. When possible, **Concrete** fuses the floating point operations into a single table lookup automatically. There are some limitations on fusing operations, which you can learn more about on the next tutorial, [Working With Floating Points](./WORKING_WITH_FLOATING_POINTS.md).
Direct tables are tedious to prepare by hand. When possible, **Concrete Framework** fuses the floating point operations into table lookups automatically. There are some limitations on fusing operations, which you can learn more about on the next tutorial, [Working With Floating Points](./WORKING_WITH_FLOATING_POINTS.md).
Here is an example function that results in fused table lookup:
@@ -48,7 +90,7 @@ def f(x):
where
- `x = EncryptedScalar(UnsignedInteger(3))`
- `x = "encrypted"` scalar
results in
@@ -76,7 +118,7 @@ Internally, it uses the following lookup table
<!--python-test:skip-->
```python
table = LookupTable([50, 92, 95, 57, 12, 2, 36, 82])
table = hnp.LookupTable([50, 92, 95, 57, 12, 2, 36, 82])
```
which is calculated by: