github/concrete
1
1
Fork 0
mirror of https://github.com/zama-ai/concrete.git synced 2026-02-09 03:55:04 -05:00
Code Issues Packages Projects Releases Wiki Activity

docs: re-write documentation

Browse Source
This commit is contained in:
aquint-zama
2022-06-01 15:41:37 +02:00
committed by Umut
parent 546ed48765
commit 35e46aca69
53 changed files with 1505 additions and 1673 deletions
Download Patch File Download Diff File Expand all files Collapse all files

247
docs/tutorial/compilation_artifacts.md
View File

@@ -1,247 +0,0 @@
# Compilation artifacts
In this tutorial, we are going to go over the artifact system, which is designed to inspect/debug the compilation process easily.
## Automatic export
In case of compilation failures, artifacts are exported automatically to `.artifacts` directory under the working directory. Let's intentionally create a compilation failure and show what kinds of things are exported.
<!--pytest-codeblocks:skip-->
```python
def f(x):
return np.sin(x)
```
This function fails to compile because **Concrete Numpy** doesn't support floating point outputs. When you try to compile it (you might want to check [this](../basics/compiling_and_executing.md) to see how you can do that), an exception will be raised and the artifacts will be exported automatically.
### environment.txt
This file contains information about your setup (i.e., your operating system and python version).
```
Linux-5.12.13-arch1-2-x86_64-with-glibc2.29 #1 SMP PREEMPT Fri, 25 Jun 2021 22:56:51 +0000
Python 3.8.10
```
### requirements.txt
This file contains information about python packages and their versions installed on your system.
```
alabaster==0.7.12
appdirs==1.4.4
argon2-cffi==21.1.0
...
wheel==0.37.0
widgetsnbextension==3.5.1
wrapt==1.12.1
```
### function.txt
This file contains information about the function you are trying to compile.
```
def f(x):
return np.sin(x)
```
### parameters.txt
This file contains information about the parameters of the function you are trying to compile.
```
x :: EncryptedScalar<Integer<unsigned, 3 bits>>
```
### 1.initial.graph.txt
This file contains textual representation of the initial computation graph right after tracing.
```
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
### 1.initial.graph.png
This file contains the visual representation of the initial computation graph right after tracing.
![](../_static/tutorials/artifacts/auto/1.initial.graph.png)
### 2.final.graph.txt
This file contains textual representation of the final computation graph right before MLIR conversion.
```
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
### 2.final.graph.png
This file contains the visual representation of the final computation graph right before MLIR conversion.
![](../_static/tutorials/artifacts/auto/2.final.graph.png)
### traceback.txt
This file contains information about the error you got.
```
Traceback (most recent call last):
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 141, in run_compilation_function_with_error_management
return compilation_function()
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 769, in compilation_function
return _compile_numpy_function_internal(
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 722, in _compile_numpy_function_internal
fhe_circuit = _compile_op_graph_to_fhe_circuit_internal(
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 626, in _compile_op_graph_to_fhe_circuit_internal
prepare_op_graph_for_mlir(op_graph)
File "/home/default/Documents/Projects/Zama/hdk/concrete/numpy/compile.py", line 597, in prepare_op_graph_for_mlir
raise RuntimeError(
RuntimeError: function you are trying to compile isn't supported for MLIR lowering
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer outputs are supported
return %1
```
## Manual export
Manual exports are mostly used for visualization. Nonetheless, they can be very useful for demonstrations. Here is how to do it:
```python
import concrete.numpy as cnp
import numpy as np
import pathlib
artifacts = cnp.DebugArtifacts("/tmp/custom/export/path")
@cnp.compiler({"x": "encrypted"})
def f(x):
return 127 - (50 * (np.sin(x) + 1)).astype(np.int64)
f.compile(range(2 ** 3), artifacts=artifacts)
artifacts.export()
```
### 1.initial.graph.txt
This file contains textual representation of the initial computation graph right after tracing.
```
%0 = 127 # ClearScalar<uint7>
%1 = 50 # ClearScalar<uint6>
%2 = 1 # ClearScalar<uint1>
%3 = x # EncryptedScalar<uint3>
%4 = sin(%3) # EncryptedScalar<float64>
%5 = add(%4, %2) # EncryptedScalar<float64>
%6 = mul(%5, %1) # EncryptedScalar<float64>
%7 = astype(%6, dtype=uint32) # EncryptedScalar<uint32>
%8 = sub(%0, %7) # EncryptedScalar<uint32>
return %8
```
### 1.initial.graph.png
This file contains the visual representation of the initial computation graph right after tracing.
![](../_static/tutorials/artifacts/manual/1.initial.graph.png)
### 2.after-float-fuse-0.graph.txt
This file contains textual representation of the intermediate computation graph after fusing.
```
%0 = 127 # ClearScalar<uint7>
%1 = x # EncryptedScalar<uint3>
%2 = subgraph(%1) # EncryptedScalar<uint32>
%3 = sub(%0, %2) # EncryptedScalar<uint32>
return %3
Subgraphs:
%2 = subgraph(%1):
%0 = 50 # ClearScalar<uint6>
%1 = 1 # ClearScalar<uint1>
%2 = float_subgraph_input # EncryptedScalar<uint3>
%3 = sin(%2) # EncryptedScalar<float64>
%4 = add(%3, %1) # EncryptedScalar<float64>
%5 = mul(%4, %0) # EncryptedScalar<float64>
%6 = astype(%5, dtype=uint32) # EncryptedScalar<uint32>
return %6
```
### 2.after-float-fuse-0.graph.png
This file contains the visual representation of the intermediate computation graph after fusing.
![](../_static/tutorials/artifacts/manual/2.after-float-fuse-0.graph.png)
### 3.final.graph.txt
This file contains textual representation of the final computation graph right before MLIR conversion.
```
%0 = 127 # ClearScalar<uint7>
%1 = x # EncryptedScalar<uint3>
%2 = subgraph(%1) # EncryptedScalar<uint7>
%3 = sub(%0, %2) # EncryptedScalar<uint7>
return %3
Subgraphs:
%2 = subgraph(%1):
%0 = 50 # ClearScalar<uint6>
%1 = 1 # ClearScalar<uint1>
%2 = float_subgraph_input # EncryptedScalar<uint3>
%3 = sin(%2) # EncryptedScalar<float64>
%4 = add(%3, %1) # EncryptedScalar<float64>
%5 = mul(%4, %0) # EncryptedScalar<float64>
%6 = astype(%5, dtype=uint32) # EncryptedScalar<uint7>
return %6
```
### 3.final.graph.png
This file contains the visual representation of the final computation graph right before MLIR conversion.
![](../_static/tutorials/artifacts/manual/3.final.graph.png)
### bounds.txt
This file contains information about the bounds of the final computation graph of the function you are trying to compile using the input set you provide.
```
%0 :: [127, 127]
%1 :: [0, 7]
%2 :: [2, 95]
%3 :: [32, 125]
```
You can learn what bounds are [here](../../dev/explanation/terminology_and_structure.md).
### mlir.txt
This file contains information about the MLIR of the function you are trying to compile using the input set you provide.
```
module {
func @main(%arg0: !FHE.eint<7>) -> !FHE.eint<7> {
%c127_i8 = arith.constant 127 : i8
%cst = arith.constant dense<"..."> : tensor<128xi64>
%0 = "FHE.apply_lookup_table"(%arg0, %cst) : (!FHE.eint<7>, tensor<128xi64>) -> !FHE.eint<7>
%1 = "FHE.sub_int_eint"(%c127_i8, %0) : (i8, !FHE.eint<7>) -> !FHE.eint<7>
return %1 : !FHE.eint<7>
}
}
```
You can learn more about MLIR [here](../../dev/explanation/mlir.md).

20
docs/tutorial/decorator.md Normal file
View File

@@ -0,0 +1,20 @@
# Decorator
If you are trying to compile a regular function, you can use the decorator interface instead of the explicit `Compiler` interface to simplify your code:
```python
import concrete.numpy as cnp
@cnp.compiler({"x": "encrypted"})
def f(x):
return x + 42
inputset = range(10)
circuit = f.compile(inputset)
assert circuit.encrypt_run_decrypt(10) == f(10)
```
{% hint style="info" %}
Think of this decorator as a way to add the `compile` method to the function object without changing its name elsewhere.
{% endhint %}

153
docs/tutorial/extensions.md Normal file
View File

@@ -0,0 +1,153 @@
# Extensions
**Concrete Numpy** tries to support **NumPy** as much as possible, but due to some technical limitations, not everything can be supported. On top of that, there are some things **NumPy** lack, which are useful. In some of these situations, we provide extesions in **Concrete Numpy** to improve your experience.
## cnp.zero()
Allows you to create encrypted scalar zero:
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
z = cnp.zero()
return x + z
inputset = range(10)
circuit = f.compile(inputset)
for x in range(10):
assert circuit.encrypt_run_decrypt(x) == x
```
## cnp.zeros(shape)
Allows you to create encrypted tensor of zeros:
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
z = cnp.zeros((2, 3))
return x + z
inputset = range(10)
circuit = f.compile(inputset)
for x in range(10):
assert np.array_equal(circuit.encrypt_run_decrypt(x), np.array([[x, x, x], [x, x, x]]))
```
## cnp.one()
Allows you to create encrypted scalar one:
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
z = cnp.one()
return x + z
inputset = range(10)
circuit = f.compile(inputset)
for x in range(10):
assert circuit.encrypt_run_decrypt(x) == x + 1
```
## cnp.ones(shape)
Allows you to create encrypted tensor of ones:
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
z = cnp.ones((2, 3))
return x + z
inputset = range(10)
circuit = f.compile(inputset)
for x in range(10):
assert np.array_equal(circuit.encrypt_run_decrypt(x), np.array([[x, x, x], [x, x, x]]) + 1)
```
## cnp.univariate(function)
Allows you to wrap any univariate function into a single table lookup:
```python
import concrete.numpy as cnp
import numpy as np
def complex_univariate_function(x):
def per_element(element):
result = 0
for i in range(element):
result += i
return result
return np.vectorize(per_element)(x)
@cnp.compiler({"x": "encrypted"})
def f(x):
return cnp.univariate(complex_univariate_function)(x)
inputset = [np.random.randint(0, 5, size=(3, 2)) for _ in range(10)]
circuit = f.compile(inputset)
sample = np.array([
[0, 4],
[2, 1],
[3, 0],
])
assert np.array_equal(circuit.encrypt_run_decrypt(sample), complex_univariate_function(sample))
```
{% hint style="danger" %}
The wrapped function shouldn't have any side effects, and it should be deterministic.
{% endhint %}
## coonx.conv(...)
Allows you to perform a convolution operation, with the same semantic of [onnx.Conv](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Conv):
```python
import concrete.numpy as cnp
import concrete.onnx as connx
import numpy as np
weight = np.array([[2, 1], [3, 2]]).reshape(1, 1, 2, 2)
@cnp.compiler({"x": "encrypted"})
def f(x):
return connx.conv(x, weight, strides=(2, 2), dilations=(1, 1), group=1)
inputset = [np.random.randint(0, 4, size=(1, 1, 4, 4)) for _ in range(10)]
circuit = f.compile(inputset)
sample = np.array(
[
[3, 2, 1, 0],
[3, 2, 1, 0],
[3, 2, 1, 0],
[3, 2, 1, 0],
]
).reshape(1, 1, 4, 4)
assert np.array_equal(circuit.encrypt_run_decrypt(sample), f(sample))
```
{% hint style="danger" %}
Only 2D convolutions with one groups and without padding are supported for the time being.
{% endhint %}

79
docs/tutorial/floating_points.md Normal file
View File

@@ -0,0 +1,79 @@
# Floating Points
**Concrete Numpy** partly supports floating points:
* They cannot be inputs
* They cannot be outputs
* They can be intermediate values under certain constraints
## As intermediate values
**Concrete Compile**, which is used for compiling the circuit, doesn't support floating points at all. However, it supports table lookups. They take an integer and map it to another integer. It does not care how the lookup table is calculated. Further, the constraints of this operation are such that there should be a single integer input and it should result in a single integer output.
As long as your floating point operations comply with those constraints, **Concrete Numpy** automatically converts your operations to a table lookup operation:
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
a = x + 1.5
b = np.sin(x)
c = np.around(a + b)
d = c.astype(np.int64)
return d
inputset = range(8)
circuit = f.compile(inputset)
for x in range(8):
assert circuit.encrypt_run_decrypt(x) == f(x)
```
In the example above, `a`, `b`, and `c` are all floating point intermediates. However, they are just used to calculate `d`, which is an integer and value of `d` dependent upon `x` , which is another integer. **Concrete Numpy** detects this and fuses all of those operations into a single table lookup from `x` to `d`.
This approach works for a variety of use cases, but it comes up short for some:
<!--pytest-codeblocks:skip-->
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted", "y": "encrypted"})
def f(x, y):
a = x + 1.5
b = np.sin(y)
c = np.around(a + b)
d = c.astype(np.int64)
return d
inputset = [(1, 2), (3, 0), (2, 2), (1, 3)]
circuit = f.compile(inputset)
for x in range(8):
assert circuit.encrypt_run_decrypt(x) == f(x)
```
results in
```
RuntimeError: Function you are trying to compile cannot be converted to MLIR
%0 = x # EncryptedScalar<uint2>
%1 = 1.5 # ClearScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer constants are supported
%2 = y # EncryptedScalar<uint2>
%3 = add(%0, %1) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer operations are supported
%4 = sin(%2) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer operations are supported
%5 = add(%3, %4) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer operations are supported
%6 = around(%5) # EncryptedScalar<float64>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer operations are supported
%7 = astype(%6, dtype=int_) # EncryptedScalar<uint3>
return %7
```
The reason for that is that `d` no longer depends solely on `x`, it depends on `y` as well. Thus, **Concrete Numpy** cannot fuse these operations, so it raises an exception.

64
docs/tutorial/formatting_and_drawing.md Normal file
View File

@@ -0,0 +1,64 @@
# Format and Draw
Sometimes, it can be useful to format or draw circuits. We provide methods to just do that.
## Formatting
You can convert your compiled circuit into its textual representation by converting it to string:
<!--pytest-codeblocks:skip-->
```python
str(circuit)
```
If you just want to see the output on your terminal, you can directly print it as well:
<!--pytest-codeblocks:skip-->
```python
print(circuit)
```
## Drawing
{% hint style="danger" %}
Drawing functionality requires the installation of the package with a full feature set. See the Installation section to learn how to do that.
{% endhint %}
You can use the `draw` method of your compiled circuit to draw it:
<!--pytest-codeblocks:skip-->
```python
drawing = circuit.draw()
```
This method will draw the circuit on a temporary PNG file and return the path to this file.
You can show the drawing in a Jupyter notebook, like this:
<!--pytest-codeblocks:skip-->
```python
from PIL import Image
drawing = Image.open(circuit.draw())
drawing.show()
drawing.close()
```
Or, you can use the `show` option of the `draw` method to show the drawing with matplotlib.
<!--pytest-codeblocks:skip-->
```python
circuit.draw(show=True)
```
{% hint style="danger" %}
Beware that this will clear the matplotlib plots you have.
{% endhint %}
Lastly, you can save the drawing to a specific path:
<!--pytest-codeblocks:skip-->
```python
destination = "/tmp/path/of/your/choice.png"
drawing = circuit.draw(save_to=destination)
assert drawing == destination
```

95
docs/tutorial/indexing.md
View File

@@ -1,95 +0,0 @@
# Indexing
## Constant Indexing
Constant indexing refers to the index being static (i.e., known during compilation).
Here are some examples of constant indexing:
### Extracting a single element
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
return x[1]
inputset = [np.random.randint(0, 2 ** 3, size=(3,), dtype=np.uint8) for _ in range(10)]
circuit = f.compile(inputset)
test_input = np.array([4, 2, 6], dtype=np.uint8)
expected_output = 2
assert np.array_equal(circuit.encrypt_run_decrypt(test_input), expected_output)
```
You can use negative indexing.
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
return x[-1]
inputset = [np.random.randint(0, 2 ** 3, size=(3,), dtype=np.uint8) for _ in range(10)]
circuit = f.compile(inputset)
test_input = np.array([4, 2, 6], dtype=np.uint8)
expected_output = 6
assert np.array_equal(circuit.encrypt_run_decrypt(test_input), expected_output)
```
You can use multidimensional indexing as well.
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
return x[-1, 1]
inputset = [np.random.randint(0, 2 ** 3, size=(3, 2), dtype=np.uint8) for _ in range(10)]
circuit = f.compile(inputset)
test_input = np.array([[4, 2], [1, 5], [7, 6]], dtype=np.uint8)
expected_output = 6
assert np.array_equal(circuit.encrypt_run_decrypt(test_input), expected_output)
```
### Extracting a slice
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
return x[1:4]
inputset = [np.random.randint(0, 2 ** 3, size=(5,), dtype=np.uint8) for _ in range(10)]
circuit = f.compile(inputset)
test_input = np.array([4, 2, 6, 1, 7], dtype=np.uint8)
expected_output = np.array([2, 6, 1], dtype=np.uint8)
assert np.array_equal(circuit.encrypt_run_decrypt(test_input), expected_output)
```
You can use multidimensional slicing as well.
{% hint style='tip' %}
There are certain limitations of slicing due to MLIR. So if you stumple into `RuntimeError: Compilation failed: Failed to lower to LLVM dialect`, know that we are aware of it, and we are trying to make such cases compilable.
{% endhint %}
## Dynamic Indexing
Dynamic indexing refers to the index being dynamic (i.e., can change during runtime).
Such indexing is especially useful for things like decision trees.
Unfortunately, we don't support dynamic indexing for the time being.

215
docs/tutorial/table_lookup.md
View File

@@ -1,87 +1,108 @@
# Table lookup
# Table Lookups
In this tutorial, we are going to go over the ways to perform direct table lookups in **Concrete Numpy**. Please read [Compiling and Executing](../basics/compiling\_and\_executing.md) before reading further to see how you can compile the functions below.
In this tutorial, we will review the ways to perform direct table lookups in **Concrete Numpy**.
## Direct table lookup
**Concrete Numpy** provides a special class to allow direct table lookups. Here is how to use it:
**Concrete Numpy** provides a `LookupTable` class for you to create your own tables and apply them in your circuits.
{% hint style="info" %}
`LookupTable`s can have any number of elements. Let's call them **N**. As long as the lookup variable is in range \[-**N**, **N**), table lookup is valid.
If you go out of bounds of this range, you will get the following error:
```
IndexError: index 10 is out of bounds for axis 0 with size 6
```
{% endhint %}
{% hint style="info" %}
The number of elements in the lookup table doesn't affect performance in any way.
{% endhint %}
### With scalars.
You can create the lookup table using a list of integers and apply it using indexing:
```python
import concrete.numpy as cnp
table = cnp.LookupTable([2, -1, 3, 0])
@cnp.compiler({"x": "encrypted"})
def f(x):
return table[x]
inputset = range(4)
circuit = f.compile(inputset)
assert circuit.encrypt_run_decrypt(0) == table[0] == 2
assert circuit.encrypt_run_decrypt(1) == table[1] == -1
assert circuit.encrypt_run_decrypt(2) == table[2] == 3
assert circuit.encrypt_run_decrypt(3) == table[3] == 0
```
where
### With tensors.
* `x = "encrypted"` scalar
When you apply the table lookup to a tensor, you apply the scalar table lookup to each element of the tensor:
results in
<!--pytest-codeblocks:skip-->
```python
circuit.encrypt_run_decrypt(0) == 2
circuit.encrypt_run_decrypt(1) == -1
circuit.encrypt_run_decrypt(2) == 3
circuit.encrypt_run_decrypt(3) == 0
import concrete.numpy as cnp
import numpy as np
table = cnp.LookupTable([2, -1, 3, 0])
@cnp.compiler({"x": "encrypted"})
def f(x):
return table[x]
inputset = [np.random.randint(0, 4, size=(2, 3)) for _ in range(10)]
circuit = f.compile(inputset)
sample = [
[0, 1, 3],
[2, 3, 1],
]
expected_output = [
[2, -1, 0],
[3, 0, -1],
]
actual_output = circuit.encrypt_run_decrypt(np.array(sample))
for i in range(2):
for j in range(3):
assert actual_output[i][j] == expected_output[i][j] == table[sample[i][j]]
```
Moreover, direct lookup tables can be used with tensors where the same table lookup is applied to each value in the tensor, so
### With negative values.
* `x = "encrypted"` tensor of shape `(2, 3)`
results in
<!--pytest-codeblocks:skip-->
```python
input = np.array([[0, 1, 3], [2, 3, 1]], dtype=np.uint8)
circuit.encrypt_run_decrypt(input) == [[2, 1, 0], [3, 0, 1]]
```
Direct table lookups behaves like array indexing in python. Which means, if the lookup variable is negative, table is looked up from the back.
`LookupTable` mimics array indexing in Python, which means if the lookup variable is negative, the table is looked up from the back:
```python
import concrete.numpy as cnp
table = cnp.LookupTable([2, 1, 3, 0])
table = cnp.LookupTable([2, -1, 3, 0])
@cnp.compiler({"x": "encrypted"})
def f(x):
return table[-x]
inputset = range(1, 5)
circuit = f.compile(inputset)
assert circuit.encrypt_run_decrypt(1) == table[-1] == 0
assert circuit.encrypt_run_decrypt(2) == table[-2] == 3
assert circuit.encrypt_run_decrypt(3) == table[-3] == -1
assert circuit.encrypt_run_decrypt(4) == table[-4] == 2
```
where
* `x = "encrypted"` scalar
results in
<!--pytest-codeblocks:skip-->
```python
circuit.encrypt_run_decrypt(0) == 2
circuit.encrypt_run_decrypt(1) == 0
circuit.encrypt_run_decrypt(2) == 3
circuit.encrypt_run_decrypt(3) == 1
circuit.encrypt_run_decrypt(4) == 2
```
Lastly, a `LookupTable` can have any number of elements, let's call it **N**, as long as the lookup variable is in range \[-**N**, **N**). If you go out of bounds of this range, you will get the following error:
```
IndexError: index 10 is out of bounds for axis 0 with size 6
```
Note that, number of elements in the lookup table doesn't affect the performance in any way.
## Direct multi table lookup
Sometimes you may want to apply a different lookup table to each value in a tensor. That's where direct multi lookup table becomes handy. Here is how to use it:
In case you want to apply a different lookup table to each element of a tensor, you can have a `LookupTable` of `LookupTable`s:
<!--pytest-codeblocks:skip-->
```python
import concrete.numpy as cnp
import numpy as np
squared = cnp.LookupTable([i ** 2 for i in range(4)])
cubed = cnp.LookupTable([i ** 3 for i in range(4)])
@@ -92,72 +113,66 @@ table = cnp.LookupTable([
[squared, cubed],
])
@cnp.compiler({"x": "encrypted"})
def f(x):
return table[x]
inputset = [np.random.randint(0, 4, size=(3, 2)) for _ in range(10)]
circuit = f.compile(inputset)
sample = [
[0, 1],
[2, 3],
[3, 0],
]
expected_output = [
[0, 1],
[4, 27],
[9, 0]
]
actual_output = circuit.encrypt_run_decrypt(np.array(sample))
for i in range(3):
for j in range(2):
if j == 0:
assert actual_output[i][j] == expected_output[i][j] == squared[sample[i][j]]
else:
assert actual_output[i][j] == expected_output[i][j] == cubed[sample[i][j]]
```
where
* `x = "encrypted"` tensor of shape `(3, 2)`
results in
<!--pytest-codeblocks:skip-->
```python
input = np.array([[2, 3], [1, 2], [3, 0]], dtype=np.uint8)
circuit.encrypt_run_decrypt(input) == [[4, 27], [1, 8], [9, 0]]
```
Basically, we applied `squared` table to the first column and `cubed` to the second one.
In this example, we applied a `squared` table to the first column and a `cubed` table to the second one.
## Fused table lookup
Direct tables are tedious to prepare by hand. When possible, **Concrete Numpy** fuses the floating point operations into table lookups automatically. There are some limitations on fusing operations, which you can learn more about on the next tutorial, [Working With Floating Points](working\_with\_floating\_points.md).
**Concrete Numpy** tries to fuse some operations into table lookups automatically, so you don't need to create the lookup tables manually:
Here is an example function that results in fused table lookup:
<!--pytest-codeblocks:skip-->
```python
import concrete.numpy as cnp
import numpy as np
@cnp.compiler({"x": "encrypted"})
def f(x):
return 127 - (50 * (np.sin(x) + 1)).astype(np.int64) # astype is to go back to integer world
return (42 * np.sin(x)).astype(np.int64) // 10
inputset = range(8)
circuit = f.compile(inputset)
for x in range(8):
assert circuit.encrypt_run_decrypt(x) == f(x)
```
where
{% hint style="info" %}
All lookup tables need to be from integers to integers. So, without `.astype(np.int64)`, **Concrete Numpy** will not be able to fuse.
{% endhint %}
* `x = "encrypted"` scalar
results in
<!--pytest-codeblocks:skip-->
```python
circuit.encrypt_run_decrypt(0) == 77
circuit.encrypt_run_decrypt(1) == 35
circuit.encrypt_run_decrypt(2) == 32
circuit.encrypt_run_decrypt(3) == 70
circuit.encrypt_run_decrypt(4) == 115
circuit.encrypt_run_decrypt(5) == 125
circuit.encrypt_run_decrypt(6) == 91
circuit.encrypt_run_decrypt(7) == 45
```
Initially, the function is converted to this operation graph
The function is first traced into:
![](../\_static/tutorials/table-lookup/1.initial.graph.png)
and after floating point operations are fused, we get the following operation graph
Then, **Concrete Numpy** fuses appropriate nodes:
![](../\_static/tutorials/table-lookup/3.final.graph.png)
Internally, it uses the following lookup table
<!--pytest-codeblocks:skip-->
```python
table = cnp.LookupTable([50, 92, 95, 57, 12, 2, 36, 82])
```
which is calculated by:
<!--pytest-codeblocks:skip-->
```python
[(50 * (np.sin(x) + 1)).astype(np.int64) for x in range(2 ** 3)]
```
{% hint style="info" %}
Fusing makes the code more readable and easier to modify. So try to utilize it over manual `LookupTable`s as much as possible.
{% endhint %}

36
docs/tutorial/working_with_floating_points.md
View File

@@ -1,36 +0,0 @@
# Working with floating points
## An example
```python
import concrete.numpy as cnp
import numpy as np
# Function using floating points values converted back to integers at the end
@cnp.compiler({"x": "encrypted"})
def f(x):
return np.fabs(50 * (2 * np.sin(x) * np.cos(x))).astype(np.int64)
# astype is to go back to the integer world
circuit = f.compile(range(64))
print(circuit.encrypt_run_decrypt(3) == f(3))
print(circuit.encrypt_run_decrypt(0) == f(0))
print(circuit.encrypt_run_decrypt(1) == f(1))
print(circuit.encrypt_run_decrypt(10) == f(10))
print(circuit.encrypt_run_decrypt(60) == f(60))
print("All good!")
```
You can look to [numpy supported functions](../howto/numpy\_support.md) for information about possible float operations.
## Limitations
Floating point support in **Concrete Numpy** is very limited for the time being. They can't appear on inputs, or they can't be outputs. However, they can be used in intermediate results. Unfortunately, there are limitations on that front as well.
This biggest one is that, because floating point operations are fused into table lookups with a single unsigned integer input and single unsigned integer output, only univariate portion of code can be replaced with table lookups, which means multivariate portions cannot be compiled.
To give a precise example, `100 - np.fabs(50 * (np.sin(x) + np.sin(y)))` cannot be compiled because the floating point part depends on both `x` and `y` (i.e., it cannot be rewritten in the form `100 - table[z]` for a `z` that could be computed easily from `x` and `y`).
To dive into implementation details, you may refer to [Fusing Floating Point Operations](../../developer/float-fusing.md) document.