github/concrete
1
1
Fork 0
mirror of https://github.com/zama-ai/concrete.git synced 2026-04-17 03:00:54 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs(frontend/python): overhaul python frontend docs

Browse Source
This commit is contained in:
aquint-zama
2023-03-13 09:17:59 +01:00
committed by Alex Quint
parent 8a672a0c59
commit fdf6f41a89
69 changed files with 1000 additions and 1219 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
docs/howto/configure.md Normal file
View File

@@ -0,0 +1,104 @@
# Configure
The behavior of **Concrete** can be customized using `Configuration`s:
```python
from concrete import fhe
import numpy as np
configuration = fhe.Configuration(p_error=0.01, dataflow_parallelize=True)
@fhe.compiler({"x": "encrypted"})
def f(x):
return x + 42
inputset = range(10)
circuit = f.compile(inputset, configuration=configuration)
```
Alternatively, you can overwrite individual options as kwargs to `compile` method:
```python
from concrete import fhe
import numpy as np
@fhe.compiler({"x": "encrypted"})
def f(x):
return x + 42
inputset = range(10)
circuit = f.compile(inputset, p_error=0.01, dataflow_parallelize=True)
```
Or you can combine both:
```python
from concrete import fhe
import numpy as np
configuration = fhe.Configuration(p_error=0.01)
@fhe.compiler({"x": "encrypted"})
def f(x):
return x + 42
inputset = range(10)
circuit = f.compile(inputset, configuration=configuration, loop_parallelize=True)
```
{% hint style="info" %}
Additional kwarg to `compile` function have higher precedence. So if you set an option in both `configuration` and in `compile` methods, the value in the `compile` method will be used.
{% endhint %}
## Options
* **show\_graph**: Optional[bool] = None
* Whether to print computation graph during compilation.
`True` means always to print, `False` means always to not print, `None` means print depending on verbose configuration below.
* **show\_mlir**: Optional[bool] = None
* Whether to print MLIR during compilation.
`True` means always to print, `False` means always to not print, `None` means print depending on verbose configuration below.
* **show\_optimizer**: Optional[bool] = None
* Whether to print optimizer output during compilation.
`True` means always to print, `False` means always to not print, `None` means print depending on verbose configuration below.
* **verbose**: bool = False
* Whether to print details related to compilation.
* **dump\_artifacts\_on\_unexpected\_failures**: bool = True
* Whether to export debugging artifacts automatically on compilation failures.
* **auto\_adjust\_rounders**: bool = False
* Whether to adjust rounders automatically.
* **p\_error**: Optional[float] = None
* Error probability for individual table lookups. If set, all table lookups will have the probability of non-exact result smaller than the set value. See [Exactness](../getting-started/exactness.md) to learn more.
* **global\_p\_error**: Optional[float] = None
* Global error probability for the whole circuit. If set, the whole circuit will have the probability of non-exact result smaller than the set value. See [Exactness](../getting-started/exactness.md) to learn more.
* **single_precision**: bool = True
* Whether to use single precision for the whole circuit.
* **jit**: bool = False
* Whether to use JIT compilation.
* **loop\_parallelize**: bool = True
* Whether to enable loop parallelization in the compiler.
* **dataflow\_parallelize**: bool = False
* Whether to enable dataflow parallelization in the compiler.
* **auto\_parallelize**: bool = False
* Whether to enable auto parallelization in the compiler.
* **enable\_unsafe\_features**: bool = False
* Whether to enable unsafe features.
* **use\_insecure\_key\_cache**: bool = False _(Unsafe)_
* Whether to use the insecure key cache.
* **insecure\_key\_cache\_location**: Optional\[Union\[Path, str]] = None
* Location of insecure key cache.

305
docs/howto/debug.md Normal file
View File

@@ -0,0 +1,305 @@
# Debug
In this section, you will learn how to debug the compilation process easily as well as how to get help in case you cannot resolve your issue.
## Debug Artifacts
**Concrete** has an artifact system to simplify the process of debugging issues.
### Automatic export.
In case of compilation failures, artifacts are exported automatically to the `.artifacts` directory under the working directory. Let's intentionally create a compilation failure to show what kinds of things are exported.
```python
def f(x):
return np.sin(x)
```
This function fails to compile because **Concrete** does not support floating-point outputs. When you try to compile it, an exception will be raised and the artifacts will be exported automatically. If you go the `.artifacts` directory under the working directory, you'll see the following files:
#### environment.txt
This file contains information about your setup (i.e., your operating system and python version).
```
Linux-5.12.13-arch1-2-x86_64-with-glibc2.29 #1 SMP PREEMPT Fri, 25 Jun 2021 22:56:51 +0000
Python 3.8.10
```
#### requirements.txt
This file contains information about python packages and their versions installed on your system.
```
astroid==2.15.0
attrs==22.2.0
auditwheel==5.3.0
...
wheel==0.40.0
wrapt==1.15.0
zipp==3.15.0
```
#### function.txt
This file contains information about the function you tried to compile.
```
def f(x):
return np.sin(x)
```
#### parameters.txt
This file contains information about the encryption status of the parameters of the function you tried to compile.
```
x :: encrypted
```
#### 1.initial.graph.txt
This file contains the textual representation of the initial computation graph right after tracing.
```
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
#### 2.final.graph.txt
This file contains the textual representation of the final computation graph right before MLIR conversion.
```
%0 = x # EncryptedScalar<uint3>
%1 = sin(%0) # EncryptedScalar<float64>
return %1
```
#### traceback.txt
This file contains information about the error you received.
```
Traceback (most recent call last):
File "/path/to/your/script.py", line 9, in <module>
circuit = f.compile(inputset)
File "/usr/local/lib/python3.10/site-packages/concrete/fhe/compilation/decorators.py", line 159, in compile
return self.compiler.compile(inputset, configuration, artifacts, **kwargs)
File "/usr/local/lib/python3.10/site-packages/concrete/fhe/compilation/compiler.py", line 437, in compile
mlir = GraphConverter.convert(self.graph)
File "/usr/local/lib/python3.10/site-packages/concrete/fhe/mlir/graph_converter.py", line 677, in convert
GraphConverter._check_graph_convertibility(graph)
File "/usr/local/lib/python3.10/site-packages/concrete/fhe/mlir/graph_converter.py", line 240, in _check_graph_convertibility
raise RuntimeError(message)
RuntimeError: Function you are trying to compile cannot be converted to MLIR
%0 = x # EncryptedScalar<uint3> ∈ [3, 5]
%1 = sin(%0) # EncryptedScalar<float64> ∈ [-0.958924, 0.14112]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only integer operations are supported
/path/to/your/script.py:6
return %1
```
### Manual export.
Manual exports are mostly used for visualization. Nonetheless, they can be very useful for demonstrations. Here is how to perform one:
```python
from concrete import fhe
import numpy as np
artifacts = fhe.DebugArtifacts("/tmp/custom/export/path")
@fhe.compiler({"x": "encrypted"})
def f(x):
return 127 - (50 * (np.sin(x) + 1)).astype(np.int64)
inputset = range(2 ** 3)
circuit = f.compile(inputset, artifacts=artifacts)
artifacts.export()
```
If you go to the `/tmp/custom/export/path` directory, you'll see the following files:
#### 1.initial.graph.txt
This file contains the textual representation of the initial computation graph right after tracing.
```
%0 = x # EncryptedScalar<uint1>
%1 = sin(%0) # EncryptedScalar<float64>
%2 = 1 # ClearScalar<uint1>
%3 = add(%1, %2) # EncryptedScalar<float64>
%4 = 50 # ClearScalar<uint6>
%5 = multiply(%4, %3) # EncryptedScalar<float64>
%6 = astype(%5, dtype=int_) # EncryptedScalar<uint1>
%7 = 127 # ClearScalar<uint7>
%8 = subtract(%7, %6) # EncryptedScalar<uint1>
return %8
```
#### 2.after-fusing.graph.txt
This file contains the textual representation of the intermediate computation graph after fusing.
```
%0 = x # EncryptedScalar<uint1>
%1 = subgraph(%0) # EncryptedScalar<uint1>
%2 = 127 # ClearScalar<uint7>
%3 = subtract(%2, %1) # EncryptedScalar<uint1>
return %3
Subgraphs:
%1 = subgraph(%0):
%0 = input # EncryptedScalar<uint1>
%1 = sin(%0) # EncryptedScalar<float64>
%2 = 1 # ClearScalar<uint1>
%3 = add(%1, %2) # EncryptedScalar<float64>
%4 = 50 # ClearScalar<uint6>
%5 = multiply(%4, %3) # EncryptedScalar<float64>
%6 = astype(%5, dtype=int_) # EncryptedScalar<uint1>
return %6
```
#### 3.final.graph.txt
This file contains the textual representation of the final computation graph right before MLIR conversion.
```
%0 = x # EncryptedScalar<uint3> ∈ [0, 7]
%1 = subgraph(%0) # EncryptedScalar<uint7> ∈ [2, 95]
%2 = 127 # ClearScalar<uint7> ∈ [127, 127]
%3 = subtract(%2, %1) # EncryptedScalar<uint7> ∈ [32, 125]
return %3
Subgraphs:
%1 = subgraph(%0):
%0 = input # EncryptedScalar<uint1>
%1 = sin(%0) # EncryptedScalar<float64>
%2 = 1 # ClearScalar<uint1>
%3 = add(%1, %2) # EncryptedScalar<float64>
%4 = 50 # ClearScalar<uint6>
%5 = multiply(%4, %3) # EncryptedScalar<float64>
%6 = astype(%5, dtype=int_) # EncryptedScalar<uint1>
return %6
```
#### mlir.txt
This file contains information about the MLIR of the function you compiled using the inputset you provided.
```
module {
func.func @main(%arg0: !FHE.eint<7>) -> !FHE.eint<7> {
%c127_i8 = arith.constant 127 : i8
%cst = arith.constant dense<"..."> : tensor<128xi64>
%0 = "FHE.apply_lookup_table"(%arg0, %cst) : (!FHE.eint<7>, tensor<128xi64>) -> !FHE.eint<7>
%1 = "FHE.sub_int_eint"(%c127_i8, %0) : (i8, !FHE.eint<7>) -> !FHE.eint<7>
return %1 : !FHE.eint<7>
}
}
```
#### client_parameters.json
This file contains information about the client parameters chosen by **Concrete**.
```
{
"bootstrapKeys": [
{
"baseLog": 22,
"glweDimension": 1,
"inputLweDimension": 908,
"inputSecretKeyID": 1,
"level": 1,
"outputSecretKeyID": 0,
"polynomialSize": 8192,
"variance": 4.70197740328915e-38
}
],
"functionName": "main",
"inputs": [
{
"encryption": {
"encoding": {
"isSigned": false,
"precision": 7
},
"secretKeyID": 0,
"variance": 4.70197740328915e-38
},
"shape": {
"dimensions": [],
"sign": false,
"size": 0,
"width": 7
}
}
],
"keyswitchKeys": [
{
"baseLog": 3,
"inputSecretKeyID": 0,
"level": 6,
"outputSecretKeyID": 1,
"variance": 1.7944329123150665e-13
}
],
"outputs": [
{
"encryption": {
"encoding": {
"isSigned": false,
"precision": 7
},
"secretKeyID": 0,
"variance": 4.70197740328915e-38
},
"shape": {
"dimensions": [],
"sign": false,
"size": 0,
"width": 7
}
}
],
"packingKeyswitchKeys": [],
"secretKeys": [
{
"dimension": 8192
},
{
"dimension": 908
}
]
}
```
## Asking the community
You can seek help with your issue by asking a question directly in the [community forum](https://community.zama.ai/).
## Submitting an issue
If you cannot find a solution in the community forum, or you found a bug in the library, you could create an issue in our GitHub repository.
In case of a bug:
* try to minimize randomness
* try to minimize your function as much as possible while keeping the bug - this will help to fix the bug faster
* try to include your inputset in the issue
* try to include reproduction steps in the issue
* try to include debug artifacts in the issue
In case of a feature request:
* try to give a minimal example of the desired behavior
* try to explain your use case

132
docs/howto/deploy.md Normal file
View File

@@ -0,0 +1,132 @@
# Deploy
After developing your circuit, you may want to deploy it. However, sharing the details of your circuit with every client might not be desirable. Further, you might want to perform the computation in dedicated servers. In this case, you can use the `Client` and `Server` features of **Concrete**.
## Development of the circuit
You can develop your circuit like we've discussed in the previous chapters. Here is a simple example:
<!--pytest-codeblocks:skip-->
```python
from concrete import fhe
@fhe.compiler({"x": "encrypted"})
def function(x):
return x + 42
inputset = range(10)
circuit = function.compile(inputset)
```
Once you have your circuit, you can save everything the server needs like so:
<!--pytest-codeblocks:skip-->
```python
circuit.server.save("server.zip")
```
All you need to do now is to send `server.zip` to your computation server.
## Setting up a server
You can load the `server.zip` you get from the development machine as follows:
<!--pytest-codeblocks:skip-->
```python
from concrete import fhe
server = fhe.Server.load("server.zip")
```
At this point, you will need to wait for requests from clients. The first likely request is for `ClientSpecs`.
Clients need `ClientSpecs` to generate keys and request computation. You can serialize `ClientSpecs` like so:
<!--pytest-codeblocks:skip-->
```python
serialized_client_specs: str = server.client_specs.serialize()
```
Then, you can send it to the clients requesting it.
## Setting up clients
After getting the serialized `ClientSpecs` from a server, you can create the client object like this:
<!--pytest-codeblocks:skip-->
```python
client_specs = fhe.ClientSpecs.deserialize(serialized_client_specs)
client = fhe.Client(client_specs)
```
## Generating keys (on the client)
Once you have the `Client` object, you can perform key generation:
<!--pytest-codeblocks:skip-->
```python
client.keygen()
```
This method generates encryption/decryption keys and evaluation keys.
The server requires evaluation keys linked to the encryption keys that you just generated. You can serialize your evaluation keys as shown below:
<!--pytest-codeblocks:skip-->
```python
serialized_evaluation_keys: bytes = client.evaluation_keys.serialize()
```
After serialization, you can send the evaluation keys to the server.
{% hint style="info" %}
Serialized evaluation keys are very big in size, so you may want to cache them on the server instead of sending them with each request.
{% endhint %}
## Encrypting inputs (on the client)
You are now ready to encrypt your inputs and request the server to perform the computation. You can do it like so:
<!--pytest-codeblocks:skip-->
```python
serialized_args: bytes = client.encrypt(7).serialize()
```
The only thing left to do is to send serialized args to the server.
## Performing computation (on the server)
Upon having the serialized evaluation keys and serialized arguments, you can deserialize them like so:
<!--pytest-codeblocks:skip-->
```python
deserialized_evaluation_keys = fhe.EvaluationKeys.deserialize(serialized_evaluation_keys)
deserialized_args = server.client_specs.deserialize_public_args(serialized_args)
```
And you can perform the computation as well:
<!--pytest-codeblocks:skip-->
```python
public_result = server.run(deserialized_args, deserialized_evaluation_keys)
serialized_public_result: bytes = public_result.serialize()
```
Finally, you can send the serialized public result back to the client, so they can decrypt it and get the result of the computation.
## Decrypting the result (on the client)
Once you have received the public result of the computation from the server, you can deserialize it:
<!--pytest-codeblocks:skip-->
```python
deserialized_public_result = client.specs.deserialize_public_result(serialized_public_result)
```
Finally, you can decrypt the result like so:
<!--pytest-codeblocks:skip-->
```python
result = client.decrypt(deserialized_public_result)
assert result == 49
```