mirror of
https://github.com/zama-ai/concrete.git
synced 2026-02-09 03:55:04 -05:00
@@ -4,23 +4,21 @@
|
||||
|
||||
<figure><img src="_static/zama_home_docs.png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
**Concrete Numpy** is an open-source library which simplifies the use of fully homomorphic encryption (FHE).
|
||||
**Concrete-Numpy** is an open-source library which simplifies the use of fully homomorphic encryption (FHE).
|
||||
|
||||
FHE is a powerful cryptographic tool, which allows computation to be performed directly on encrypted data without needing to decrypt it first.
|
||||
|
||||
With FHE, you can build services that preserve privacy for all users. FHE is also great against data breaches as everything is done on encrypted data. Even if the server is compromised, in the end no sensitive data is leaked.
|
||||
FHE is a powerful cryptographic tool, which allows computation to be performed directly on encrypted data without needing to decrypt it first. With FHE, you can build services that preserve privacy for all users. FHE is also great against data breaches as everything is done on encrypted data. Even if the server is compromised, in the end no sensitive data is leaked.
|
||||
|
||||
## Organization of this documentation
|
||||
|
||||
This documentation is split into several sections:
|
||||
|
||||
* **Getting started** section to give you the basics
|
||||
* **Tutorials** section to give you some essential examples on various features of the library
|
||||
* **How to** section to help you perform specific tasks
|
||||
* **Developer** section to explain the inner workings of the library and everything related to contributing to the project
|
||||
* **Getting Started** gives you the basics,
|
||||
* **Tutorials** gives you some essential examples on various features of the library,
|
||||
* **How to** helps you perform specific tasks,
|
||||
* and **Developer** explains the inner workings of the library and everything related to contributing to the project.
|
||||
|
||||
## Looking for support? Ask our team!
|
||||
|
||||
* Support forum: [https://community.zama.ai](https://community.zama.ai) (we answer in less than 24 hours).
|
||||
* Live discussion on the FHE.org discord server: [https://discord.fhe.org](https://discord.fhe.org) (inside the #**concrete** channel).
|
||||
* A question about Zama? You can write us on [Twitter](https://twitter.com/zama\_fhe) or send us an email at: **hello@zama.ai**
|
||||
* Do you have a question about Zama? You can write us on [Twitter](https://twitter.com/zama\_fhe) or send us an email at: **hello@zama.ai**
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
## Getting Started
|
||||
|
||||
* [Installation](getting-started/installing.md)
|
||||
* [Quick start](getting-started/quick\_start.md)
|
||||
* [Quick Start](getting-started/quick\_start.md)
|
||||
* [Compatibility](getting-started/compatibility.md)
|
||||
|
||||
## Tutorials
|
||||
|
||||
@@ -8,18 +8,18 @@ After transformations are applied, we need to determine the bounds (i.e., the mi
|
||||
|
||||
The final step is to transform the computation graph to equivalent `MLIR` code. How this is done will be explained in detail in its own chapter.
|
||||
|
||||
Once the MLIR is generated, we send it to the **Concrete Compiler**, and it completes the compilation process.
|
||||
Once the MLIR is generated, we send it to the **Concrete-Compiler**, and it completes the compilation process.
|
||||
|
||||
## Tracing
|
||||
|
||||
Given a Python function `f` such as this one,
|
||||
Given a Python function `f` such as this one:
|
||||
|
||||
```
|
||||
def f(x):
|
||||
return (2 * x) + 3
|
||||
```
|
||||
|
||||
the goal of tracing is to create the following computation graph without needing any change from the user.
|
||||
...the goal of tracing is to create the following computation graph without needing any change from the user.
|
||||
|
||||

|
||||
|
||||
@@ -49,7 +49,7 @@ Tracing is also responsible for indicating whether the values in the node would
|
||||
|
||||
The goal of topological transforms is to make more functions compilable.
|
||||
|
||||
With the current version of **Concrete Numpy**, floating point inputs and floating point outputs are not supported. However, if the floating points operations are intermediate operations, they can sometimes be fused into a single table lookup from integer to integer thanks to some specific transforms.
|
||||
With the current version of **Concrete-Numpy**, floating-point inputs and floating-point outputs are not supported. However, if the floating-point operations are intermediate operations, they can sometimes be fused into a single table lookup from integer to integer, thanks to some specific transforms.
|
||||
|
||||
Let's take a closer look at the transforms we can currently perform.
|
||||
|
||||
@@ -145,4 +145,4 @@ Assigned Data Types:
|
||||
|
||||
## MLIR conversion
|
||||
|
||||
The actual compilation will be done by the **Concrete Compiler**, which is expecting an MLIR input. The MLIR conversion goes from a computation graph to its MLIR equivalent. You can read more about it [here](mlir.md).
|
||||
The actual compilation will be done by the **Concrete-Compiler**, which is expecting an MLIR input. The MLIR conversion goes from a computation graph to its MLIR equivalent. You can read more about it [here](mlir.md).
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Contribute
|
||||
|
||||
{% hint style="info" %}
|
||||
There are two ways to contribute to **Concrete Numpy** or to **Concrete** tools in general:
|
||||
There are two ways to contribute to **Concrete-Numpy** or to **Concrete** tools in general:
|
||||
|
||||
* You can open issues to report bugs and typos and to suggest ideas.
|
||||
* You can ask to become an official contributor by emailing hello@zama.ai. Only approved contributors can send pull requests, so please make sure to get in touch before you do!
|
||||
* You can ask to become an official contributor by emailing hello@zama.ai. Only approved contributors can send pull requests (PRs), so please make sure to get in touch before you do!
|
||||
{% endhint %}
|
||||
|
||||
Now, let's go over some other important items that you need to know.
|
||||
@@ -17,7 +17,7 @@ We are using a consistent branch naming scheme, and you are expected to follow i
|
||||
git checkout -b {feat|fix|refactor|test|benchmark|doc|style|chore}/short-description
|
||||
```
|
||||
|
||||
...and some examples:
|
||||
...and here are some examples:
|
||||
|
||||
```shell
|
||||
git checkout -b feat/direct-tlu
|
||||
@@ -28,7 +28,7 @@ git checkout -b fix/tracing-indexing
|
||||
|
||||
### Conformance.
|
||||
|
||||
Each commit to **Concrete Numpy** should conform to the standards decided by the team. Conformance can be checked using the following command:
|
||||
Each commit to **Concrete-Numpy** should conform to the standards decided by the team. Conformance can be checked using the following command:
|
||||
|
||||
```shell
|
||||
make pcc
|
||||
@@ -96,4 +96,4 @@ git rebase --continue
|
||||
git push --force
|
||||
```
|
||||
|
||||
You can learn more about rebasing in [here](https://git-scm.com/docs/git-rebase).
|
||||
You can learn more about rebasing [here](https://git-scm.com/docs/git-rebase).
|
||||
|
||||
@@ -8,7 +8,7 @@ Before you start this section, go ahead and install Docker. You can follow [this
|
||||
|
||||
### Linux.
|
||||
|
||||
You can use xhost command:
|
||||
You can use this xhost command:
|
||||
|
||||
```shell
|
||||
xhost +localhost
|
||||
@@ -16,7 +16,7 @@ xhost +localhost
|
||||
|
||||
### macOS.
|
||||
|
||||
To use X forwarding on Mac OS:
|
||||
To use X forwarding on macOS:
|
||||
|
||||
* Install XQuartz
|
||||
* Open XQuartz.app application, make sure in the application parameters that `authorize network connections` are set (currently in the Security settings)
|
||||
@@ -30,7 +30,7 @@ X server should be all set for Docker in the regular terminal.
|
||||
|
||||
## Building
|
||||
|
||||
You can use the dedicated target in the Makefile to build the docker image:
|
||||
You can use the dedicated target in the makefile to build the docker image:
|
||||
|
||||
```shell
|
||||
make docker_build
|
||||
@@ -38,7 +38,7 @@ make docker_build
|
||||
|
||||
## Starting
|
||||
|
||||
You can use the dedicated target in the Makefile to start the docker session:
|
||||
You can use the dedicated target in the makefile to start the docker session:
|
||||
|
||||
```shell
|
||||
make docker_start
|
||||
|
||||
@@ -4,34 +4,34 @@ Fusing is the act of combining multiple nodes into a single node, which is conve
|
||||
|
||||
## How is it done?
|
||||
|
||||
Code related to fusing is in the `concrete/numpy/compilation/utils.py` file.
|
||||
Code related to fusing is in the `concrete/numpy/compilation/utils.py` file. Fusing can be performed using the `fuse` function.
|
||||
|
||||
Fusing can be performed using the `fuse` function. Within `fuse`:
|
||||
Within `fuse`:
|
||||
|
||||
1. We loop until there are no more subgraphs to fuse.
|
||||
2. Within each iteration:
|
||||
3. We find a subgraph to fuse.
|
||||
2. <mark style="background-color:yellow;">Within each iteration:</mark>
|
||||
2.1. We find a subgraph to fuse.
|
||||
|
||||
3.1. We search for a terminal node that is appropriate for fusing.
|
||||
2.2. We search for a terminal node that is appropriate for fusing.
|
||||
|
||||
3.2. We crawl backwards to find the closest integer nodes to this node.
|
||||
2.3. We crawl backwards to find the closest integer nodes to this node.
|
||||
|
||||
3.3. If there is a single node as such, we return the subgraph from this node to the terminal node.
|
||||
2.4. If there is a single node as such, we return the subgraph from this node to the terminal node.
|
||||
|
||||
3.4. Otherwise, we try to find the lowest common ancestor (lca) of this list of nodes.
|
||||
2.5. Otherwise, we try to find the lowest common ancestor (lca) of this list of nodes.
|
||||
|
||||
3.5. If lca doesn't exist, we say this particular terminal node is not fusable, and we go back to search for another subgraph.
|
||||
2.6. If an lca doesn't exist, we say this particular terminal node is not fusable, and we go back to search for another subgraph.
|
||||
|
||||
3.6. Otherwise, we use this lca as the input of the subgraph and continue with `subgraph` node creation below.
|
||||
4. We convert the subgraph into a `subgraph` node
|
||||
2.7. Otherwise, we use this lca as the input of the subgraph and continue with `subgraph` node creation below.
|
||||
|
||||
4.1. We check fusability status of the nodes of the subgraph in this step.
|
||||
5. We substitute the `subgraph` node to the original graph.
|
||||
2.8. We convert the subgraph into a `subgraph` node by checking fusability status of the nodes of the subgraph in this step.
|
||||
|
||||
2.10. We substitute the `subgraph` node to the original graph.
|
||||
|
||||
## Limitations
|
||||
|
||||
With the current implementation, we cannot fuse subgraphs that depend on multiple encrypted values where those values doesn't have a common lca (e.g., `np.round(np.sin(x) + np.cos(y))`).
|
||||
|
||||
{% hint style="info" %}
|
||||
[Kolmogorov–Arnold representation theorem](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold\_representation\_theorem) states that every multivariate continuous function can be represented as a superposition of continuous functions of one variable. Therefore, the case above could be handled in future versions of **Concrete Numpy**.
|
||||
[Kolmogorov–Arnold representation theorem](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold\_representation\_theorem) states that every multivariate continuous function can be represented as a superposition of continuous functions of one variable. Therefore, the case above could be handled in future versions of **Concrete-Numpy**.
|
||||
{% endhint %}
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# MLIR
|
||||
|
||||
The MLIR project is a sub-project of the LLVM project. It's designed to simplify building domain-specific compilers such as our **Concrete Compiler**.
|
||||
The MLIR project is a sub-project of the LLVM project. It's designed to simplify building domain-specific compilers such as our **Concrete-Compiler**.
|
||||
|
||||
**Concrete Compiler** accepts MLIR as an input and emits compiled assembly code for a target architecture.
|
||||
**Concrete-Compiler** accepts MLIR as an input and emits compiled assembly code for a target architecture.
|
||||
|
||||
**Concrete Numpy** performs the MLIR generation from the computation graph. Code related to this conversion is in the `concrete/numpy/mlir` folder.
|
||||
**Concrete-Numpy** performs the MLIR generation from the computation graph. Code related to this conversion is in the `concrete/numpy/mlir` folder.
|
||||
|
||||
The conversion can be performed using the `convert` method of the `GraphConverter` class.
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Project Setup
|
||||
|
||||
{% hint style="info" %}
|
||||
It is **strongly** recommended to use the development tool Docker. Though, you can set the project up on a bare Linux or macOS as long as you have the required dependencies. You can see the required dependencies in `Dockerfile.dev` under `docker` directory.
|
||||
It is **strongly** recommended to use the development tool Docker. However, you are able to set the project up on a bare Linux or macOS as long as you have the required dependencies. You can see the required dependencies in `Dockerfile.dev` under `docker` directory.
|
||||
{% endhint %}
|
||||
|
||||
## Installing `Python`
|
||||
|
||||
**Concrete Numpy** is a `Python` library, so `Python` should be installed to develop it. `v3.8` and `v3.9` are, currently, the only supported versions.
|
||||
**Concrete-Numpy** is a `Python` library, so `Python` should be installed to develop it. `v3.8` and `v3.9` are, currently, the only supported versions.
|
||||
|
||||
You probably have Python already, but in case you don't, or in case you have an unsupported version, you can google `how to install python 3.8` and follow one of the results.
|
||||
|
||||
@@ -34,7 +34,7 @@ In the following sections, be sure to use the proper `make` tool for your system
|
||||
|
||||
## Cloning the repository
|
||||
|
||||
Now, it's time to get the source code of **Concrete Numpy**.
|
||||
Now, it's time to get the source code of **Concrete-Numpy**.
|
||||
|
||||
Clone the git repository from GitHub using the protocol of your choice (ssh or https).
|
||||
|
||||
@@ -58,9 +58,7 @@ source .venv/bin/activate
|
||||
|
||||
## Syncing the environment
|
||||
|
||||
From time to time, new dependencies will be added to the project and old ones will be removed.
|
||||
|
||||
The command below will make sure the project has the proper environment, so run it regularly.
|
||||
From time to time, new dependencies will be added to the project and old ones will be removed.mThe command below will make sure the project has the proper environment, so run it regularly.
|
||||
|
||||
```shell
|
||||
make sync_env
|
||||
|
||||
@@ -5,13 +5,13 @@
|
||||
Some terms used throughout the project include:
|
||||
|
||||
* computation graph - a data structure to represent a computation. This is basically a directed acyclic graph in which nodes are either inputs, constants or operations on other nodes.
|
||||
* tracing - the technique that takes a Python function from the user and generates the corresponding computation graph in an easy to read format.
|
||||
* bounds - before a computation graph is converted to MLIR, we need to know which node will output which type (e.g., uint3 vs euint5). Computation graphs with different inputs must remember the minimum and maximum values for each node, which is what we call bounds, and use bounds to determine the appropriate type for each node
|
||||
* tracing - the technique that takes a Python function from the user and generates the corresponding computation graph in an easy-to-read format.
|
||||
* bounds - before a computation graph is converted to MLIR, we need to know which node will output which type (e.g., uint3 vs euint5). Computation graphs with different inputs must remember the minimum and maximum values for each node, which is what we call bounds, and use bounds to determine the appropriate type for each node.
|
||||
* circuit - the result of compilation. A circuit is made of the client and server components and has methods, everything from printing to evaluation.
|
||||
|
||||
## Module structure
|
||||
|
||||
In this section, we will discuss the module structure of **Concrete Numpy** briefly. You are encouraged to check individual `.py` files to learn more.
|
||||
In this section, we will briefly discuss the module structure of **Concrete-Numpy**. You are encouraged to check individual `.py` files to learn more.
|
||||
|
||||
* Concrete
|
||||
* Numpy
|
||||
|
||||
@@ -8,7 +8,7 @@ Here are the operations you can use inside the function you are compiling.
|
||||
Some of these operations are not supported between two encrypted values. A detailed error will be raised if you try to do something that is not supported.
|
||||
{% endhint %}
|
||||
|
||||
### Supported Python operators
|
||||
### Supported Python operators.
|
||||
|
||||
* [\_\_abs\_\_](https://docs.python.org/3/reference/datamodel.html#object.\_\_abs\_\_)
|
||||
* [\_\_add\_\_](https://docs.python.org/3/reference/datamodel.html#object.\_\_add\_\_)
|
||||
@@ -49,10 +49,8 @@ Some of these operations are not supported between two encrypted values. A detai
|
||||
* [\_\_truediv\_\_](https://docs.python.org/3/reference/datamodel.html#object.\_\_truediv\_\_)
|
||||
* [\_\_xor\_\_](https://docs.python.org/3/reference/datamodel.html#object.\_\_xor\_\_)
|
||||
|
||||
### Supported NumPy functions
|
||||
### Supported NumPy functions.
|
||||
|
||||
<!--- gen_supported_ufuncs.py: inject supported operations [BEGIN] -->
|
||||
<!--- do not edit, auto generated part by `python3 gen_supported_ufuncs.py` in docker -->
|
||||
* [np.absolute](https://numpy.org/doc/stable/reference/generated/numpy.absolute.html)
|
||||
* [np.add](https://numpy.org/doc/stable/reference/generated/numpy.add.html)
|
||||
* [np.arccos](https://numpy.org/doc/stable/reference/generated/numpy.arccos.html)
|
||||
@@ -147,9 +145,8 @@ Some of these operations are not supported between two encrypted values. A detai
|
||||
* [np.trunc](https://numpy.org/doc/stable/reference/generated/numpy.trunc.html)
|
||||
* [np.where](https://numpy.org/doc/stable/reference/generated/numpy.where.html)
|
||||
* [np.zeros\_like](https://numpy.org/doc/stable/reference/generated/numpy.zeros\_like.html)
|
||||
<!--- gen_supported_ufuncs.py: inject supported operations [END] -->
|
||||
|
||||
### Supported `ndarray` methods
|
||||
### Supported `ndarray` methods.
|
||||
|
||||
* [np.ndarray.astype](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.astype.html)
|
||||
* [np.ndarray.clip](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.clip.html)
|
||||
@@ -158,7 +155,7 @@ Some of these operations are not supported between two encrypted values. A detai
|
||||
* [np.ndarray.reshape](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.reshape.html)
|
||||
* [np.ndarray.transpose](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.transpose.html)
|
||||
|
||||
### Supported `ndarray` properties
|
||||
### Supported `ndarray` properties.
|
||||
|
||||
* [np.ndarray.shape](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.shape.html)
|
||||
* [np.ndarray.ndim](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.ndim.html)
|
||||
@@ -167,19 +164,19 @@ Some of these operations are not supported between two encrypted values. A detai
|
||||
|
||||
## Limitations
|
||||
|
||||
### Operational constraints
|
||||
### Operational constraints.
|
||||
|
||||
Some Python control flow statements are not supported. For example, you cannot have an `if` statement or a `while` statement for which the condition depends on an encrypted value. However, such statements are supported with constant values (e.g., `for i in range(SOME_CONSTANT)`, `if os.environ.get("SOME_FEATURE") == "ON":`).
|
||||
|
||||
Another constraint is that you cannot have floating-point inputs or floating-point outputs. You can have floating-point intermediate values as long as they can be converted to an integer Table Lookup (e.g., `(60 * np.sin(x)).astype(np.int64)`).
|
||||
|
||||
### Bit width constraints
|
||||
### Bit width constraints.
|
||||
|
||||
There is a limit on the bit width of encrypted values. We are constantly working on increasing this bit width. If you go above this limit, you will get an error.
|
||||
There is a limit on the bit width of encrypted values. We are constantly working on increasing this bit width. If you go above the limit, you will get an error.
|
||||
|
||||
### Computation constraints
|
||||
### Computation constraints.
|
||||
|
||||
One of the most common operations in **Concrete Numpy** is `Table Lookups` (TLUs). TLUs are performed with an FHE operation called `Programmable Bootstrapping` (PBS). PBSes have a certain probability of error, which, when triggered, result in inaccurate results.
|
||||
One of the most common operations in **Concrete-Numpy** is `Table Lookups` (TLUs). TLUs are performed with an FHE operation called `Programmable Bootstrapping` (PBS). PBSes have a certain probability of error, which, when triggered, result in inaccurate results.
|
||||
|
||||
Let's say you have the table:
|
||||
|
||||
@@ -187,7 +184,7 @@ Let's say you have the table:
|
||||
[1, 4, 9, 16, 25, 36, 49, 64]
|
||||
```
|
||||
|
||||
And you performed a table lookup using `4`, The result you should get is `16`, but because of the possibility of error, you can sometimes get `9` or `25`.
|
||||
And you performed a table lookup using `4`. The result you should get is `16`, but because of the possibility of error, you can sometimes get `9` or `25`.
|
||||
|
||||
{% hint style="info" %}
|
||||
The probability of this error can be configured through the `p_error` configuration option, which has the default value of `0.000063342483999973` (i.e., probability of success is `99.993`%). Keep in mind that changing it could affect compilation and key generation times.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Installation
|
||||
|
||||
**Concrete Numpy** is natively supported on Linux and macOS for Python 3.8 and 3.9, but if you have Docker support in your platform, you can use the docker image to use **Concrete Numpy**.
|
||||
**Concrete-Numpy** is natively supported on Linux and macOS for Python 3.8 and 3.9, but if you have Docker support in your platform, you can use the docker image to use **Concrete-Numpy**.
|
||||
|
||||
## Using PyPI
|
||||
|
||||
You can install **Concrete Numpy** from PyPI:
|
||||
You can install **Concrete-Numpy** from PyPI:
|
||||
|
||||
```shell
|
||||
pip install -U pip wheel setuptools
|
||||
@@ -17,15 +17,15 @@ Apple silicon users must use docker installation (explained below) as there is n
|
||||
|
||||
## Using Docker
|
||||
|
||||
You can also get the **Concrete Numpy** docker image:
|
||||
You can also get the **Concrete-Numpy** docker image:
|
||||
|
||||
```shell
|
||||
docker pull zamafhe/concrete-numpy:v0.9.0
|
||||
```
|
||||
|
||||
### Starting Jupyter server
|
||||
### Starting a Jupyter server.
|
||||
|
||||
By default, the entry point of the **Concrete Numpy** docker image is a jupyter server that you can access from your browser:
|
||||
By default, the entry point of the **Concrete-Numpy** docker image is a jupyter server that you can access from your browser:
|
||||
|
||||
```shell
|
||||
docker run --rm -it -p 8888:8888 zamafhe/concrete-numpy:v0.9.0
|
||||
@@ -37,7 +37,7 @@ To save notebooks on host, you can use a local volume:
|
||||
docker run --rm -it -p 8888:8888 -v /path/to/notebooks:/data zamafhe/concrete-numpy:v0.9.0
|
||||
```
|
||||
|
||||
### Starting Bash session
|
||||
### Starting a Bash session.
|
||||
|
||||
Alternatively, you can launch a Bash session:
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Quick Start
|
||||
|
||||
To compute on encrypted data, you first need to define the function that you want to compute, then compile it into a Concrete Numpy `Circuit`, which you can use to perform homomorphic evaluation.
|
||||
To compute on encrypted data, you first need to define the function that you want to compute, then compile it into a Concrete-Numpy `Circuit`, which you can use to perform homomorphic evaluation.
|
||||
|
||||
Here is the full example that we will walk through:
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Configure
|
||||
|
||||
Behavior of **Concrete Numpy** can be customized using `Configuration`s:
|
||||
The behavior of **Concrete-Numpy** can be customized using `Configuration`s:
|
||||
|
||||
```python
|
||||
import concrete.numpy as cnp
|
||||
@@ -47,21 +47,20 @@ circuit = f.compile(inputset, configuration=configuration, loop_parallelize=True
|
||||
```
|
||||
|
||||
{% hint style="info" %}
|
||||
Additional kwarg to `compile` function have higher precedence. So if you set an option in both `configuration` and in `compile` method, the value in the `compile` method will be used.
|
||||
Additional kwarg to `compile` function have higher precedence. So if you set an option in both `configuration` and in `compile` methods, the value in the `compile` method will be used.
|
||||
{% endhint %}
|
||||
|
||||
## Options
|
||||
|
||||
* **show_graph**: bool = False
|
||||
* **show\_graph**: bool = False
|
||||
* Whether to print computation graph during compilation.
|
||||
|
||||
* **show_mlir**: bool = False
|
||||
* **show\_mlir**: bool = False
|
||||
* Whether to print MLIR during compilation.
|
||||
|
||||
* **verbose**: bool = False
|
||||
* Whether to print computation graph and MLIR during compilation.
|
||||
|
||||
* **dump_artifacts_on_unexpected_failures**: bool = True
|
||||
* **dump\_artifacts\_on\_unexpected\_failures**: bool = True
|
||||
* Whether to export debugging artifacts automatically on compilation failures.
|
||||
|
||||
* **p_error**: Optional[float] = None
|
||||
@@ -73,23 +72,23 @@ Additional kwarg to `compile` function have higher precedence. So if you set an
|
||||
* **jit**: bool = False
|
||||
* Whether to use JIT compilation.
|
||||
|
||||
* **loop_parallelize**: bool = True
|
||||
* **loop\_parallelize**: bool = True
|
||||
* Whether to enable loop parallelization in the compiler.
|
||||
|
||||
* **dataflow_parallelize**: bool = False
|
||||
* **dataflow\_parallelize**: bool = False
|
||||
* Whether to enable dataflow parallelization in the compiler.
|
||||
|
||||
* **auto_parallelize**: bool = False
|
||||
* **auto\_parallelize**: bool = False
|
||||
* Whether to enable auto parallelization in the compiler.
|
||||
|
||||
* **enable_unsafe_features**: bool = False
|
||||
* Whether to enable unsage features.
|
||||
* **enable\_unsafe\_features**: bool = False
|
||||
* Whether to enable unsafe features.
|
||||
|
||||
* **virtual**: bool = False _(Unsafe)_
|
||||
* Whether to create a virtual circuit.
|
||||
|
||||
* **use_insecure_key_cache**: bool = False _(Unsafe)_
|
||||
* **use\_insecure\_key\_cache**: bool = False _(Unsafe)_
|
||||
* Whether to use the insecure key cache.
|
||||
|
||||
* **insecure_key_cache_location**: Optional[Union[Path, str]] = None
|
||||
* **insecure\_key\_cache\_location**: Optional\[Union\[Path, str]] = None
|
||||
* Location of insecure key cache.
|
||||
|
||||
@@ -4,19 +4,18 @@ In this section, you will learn how to debug the compilation process easily as w
|
||||
|
||||
## Debug Artifacts
|
||||
|
||||
**Concrete Numpy** has an artifact system to simplify the process of debugging issues.
|
||||
**Concrete-Numpy** has an artifact system to simplify the process of debugging issues.
|
||||
|
||||
### Automatic export.
|
||||
|
||||
In case of compilation failures, artifacts are exported automatically to the `.artifacts` directory under the working directory. Let's intentionally create a compilation failure to show what kinds of things are exported.
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
def f(x):
|
||||
return np.sin(x)
|
||||
```
|
||||
|
||||
This function fails to compile because **Concrete Numpy** does not support floating-point outputs. When you try to compile it, an exception will be raised and the artifacts will be exported automatically. If you go the `.artifacts` directory under the working directory, you'll see the following files:
|
||||
This function fails to compile because **Concrete-Numpy** does not support floating-point outputs. When you try to compile it, an exception will be raised and the artifacts will be exported automatically. If you go the `.artifacts` directory under the working directory, you'll see the following files:
|
||||
|
||||
#### environment.txt
|
||||
|
||||
@@ -60,7 +59,7 @@ x :: encrypted
|
||||
|
||||
#### 1.initial.graph.txt
|
||||
|
||||
This file contains textual representation of the initial computation graph right after tracing.
|
||||
This file contains the textual representation of the initial computation graph right after tracing.
|
||||
|
||||
```
|
||||
%0 = x # EncryptedScalar<uint3>
|
||||
@@ -76,7 +75,7 @@ This file contains the visual representation of the initial computation graph ri
|
||||
|
||||
#### 2.final.graph.txt
|
||||
|
||||
This file contains textual representation of the final computation graph right before MLIR conversion.
|
||||
This file contains the textual representation of the final computation graph right before MLIR conversion.
|
||||
|
||||
```
|
||||
%0 = x # EncryptedScalar<uint3>
|
||||
@@ -134,7 +133,7 @@ If you go to the `/tmp/custom/export/path` directory, you'll see the following f
|
||||
|
||||
#### 1.initial.graph.txt
|
||||
|
||||
This file contains textual representation of the initial computation graph right after tracing.
|
||||
This file contains the textual representation of the initial computation graph right after tracing.
|
||||
|
||||
```
|
||||
%0 = 127 # ClearScalar<uint7>
|
||||
@@ -157,7 +156,7 @@ This file contains the visual representation of the initial computation graph ri
|
||||
|
||||
#### 2.after-float-fuse-0.graph.txt
|
||||
|
||||
This file contains textual representation of the intermediate computation graph after fusing.
|
||||
This file contains the textual representation of the intermediate computation graph after fusing.
|
||||
|
||||
```
|
||||
%0 = 127 # ClearScalar<uint7>
|
||||
@@ -246,7 +245,7 @@ module {
|
||||
|
||||
## Asking the community
|
||||
|
||||
You can seek help with your issue by asking directly in the [community forum](https://community.zama.ai/).
|
||||
You can seek help with your issue by asking a question directly in the [community forum](https://community.zama.ai/).
|
||||
|
||||
## Submitting an issue
|
||||
|
||||
@@ -255,7 +254,7 @@ If you cannot find a solution in the community forum, or you found a bug in the
|
||||
In case of a bug:
|
||||
|
||||
* try to minimize randomness
|
||||
* try to minimize your function as much as possible while keeping the bug, this will help to fix the bug faster
|
||||
* try to minimize your function as much as possible while keeping the bug - this will help to fix the bug faster
|
||||
* try to include your inputset in the issue
|
||||
* try to include reproduction steps in the issue
|
||||
* try to include debug artifacts in the issue
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Deploy
|
||||
|
||||
After developing your circuit, you may want to deploy it. Sharing the details of your circuit with every client might not be desirable. Further, you might want to perform the computation in dedicated servers. In this case, you can use the `Client` and `Server` features of **Concrete Numpy**.
|
||||
After developing your circuit, you may want to deploy it. However, sharing the details of your circuit with every client might not be desirable. Further, you might want to perform the computation in dedicated servers. In this case, you can use the `Client` and `Server` features of **Concrete-Numpy**.
|
||||
|
||||
## Development of the circuit
|
||||
|
||||
@@ -29,7 +29,7 @@ All you need to do now is to send `server.zip` to your computation server.
|
||||
|
||||
## Setting up a server
|
||||
|
||||
You can load the `server.zip` you get from the development machine like so:
|
||||
You can load the `server.zip` you get from the development machine as follows:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
@@ -51,7 +51,7 @@ Then, you can send it to the clients requesting it.
|
||||
|
||||
## Setting up clients
|
||||
|
||||
After getting the serialized `ClientSpecs` from a server, you can create the client object like so:
|
||||
After getting the serialized `ClientSpecs` from a server, you can create the client object like this:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
@@ -61,7 +61,7 @@ client = cnp.Client(client_specs)
|
||||
|
||||
## Generating keys (on the client)
|
||||
|
||||
Once you have the `Client` object, you can perform key generation like so:
|
||||
Once you have the `Client` object, you can perform key generation:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
@@ -70,7 +70,7 @@ client.keygen()
|
||||
|
||||
This method generates encryption/decryption keys and evaluation keys.
|
||||
|
||||
The server requires evaluation keys linked to the encryption keys that you just generated. You can serialize your evaluation keys like so:
|
||||
The server requires evaluation keys linked to the encryption keys that you just generated. You can serialize your evaluation keys as shown below:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
@@ -104,7 +104,7 @@ unserialized_evaluation_keys = cnp.EvaluationKeys.unserialize(serialized_evaluat
|
||||
unserialized_args = server.client_specs.unserialize_public_args(serialized_args)
|
||||
```
|
||||
|
||||
And you can perform the computation like so:
|
||||
And you can perform the computation as well:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
@@ -116,7 +116,7 @@ Finally, you can send the serialized public result back to the client, so they c
|
||||
|
||||
## Decrypting the result (on the client)
|
||||
|
||||
Once you have received the public result of the computation from the server, you can unserialize it like so:
|
||||
Once you have received the public result of the computation from the server, you can unserialize it:
|
||||
|
||||
<!--pytest-codeblocks:skip-->
|
||||
```python
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Extensions
|
||||
|
||||
**Concrete Numpy** tries to support **NumPy** as much as possible, but due to some technical limitations, not everything can be supported. On top of that, there are some things **NumPy** lack, which are useful. In some of these situations, we provide extensions in **Concrete Numpy** to improve your experience.
|
||||
**Concrete-Numpy** tries to support **NumPy** as much as possible, but due to some technical limitations, not everything can be supported. On top of that, there are some things **NumPy** lack, which are useful. In some of these situations, we provide extensions in **Concrete-Numpy** to improve your experience.
|
||||
|
||||
## cnp.zero()
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Floating Points
|
||||
|
||||
**Concrete Numpy** partly supports floating points:
|
||||
**Concrete-Numpy** partly supports floating points:
|
||||
|
||||
* They cannot be inputs
|
||||
* They cannot be outputs
|
||||
@@ -8,9 +8,9 @@
|
||||
|
||||
## As intermediate values
|
||||
|
||||
**Concrete Compile**, which is used for compiling the circuit, doesn't support floating points at all. However, it supports table lookups. They take an integer and map it to another integer. It does not care how the lookup table is calculated. Further, the constraints of this operation are such that there should be a single integer input and it should result in a single integer output.
|
||||
**Concrete-Compile**, which is used for compiling the circuit, doesn't support floating points at all. However, it supports table lookups. They take an integer and map it to another integer. It does not care how the lookup table is calculated. Further, the constraints of this operation are such that there should be a single integer input and it should result in a single integer output.
|
||||
|
||||
As long as your floating point operations comply with those constraints, **Concrete Numpy** automatically converts your operations to a table lookup operation:
|
||||
As long as your floating point operations comply with those constraints, **Concrete-Numpy** automatically converts your operations to a table lookup operation:
|
||||
|
||||
```python
|
||||
import concrete.numpy as cnp
|
||||
@@ -31,7 +31,7 @@ for x in range(8):
|
||||
assert circuit.encrypt_run_decrypt(x) == f(x)
|
||||
```
|
||||
|
||||
In the example above, `a`, `b`, and `c` are all floating point intermediates. However, they are just used to calculate `d`, which is an integer and value of `d` dependent upon `x` , which is another integer. **Concrete Numpy** detects this and fuses all of those operations into a single table lookup from `x` to `d`.
|
||||
In the example above, `a`, `b`, and `c` are all floating point intermediates. However, they are just used to calculate `d`, which is an integer and value of `d` dependent upon `x` , which is another integer. **Concrete-Numpy** detects this and fuses all of those operations into a single table lookup from `x` to `d`.
|
||||
|
||||
This approach works for a variety of use cases, but it comes up short for some:
|
||||
|
||||
@@ -55,7 +55,7 @@ for x in range(8):
|
||||
assert circuit.encrypt_run_decrypt(x) == f(x)
|
||||
```
|
||||
|
||||
results in
|
||||
... results in:
|
||||
|
||||
```
|
||||
RuntimeError: Function you are trying to compile cannot be converted to MLIR
|
||||
@@ -76,4 +76,4 @@ RuntimeError: Function you are trying to compile cannot be converted to MLIR
|
||||
return %7
|
||||
```
|
||||
|
||||
The reason for that is that `d` no longer depends solely on `x`, it depends on `y` as well. Thus, **Concrete Numpy** cannot fuse these operations, so it raises an exception.
|
||||
The reason for this is that `d` no longer depends solely on `x`, it depends on `y` as well. **Concrete-Numpy** cannot fuse these operations, so it raises an exception instead.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Table Lookups
|
||||
|
||||
In this tutorial, we will review the ways to perform direct table lookups in **Concrete Numpy**.
|
||||
In this tutorial, we will review the ways to perform direct table lookups in **Concrete-Numpy**.
|
||||
|
||||
## Direct table lookup
|
||||
|
||||
**Concrete Numpy** provides a `LookupTable` class for you to create your own tables and apply them in your circuits.
|
||||
**Concrete-Numpy** provides a `LookupTable` class for you to create your own tables and apply them in your circuits.
|
||||
|
||||
{% hint style="info" %}
|
||||
`LookupTable`s can have any number of elements. Let's call them **N**. As long as the lookup variable is in range \[-**N**, **N**), table lookup is valid.
|
||||
@@ -144,7 +144,7 @@ In this example, we applied a `squared` table to the first column and a `cubed`
|
||||
|
||||
## Fused table lookup
|
||||
|
||||
**Concrete Numpy** tries to fuse some operations into table lookups automatically, so you don't need to create the lookup tables manually:
|
||||
**Concrete-Numpy** tries to fuse some operations into table lookups automatically, so you don't need to create the lookup tables manually:
|
||||
|
||||
```python
|
||||
import concrete.numpy as cnp
|
||||
@@ -162,17 +162,17 @@ for x in range(8):
|
||||
```
|
||||
|
||||
{% hint style="info" %}
|
||||
All lookup tables need to be from integers to integers. So, without `.astype(np.int64)`, **Concrete Numpy** will not be able to fuse.
|
||||
All lookup tables need to be from integers to integers. So, without `.astype(np.int64)`, **Concrete-Numpy** will not be able to fuse.
|
||||
{% endhint %}
|
||||
|
||||
The function is first traced into:
|
||||
|
||||

|
||||
|
||||
Then, **Concrete Numpy** fuses appropriate nodes:
|
||||
Then, **Concrete-Numpy** fuses appropriate nodes:
|
||||
|
||||

|
||||
|
||||
{% hint style="info" %}
|
||||
Fusing makes the code more readable and easier to modify. So try to utilize it over manual `LookupTable`s as much as possible.
|
||||
Fusing makes the code more readable and easier to modify, so try to utilize it over manual `LookupTable`s as much as possible.
|
||||
{% endhint %}
|
||||
|
||||
Reference in New Issue
Block a user