chore: update the doc to have clearer uses of Concrete Numpy etc

refs #1288
This commit is contained in:
Benoit Chevallier-Mames
2022-01-05 18:05:04 +01:00
committed by Benoit Chevallier
parent a835d25e15
commit 721bc06eb7
18 changed files with 47 additions and 53 deletions

View File

@@ -2,7 +2,7 @@
## What is **concrete-numpy**?
**concrete-numpy** is the python API of the **Concrete** framework for developing homomorphic applications.
**concrete-numpy** is a convenient python package, made on top of **Concrete compiler** and **Concrete library**, for developing homomorphic applications.
One of its essential functionalities is to transform Python functions to their `MLIR` equivalent.
Unfortunately, not all python functions can be converted due to the limits of current product (we are in the alpha stage), or sometimes due to inherent restrictions of FHE itself.
However, you can already build interesting and impressing use cases, and more will be available in further versions of the framework.
@@ -47,7 +47,7 @@ There are several approaches to compute bounds, and they will be discussed in th
The final step is to transform the operation graph to equivalent `MLIR` code.
How this is done will be explained in detail in its own chapter.
Once the MLIR is prepared, the rest of the stack, which you can learn more about [here](http://docs.zama.ai/concrete/), takes over and completes the compilation process.
Once the MLIR is prepared, the rest of the stack, which you can learn more about [here](http://docs.zama.ai/), takes over and completes the compilation process.
Here is the visual representation of the pipeline:
@@ -117,7 +117,7 @@ Tracing is also responsible for indicating whether the values in the node would
The goal of topological transforms is to make more functions compilable.
With the current version of **Concrete**,floating point inputs and floating point outputs are not supported.
With the current version of **Concrete Numpy**, floating point inputs and floating point outputs are not supported.
However, if the floating points operations are intermediate operations, they can sometimes be fused into a single table lookup from integer to integer thanks to some specific transforms.
Let's take a closer look at the transforms we can currently perform.

View File

@@ -35,7 +35,7 @@ The simplified graph of operations with the float subgraph condensed in a `Gener
![](../../_static/float_fusing_example/after.png)
## How is it done in **Concrete**?
## How is it done in **Concrete Numpy**?
The first step consists in detecting where we go from floating point computation back to integers. This allows to identify the potential terminal node of the float subgraph we are going to fuse.
@@ -83,4 +83,4 @@ def non_fusable(x, y):
return add_int
```
From `add_int` you will find two `Add` nodes going from int to float (`x_1` and `y_1`) which we cannot represent with a single input table look-up. KolmogorovArnold representation theorem states that every multivariate continuous function can be represented as a superposition of continuous functions of one variable ([from Wikipedia](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold_representation_theorem)), so the above case could be handled in future versions of **Concrete**.
From `add_int` you will find two `Add` nodes going from int to float (`x_1` and `y_1`) which we cannot represent with a single input table look-up. KolmogorovArnold representation theorem states that every multivariate continuous function can be represented as a superposition of continuous functions of one variable ([from Wikipedia](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold_representation_theorem)), so the above case could be handled in future versions of **Concrete** tools.

View File

@@ -2,7 +2,7 @@
# Contributing
```{important}
There are two ways to contribute to **concrete-numpy**:
There are two ways to contribute to **concrete-numpy** or to **Concrete** tools in general:
- you can open issues to report bugs, typos and suggest ideas
- you can ask to become an official contributor by emailing hello@zama.ai. Only approved contributors can send pull requests, so please make sure to get in touch before you do!
```

View File

@@ -33,7 +33,7 @@ Install Xming and use Xlaunch:
## Logging in and building the image
Docker image of **Concrete** is based on another docker image provided by the compiler team. Once you have access to this repository you should be able to launch the commands to build the dev docker image with `make docker_build`.
Docker image of **Concrete-Numpy** is based on another docker image provided by the compiler team. Once you have access to this repository you should be able to launch the commands to build the dev docker image with `make docker_build`.
Upon joining to the team, you need to log in using the following command:

View File

@@ -11,11 +11,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Trees are a popular class of algorithm in Machine Learning. In this notebook we build a simple Decision Tree Classifier using `scikit-learn` to show that they can be executed homomorphically using the Concrete Numpy.\n",
"Trees are a popular class of algorithm in Machine Learning. In this notebook we build a simple Decision Tree Classifier using `scikit-learn` to show that they can be executed homomorphically using Concrete Numpy.\n",
"\n",
"State of the art classifiers are generally a bit more complex than a single decision tree, here we wanted to demonstrate FHE decision trees so results may not compete with the best models out there!\n",
"\n",
"Converting a tree working over quantized data to its FHE equivalent takes only a few lines of code thanks to the Concrete Numpy.\n",
"Converting a tree working over quantized data to its FHE equivalent takes only a few lines of code thanks to Concrete Numpy.\n",
"\n",
"Let's dive in!"
]

View File

@@ -7,7 +7,7 @@
"source": [
"# Generalized Linear Model : Poisson Regression\n",
"\n",
"This tutorial shows how to train several Generalized Linear Models (GLM) with scikit-learn, quantize them and run them in FHE using the Concrete Numpy. We make use of strong quantization to insure the accumulator of the linear part does not overflow when computing in FHE (7-bit accumulator). We show that conversion to FHE does not degrade performance with respect to the quantized model working on values in the clear."
"This tutorial shows how to train several Generalized Linear Models (GLM) with scikit-learn, quantize them and run them in FHE using Concrete Numpy. We make use of strong quantization to insure the accumulator of the linear part does not overflow when computing in FHE (7-bit accumulator). We show that conversion to FHE does not degrade performance with respect to the quantized model working on values in the clear."
]
},
{

View File

@@ -10,7 +10,7 @@ import concrete.numpy as hnp
## Defining a function to compile
You need to have a python function that follows the [limits](../explanation/fhe_and_framework_limits.md) of the **Concrete Numpy**. Here is a simple example:
You need to have a python function that follows the [limits](../explanation/fhe_and_framework_limits.md) of **Concrete Numpy**. Here is a simple example:
<!--pytest-codeblocks:cont-->
```python

View File

@@ -2,7 +2,7 @@
## Python package
To install **Concrete** from PyPi, run the following:
To install **Concrete Numpy** from PyPi, run the following:
```shell
pip install concrete-numpy
@@ -51,7 +51,7 @@ docker run --rm -it -p 8888:8888 zamafhe/concrete-numpy:v0.2.0
docker run --rm -it -p 8888:8888 -v /host/path:/data zamafhe/concrete-numpy:v0.2.0
```
This will launch a **Concrete** enabled jupyter server in the docker, that you can access from your browser.
This will launch a **Concrete Numpy** enabled jupyter server in the docker, that you can access from your browser.
Alternatively, you can just open a shell in the docker with or without volumes:

View File

@@ -1,36 +1,30 @@
# What is **Concrete**?
# What is **Concrete Numpy**?
## Introduction
**Concrete Numpy**, or **Concrete** for short, is an open-source framework which aims to simplify the use of so-called fully homomorphic encryption (FHE) for data scientists.
**Concrete Numpy**, or **Concrete** for short, is an open-source set of tools which aims to simplify the use of so-called fully homomorphic encryption (FHE) for data scientists.
FHE is a powerful cryptographic tool, which allows servers to perform computations directly on encrypted data without needing to decrypt first. With FHE, privacy is at the center, and you can build services which ensure full privacy of the user and are the perfect equivalent of their unsecure counterpart.
FHE is also a killer feature regarding data breaches: as anything done on the server is done over encrypted data, even if the server is compromised, there is in the end no leak of useful data.
With Concrete Numpy, data scientists can implement machine learning models using a [subset of numpy](../howto/numpy_support.md) that compile to FHE. They will be able to train models with popular machine learning libraries and then convert the prediction functions of these models, that they write in numpy, to FHE.
With **Concrete Numpy**, data scientists can implement machine learning models using a [subset of numpy](../howto/numpy_support.md) that compile to FHE. They will be able to train models with popular machine learning libraries and then convert the prediction functions of these models, that they write in numpy, to FHE.
**Concrete** is made of several parts:
- a library, called concrete-lib, which contains the core cryptographic APIs for computing with FHE;
- a compiler, called concrete-compiler, which allows you to turn an MLIR program into an FHE program, on the top of concrete-lib;
- and some frontends, which convert different langages to MLIR, to finally be compiled.
**Concrete Numpy** is made of several parts:
- an entry API, which is the main function of the so-called **Concrete frontend**, which takes programs made from a subset of numpy, and converts them to an FHE program
- the **Concrete compiler**, which is called by the frontend, which allows you to turn an MLIR program into an FHE program, on the top of **Concrete Library**, which contains the core cryptographic APIs for computing with FHE;
- some ML tools, in an early version, allowing for example to turn some torch programs into numpy, and then to use the main API stack to finally get an FHE program.
```{important}
In the first version of Concrete, there is a single frontend, called homomorphic numpy (or hnp), which is the equivalent of numpy. With our toolchain, a data scientist can convert a numpy program into an FHE program, without any in-depth knowledge of cryptography.
```
```{note}
On top of the numpy frontend, we are adding an alpha-version of a torch compiler, which basically transforms a subset of torch modules into numpy, and then uses numpy frontend and the compiler. This is an early version of a more stable torch compiler which will be released later in 2022.
```
In a further release, **Concrete Numpy** will be divided into a **Concrete Framework** package, containing the compiler, the core lib and the frontend(s), and in a **Concrete ML**, which will contain ML tools, made on top of the **Concrete Framework**. Names of these packages are succeptible to change.
## Organization of the documentation
Basically, we have divided our documentation into several parts:
- one about basic elements, notably a description of the installation, that you are currently reading
- one dedicated to _users_ of **Concrete**, with tutorials, how-tos and deeper explanations
- one dedicated to _users_ of **Concrete Numpy**, with tutorials, how-tos and deeper explanations
- one detailing the APIs of the different functions of the frontend, directly done by parsing its source code
- and finally, one dedicated to _developers_ of **Concrete**, who could be internal or external contributors to the framework
- and finally, one dedicated to _developers_ of **Concrete Numpy**, who could be internal or external contributors to the framework
## A work in progress

View File

@@ -10,11 +10,11 @@ However, one still has to consider that FHE is slow, as compared to the vanilla
### Multiplying by constants
In the scheme used in the **Concrete Numpy**, namely [TFHE](https://tfhe.github.io/tfhe/), multiplications by constants is only defined for integer constants. Notably, one can't multiply by floats. As float multiplication is very usual in the data science (think of weights of dense layers, for example), this could be a problem, but quantization is at our rescue. See [this](quantization.md) section for more details.
In the scheme used in **Concrete Numpy**, namely [TFHE](https://tfhe.github.io/tfhe/), multiplications by constants is only defined for integer constants. Notably, one can't multiply by floats. As float multiplication is very usual in the data science (think of weights of dense layers, for example), this could be a problem, but quantization is at our rescue. See [this](quantization.md) section for more details.
### Achieving computations of not-linear functions
For most FHE scheme but TFHE, the application of a non-linear function is complicated and slow, if not impossible. Typically, this is a blocker, since activation functions _are_ non-linear. However, in the **Concrete Numpy**, we use an operation called _programmable bootstrapping_ (described in this [white paper](https://whitepaper.zama.ai)), which allows to apply any table lookup: by quantizing the non-linear function, any function can thus be replaced.
For most FHE scheme but TFHE, the application of a non-linear function is complicated and slow, if not impossible. Typically, this is a blocker, since activation functions _are_ non-linear. However, in the **Concrete Numpy** package, we use an operation called _programmable bootstrapping_ (described in this [white paper](https://whitepaper.zama.ai)), which allows to apply any table lookup: by quantizing the non-linear function, any function can thus be replaced.
## Limits of this project
@@ -22,7 +22,7 @@ Since this is an early version of the product, not everything is done, to say th
### Currently executing locally
As of today, the execution of the FHE program is done locally. Notably, in the current version, there is no client (on which we encrypt the private data, or decrypt the returned result) or server (on which the computation is done completely over encrypted data), but a single host. As explained in [this section](future_features.md), this limit will be removed in the next version, such that the **Concrete Numpy** can be used in production.
As of today, the execution of the FHE program is done locally. Notably, in the current version, there is no client (on which we encrypt the private data, or decrypt the returned result) or server (on which the computation is done completely over encrypted data), but a single host. As explained in [this section](future_features.md), this limit will be removed in the next version, such that **Concrete Numpy** can be used in production.
### Currently slow

View File

@@ -1,10 +1,10 @@
# Future Features
As explained in [this section](fhe_and_framework_limits.md#limits-of-this-project), the **Concrete Numpy**
As explained in [this section](fhe_and_framework_limits.md#limits-of-this-project), the **Concrete Numpy** package
is currently in a preliminary version, and quite constrained in term of functionalities. However, the good
news is that we are going to release new versions regularly, where a lot of functionalities will be added progressively.
In this page, we briefly list what the plans for next versions of the **Concrete Numpy** are:
In this page, we briefly list what the plans for next versions of **Concrete Numpy** are:
- **better performance**: further versions will contain improved versions of the **Concrete Library**, with faster
execution; also, the **Concrete Compiler** will be improved, to have faster local execution (with multi-threading
for example) and faster production execution (with distribution over a set of machines or use of hardware accelerations)

View File

@@ -18,7 +18,7 @@ The basic idea of quantization is to take a range of values represented by a _la
Let's first define some notations. Let $ [\alpha, \beta ] $ be the range of our value to quantize where $ \alpha $ is the minimum and $ \beta $ is the maximum.
To quantize a range with floating point values (in $ \mathbb{R} $) to unsigned integer values (in $ \mathbb{N} $), we first need to choose the data type that is going to be used. **ConcreteLib**, the library used in the **Concrete Numpy**, is currently limited to 7 bits unsigned integers, so we'll use that for the example. Knowing that, for a value in the range $ [\alpha, \beta ] $, we can compute the `scale` $ S $ of the quantization:
To quantize a range with floating point values (in $ \mathbb{R} $) to unsigned integer values (in $ \mathbb{N} $), we first need to choose the data type that is going to be used. **Concrete Library**, the library used in the **Concrete Numpy**, is currently limited to 7 bits unsigned integers, so we'll use that for the example. Knowing that, for a value in the range $ [\alpha, \beta ] $, we can compute the `scale` $ S $ of the quantization:
$$ S = \frac{\beta - \alpha}{2^n - 1} $$
@@ -36,9 +36,9 @@ Regarding quantization and FHE compilation, it is important to understand the di
1. the quantization is done before the compilation; notably, the quantization is completely controlled by the user, and can be done by any means, including by using third party frameworks
2. the quantization is done during the compilation (inside our framework), with much less control by the user.
For the moment, only the second method is available in Concrete Numpy, but we plan to have the first method available in a further release, since it should give more freedom and better results to the user.
For the moment, only the second method is available in **Concrete Numpy**, but we plan to have the first method available in a further release, since it should give more freedom and better results to the user.
We detail the use of quantization within Concrete Numpy in [here](../howto/use_quantization.md).
We detail the use of quantization within **Concrete Numpy** in [here](../howto/use_quantization.md).
## Resources

View File

@@ -1,6 +1,6 @@
# Compiling a Torch Model
Concrete Numpy allows to compile a torch model to its FHE counterpart.
**Concrete Numpy** allows to compile a torch model to its FHE counterpart.
A simple command can compile a torch model to its FHE counterpart. This process executes most of the concepts described in the documentation on [how to use quantization](use_quantization.md) and triggers the compilation to be able to run the model over homomorphically encrypted data.

View File

@@ -8,7 +8,7 @@ See [here](../basics/intro.md). Also, you can have a look to Zama's [website](ht
Our mission at Zama is to protect peoples privacy by preventing data breaches and unethical surveillance.
Following a recent breakthrough in fully homomorphic encryption, we are building a deep learning framework that enables fast and accurate inference over encrypted data, with minimal performance overhead, no changes to the network architecture, and no retraining necessary. Zama is open-source by design, as we believe privacy-enabling technologies should benefit the widest possible community of developers and researchers. If you are interested in the details, be sure to read our licensing agreement in the root of the repository. In a nutshell, if you plan to use the Concrete Numpy for non-commercial purposes, we welcome you and want to hear about all the exciting things you do with FHE!
Following a recent breakthrough in fully homomorphic encryption, we are building a deep learning framework that enables fast and accurate inference over encrypted data, with minimal performance overhead, no changes to the network architecture, and no retraining necessary. Zama is open-source by design, as we believe privacy-enabling technologies should benefit the widest possible community of developers and researchers. If you are interested in the details, be sure to read our licensing agreement in the root of the repository. In a nutshell, if you plan to use **Concrete Numpy** for non-commercial purposes, we welcome you and want to hear about all the exciting things you do with FHE!
## Can I use it freely?

View File

@@ -1,6 +1,6 @@
# Numpy Support
In this section, we list the operations which are supported currently in the **Concrete Numpy**. Please have a look to numpy [documentation](https://numpy.org/doc/stable/user/index.html) to know what these operations are about.
In this section, we list the operations which are supported currently in **Concrete Numpy**. Please have a look to numpy [documentation](https://numpy.org/doc/stable/user/index.html) to know what these operations are about.
## Unary operations

View File

@@ -77,9 +77,9 @@ The end result has a granularity/imprecision linked to the data types used and f
Recent quantization literature often takes a few shortcuts to reach performance similar to those achieved by floating point models. A common one is that the input is left in floating point. This is also true for the first and last layers which have more impact on the resulting model accuracy than hidden layers.
But, in Concrete Numpy the inputs, weights and the accumulator must remain on a maximum of 7 bits.
But, in **Concrete Numpy**, the inputs, weights and the accumulator must remain on a maximum of 7 bits.
Thus, in Concrete Numpy we also quantize the input data and network output activations in the same way as the rest of the network: everything is quantized to a specific number of bits. It turns out, that the number of bits used for the input or the output of any activation function is crucial to comply with the constraint on accumulator width.
Thus, in **Concrete Numpy**, we also quantize the input data and network output activations in the same way as the rest of the network: everything is quantized to a specific number of bits. It turns out, that the number of bits used for the input or the output of any activation function is crucial to comply with the constraint on accumulator width.
The core operation in neural networks is essentially matrix multiplications (matmul). This operation must be done such that the maximum value of its result requires at most 7 bits of precision.
@@ -91,6 +91,6 @@ where $ n_{\mathsf{max}} = 7 $ is the maximum precision allowed. For example, if
Above $ \Omega $ dimensions in the input and weights, the risk of overflow increases quickly. It may happen that for some distributions of weights and values the computation does not overflow, but the risk increases rapidly with the number of dimensions.
Currently, Concrete Numpy pre-computes the number of bits needed for the computation depending on the input set calibration data and does not allow the overflow[^1] to happen.
Currently, **Concrete Numpy** pre-computes the number of bits needed for the computation depending on the input set calibration data and does not allow the overflow[^1] to happen.
[^1]: [Integer overflow](https://en.wikipedia.org/wiki/Integer_overflow)

View File

@@ -1,13 +1,13 @@
# Using Quantization in Concrete Numpy
# Using Quantization in **Concrete Numpy**
In this section we detail some usage of [quantization](../explanation/quantization.md) as implemented in Concrete.
In this section we detail some usage of [quantization](../explanation/quantization.md) as implemented in **Concrete**.
## Quantization Basics
Concrete Numpy implements some basic concepts of quantization. The very basic purpose of it is to convert floating point values to integers. We can apply such conversion using `QuantizedArray` available in `concrete.quantization`.
**Concrete Numpy** implements some basic concepts of quantization. The very basic purpose of it is to convert floating point values to integers. We can apply such conversion using `QuantizedArray` available in `concrete.quantization`.
`QuantizedArray` takes 2 arguments:
- `n_bits` that defines the precision of the quantization. Currently, `n_bits` is limited to 7, due to some ConcreteLib limits.
- `n_bits` that defines the precision of the quantization. Currently, `n_bits` is limited to 7, due to some **Concrete Library** limits.
- `values` that will be converted to integers
```python
@@ -36,14 +36,14 @@ q_A.dequant()
Neural networks are implemented with a diverse set of operations, such as convolution, linear transformations, activation functions and element-wise operations. When working with quantized values, these operations can not be carried out the same way as for floating point values. With quantization it is necessary to re-scale the input and output values of each operation to fit in the quantization domain.
Re-scaling raw input values to the quantized domain implies that we need to make use of floating point operations. In the FHE setting where we only work with integers, this could be a problem, but luckily, the FHE implementation behind the Concrete framework provides a workaround. We essentially make use of a [table lookup](../tutorial/table_lookup.md) which is later translated into a [PBS](https://whitepaper.zama.ai).
Re-scaling raw input values to the quantized domain implies that we need to make use of floating point operations. In the FHE setting where we only work with integers, this could be a problem, but luckily, the FHE implementation behind **Concrete Numpy** provides a workaround. We essentially make use of a [table lookup](../tutorial/table_lookup.md) which is later translated into a [PBS](https://whitepaper.zama.ai).
Of course, having a PBS for every quantized addition isn't recommended for computational cost reasons. Also, Concrete Numpy allows PBS only for univariate operations (i.e. matrix multiplication can't be done in a PBS). Therefore, our quantized modules split the computation of floating point values and unsigned integers as it is currently done in `concrete.quantization.QuantizedLinear`.
Of course, having a PBS for every quantized addition isn't recommended for computational cost reasons. Also, **Concrete Numpy** allows PBS only for univariate operations (i.e. matrix multiplication can't be done in a PBS). Therefore, our quantized modules split the computation of floating point values and unsigned integers as it is currently done in `concrete.quantization.QuantizedLinear`.
The above operations are all implemented in Concrete Numpy and transparent to the user via our Quantized Modules.
The above operations are all implemented in **Concrete Numpy** and transparent to the user via our Quantized Modules.
Concrete Numpy allows you to convert Numpy operations to their FHE counterparts. This essentially opens the door to any python computing framework such as [PyTorch](https://pytorch.org/). Concrete Numpy implements a Torch to Numpy converter that makes it easy for the user to use a torch model.
**Concrete Numpy** allows you to convert numpy operations to their FHE counterparts. This essentially opens the door to any python computing framework such as [PyTorch](https://pytorch.org/). **Concrete Numpy** implements a Torch to Numpy converter that makes it easy for the user to use a torch model.
First we define a model:
@@ -141,7 +141,7 @@ Do not reuse a layer or an activation multiple times in the forward (i.e. self.s
It is now possible to compile the `quantized_numpy_module`. Details on how to compile the model are available in the [torch compilation documentation](compiling_torch_model.md).
## Building your own QuantizedModule
Concrete Numpy also offers the possibility to build your own models and use them in the FHE settings. The `QuantizedModule` is a very simple abstraction that allows to create any model using the available operators:
**Concrete Numpy** also offers the possibility to build your own models and use them in the FHE settings. The `QuantizedModule` is a very simple abstraction that allows to create any model using the available operators:
- QuantizedSigmoid, the quantized version of `nn.Sigmoid`
- QuantizedLinear, the quantized version of `nn.Linear`

View File

@@ -29,7 +29,7 @@ One can look to [numpy supported functions](../howto/numpy_support.md) for infor
## Limitations
Floating point support in **Concrete** is very limited for the time being. They can't appear on inputs, or they can't be outputs. However, they can be used in intermediate results. Unfortunately, there are limitations on that front as well.
Floating point support in **Concrete Numpy** is very limited for the time being. They can't appear on inputs, or they can't be outputs. However, they can be used in intermediate results. Unfortunately, there are limitations on that front as well.
This biggest one is that, because floating point operations are fused into table lookups with a single unsigned integer input and single unsigned integer output, only univariate portion of code can be replaced with table lookups, which means multivariate portions cannot be compiled.