github/concrete
1
1
Fork 0
mirror of https://github.com/zama-ai/concrete.git synced 2026-02-09 03:55:04 -05:00
Code Issues Packages Projects Releases Wiki Activity

doc: let's agree on a plan

Browse Source 
closes #305
This commit is contained in:
Benoit Chevallier-Mames
2021-09-08 11:38:49 +02:00
committed by Benoit Chevallier
parent 5041e42978
commit bcc146bd6e
25 changed files with 84 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

348
docs/dev/explanation/COMPILATION.md Normal file
View File

@@ -0,0 +1,348 @@
# Compilation Pipeline In Depth
## What is concretefhe?
`concretefhe` is the python API of the `concrete` framework for developing homomorphic applications.
One of its essential functionalities is to transform Python functions to their `MLIR` equivalent.
Unfortunately, not all python functions can be converted due to the limits of current product (we are in the alpha stage), or sometimes due to inherent restrictions of FHE itself.
However, one can already build interesting and impressing use cases, and more will be available in further versions of the framework.
## How can I use it?
```python
# Import necessary concrete components
import concrete.numpy as hnp
# Define the function to homomorphize
def f(x, y):
return (2 * x) + y
# Define the inputs of homomorphized function
x = hnp.EncryptedScalar(hnp.UnsignedInteger(2))
y = hnp.EncryptedScalar(hnp.UnsignedInteger(1))
# Compile the function to its homomorphic equivalent
engine = hnp.compile_numpy_function(
f, {"x": x, "y": y},
iter([(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1), (3, 0), (3, 1)]),
)
# Make homomorphic inference
engine.run([1, 0])
```
## Overview
The compilation journey begins with tracing to get an easy to understand and manipulate representation of the function.
We call this representation `Operation Graph` which is basically a Directed Acyclic Graph (DAG) containing nodes representing the computations done in the function.
Working with graphs is good because they have been studied extensively over the years and there are a lot of algorithms to manipulate them.
Internally, we use [networkx](https://networkx.org) which is an excellent graph library for Python.
The next step in the compilation is transforming the operation graph.
There are many transformations we perform, and they will be discussed in their own sections.
In any case, the result of transformations is just another operation graph.
After transformations are applied, we need to determine the bounds (i.e., the minimum and the maximum values) of each intermediate result.
This is required because FHE currently allows a limited precision for computations.
Bound measurement is our way to know what is the needed precision for the function.
There are several approaches to compute bounds, and they will be discussed in their own sections.
The final step is to transform the operation graph to equivalent `MLIR` code.
How this is done will be explained in detail in its own chapter.
Once the MLIR is prepared, the rest of the stack, which you can learn more about [here](http://fixme.com/), takes over and completes the compilation process.
Here is the visual representation of the pipeline:
![Frontend Flow](../../_static/compilation-pipeline/frontend_flow.svg)
## Tracing
Given a Python function `f` such as this one,
```
def f(x):
return (2 * x) + 3
```
the goal of tracing is to create the following operation graph without needing any change from the user.
![](../../_static/compilation-pipeline/two_x_plus_three.png)
(Note that the edge labels are for non-commutative operations. To give an example, a subtraction node represents `(predecessor with edge label 0) - (predecessor with edge label 1)`)
To do this, we make use of `Tracer`s, which are objects that record the operation performed during their creation.
We create a `Tracer` for each argument of the function and call the function with those tracers.
`Tracer`s make use of operator overloading feature of Python to achieve their goal.
Here is an example:
```
def f(x, y):
return x + 2 * y
x = Tracer(computation=Input("x"))
y = Tracer(computation=Input("y"))
resulting_tracer = f(x, y)
```
`2 * y` will be performed first, and `*` is overloaded for `Tracer` to return another tracer:
`Tracer(computation=Multiply(Constant(2), self.computation))` which is equal to:
`Tracer(computation=Multiply(Constant(2), Input("y")))`
`x + (2 * y)` will be performed next, and `+` is overloaded for `Tracer` to return another tracer:
`Tracer(computation=Add(self.computation, (2 * y).computation))` which is equal to:
`Tracer(computation=Add(Input("x"), Multiply(Constant(2), Input("y")))`
In the end we will have output `Tracer`s that can be used to create the operation graph.
The implementation is a bit more complex than that but the idea is the same.
Tracing is also responsible for indicating whether the values in the node would be encrypted or not, and the rule for that is if a node has an encrypted predecessor, it is encrypted as well.
## Topological Transforms
The goal of topological transforms is to make more functions compilable.
With the current version of `concrete` floating point inputs and floating point outputs are not supported.
However, if the floating points operations are intermediate operations, they can sometimes be fused into a single table lookup from integer to integer thanks to some specific transforms.
Let's take a closer look at the transforms we perform today.
### Fusing Floating Point Operations
We decided to allocate a whole new chapter to explain float fusing.
You can find it [here](./FLOAT-FUSING.md).
## Bounds Measurement
Given an operation graph, goal of the bound measurement step is to assign the minimal data type to each node in the graph.
Let's say we have an encrypted input that is always between `0` and `10`, we should assign the type `Encrypted<uint4>` to node of this input as `Encrypted<uint4>` is the minimal encrypted integer that supports all the values between `0` and `10`.
If there were negative values in the range, we could have used `intX` instead of `uintX`.
Bounds measurement is necessary because FHE supports limited precision, and we don't want unexpected behaviour during evaluation of the compiled functions.
There are several ways to perform bounds measurement.
Let's take a closer look at the options we provide today.
### Dataset Evaluation
This is the simplest approach, but it requires a dataset to be provided by the user.
The dataset is not the dataset in the usual sense of ML as it doesn't require labels.
Rather, it is a set of values which are typical inputs of the function.
The idea is to evaluate each input in the dataset and record the result of each operation in the operation graph.
Then we compare the evaluation results with the current minimum/maximum values of each node and update the minimum/maximum accordingly.
After the entire dataset is evaluated, we assign a data type to each node using the minimum and the maximum value it contained.
Here is an example, given this operation graph where `x` is encrypted:
![](../../_static/compilation-pipeline/two_x_plus_three.png)
and this dataset:
```
[2, 3, 1]
```
Evaluation Result of `2`:
- `x`: 2
- `2`: 2
- `*`: 4
- `3`: 3
- `+`: 7
New Bounds:
- `x`: [**2**, **2**]
- `2`: [**2**, **2**]
- `*`: [**4**, **4**]
- `3`: [**3**, **3**]
- `+`: [**7**, **7**]
Evaluation Result of `3`:
- `x`: 3
- `2`: 2
- `*`: 6
- `3`: 3
- `+`: 9
New Bounds:
- `x`: [2, **3**]
- `2`: [2, 2]
- `*`: [4, **6**]
- `3`: [3, 3]
- `+`: [7, **9**]
Evaluation Result of `1`:
- `x`: 1
- `2`: 2
- `*`: 2
- `3`: 3
- `+`: 5
New Bounds:
- `x`: [**1**, 3]
- `2`: [2, 2]
- `*`: [**2**, 6]
- `3`: [3, 3]
- `+`: [**5**, 9]
Assigned Data Types:
- `x`: Encrypted\<**uint2**>
- `2`: Clear\<**uint2**>
- `*`: Encrypted\<**uint3**>
- `3`: Clear\<**uint2**>
- `+`: Encrypted\<**uint4**>
## MLIR Lowering
TODO: Ayoub
## Example Walkthrough #1
### Function to Homomorphize
```
def f(x):
return (2 * x) + 3
```
### Parameters
```
x = EncryptedScalar(UnsignedInteger(2))
```
#### Corresponding Operation Graph
![](../../_static/compilation-pipeline/two_x_plus_three.png)
### Topological Transforms
#### Fusing Floating Point Operations
This transform isn't applied since the computation doesn't involve any floating point operations.
### Bounds Measurement Using [2, 3, 1] as Dataset (same settings as above)
Data Types:
- `x`: Encrypted\<**uint2**>
- `2`: Clear\<**uint2**>
- `*`: Encrypted\<**uint3**>
- `3`: Clear\<**uint2**>
- `+`: Encrypted\<**uint4**>
### MLIR Lowering
```
module {
func @main(%arg0: !HLFHE.eint<4>) -> !HLFHE.eint<4> {
%c3_i5 = constant 3 : i5
%c2_i5 = constant 2 : i5
%0 = "HLFHE.mul_eint_int"(%arg0, %c2_i5) : (!HLFHE.eint<4>, i5) -> !HLFHE.eint<4>
%1 = "HLFHE.add_eint_int"(%0, %c3_i5) : (!HLFHE.eint<4>, i5) -> !HLFHE.eint<4>
return %1 : !HLFHE.eint<4>
}
}
```
## Example Walkthrough #2
### Function to Homomorphize
```
def f(x, y):
return (42 - x) + (y * 2)
```
### Parameters
```
x = EncryptedScalar(UnsignedInteger(3))
y = EncryptedScalar(UnsignedInteger(1))
```
#### Corresponding Operation Graph
![](../../_static/compilation-pipeline/forty_two_minus_x_plus_y_times_two.png)
### Topological Transforms
#### Fusing Floating Point Operations
This transform isn't applied since the computation doesn't involve any floating point operations.
### Bounds Measurement Using [(6, 0), (5, 1), (3, 0), (4, 1)] as Dataset
Evaluation Result of `(6, 0)`:
- `42`: 42
- `x`: 6
- `y`: 0
- `2`: 2
- `-`: 36
- `*`: 0
- `+`: 36
Evaluation Result of `(5, 1)`:
- `42`: 42
- `x`: 5
- `y`: 1
- `2`: 2
- `-`: 37
- `*`: 2
- `+`: 39
Evaluation Result of `(3, 0)`:
- `42`: 42
- `x`: 3
- `y`: 0
- `2`: 2
- `-`: 39
- `*`: 0
- `+`: 39
Evaluation Result of `(4, 1)`:
- `42`: 42
- `x`: 4
- `y`: 1
- `2`: 2
- `-`: 38
- `*`: 2
- `+`: 40
Bounds:
- `42`: [42, 42]
- `x`: [3, 6]
- `y`: [0, 1]
- `2`: [2, 2]
- `-`: [36, 39]
- `*`: [0, 2]
- `+`: [36, 40]
Data Types:
- `42`: Clear\<**uint6**>
- `x`: Encrypted\<**uint3**>
- `y`: Encrypted\<**uint1**>
- `2`: Clear\<**uint2**>
- `-`: Encrypted\<**uint6**>
- `*`: Encrypted\<**uint2**>
- `+`: Encrypted\<**uint6**>
### MLIR Lowering
```
module {
func @main(%arg0: !HLFHE.eint<6>, %arg1: !HLFHE.eint<6>) -> !HLFHE.eint<6> {
%c42_i7 = constant 42 : i7
%c2_i7 = constant 2 : i7
%0 = "HLFHE.sub_int_eint"(%c42_i7, %arg0) : (i7, !HLFHE.eint<6>) -> !HLFHE.eint<6>
%1 = "HLFHE.mul_eint_int"(%arg1, %c2_i7) : (!HLFHE.eint<6>, i7) -> !HLFHE.eint<6>
%2 = "HLFHE.add_eint"(%0, %1) : (!HLFHE.eint<6>, !HLFHE.eint<6>) -> !HLFHE.eint<6>
return %2 : !HLFHE.eint<6>
}
}
```

77
docs/dev/explanation/FLOAT-FUSING.md Normal file
View File

@@ -0,0 +1,77 @@
# Fusing Floating Point Operations
## Why is it needed?
The current compiler stack only supports integers with 7 bits or less. But it's not uncommon to have numpy code using floating point numbers.
We added fusing floating point operations to make tracing numpy functions somewhat user friendly to allow in-line quantization in the numpy code e.g.:
```python
import numpy
def quantized_sin(x):
# from a 7 bit unsigned integer x, compute z in the [0; 2 * pi] range
z = 2 * numpy.pi * x * (1 / 127)
# quantize over 6 bits and offset to be >= 0, round and convert to integers in range [0; 63]
quantized_sin = numpy.rint(31 * numpy.sin(z) + 31).astype(numpy.int32)
# output quantized_sin and a further offset result
return quantized_sin, quantized_sin + 32
```
This function `quantized_sin` is not strictly supported as is by the compiler as there are floating point intermediate values. However, when looking at the function globally we can see we have a single integer input and a single integer output. As we know the input range we can compute a table to represent the whole computation for each input value, which can later be lowered to a PBS in the FHE world.
Any computation where there is a single variable integer input and a single integer output can be replaced by an equivalent table look-up.
The `quantized_sin` graph of operations:
![](../../_static/float_fusing_example/before.png)
The float subgraph that was detected:
![](../../_static/float_fusing_example/subgraph.png)
The simplified graph of operations with the float subgraph condensed in an `ArbitraryFunction` node:
![](../../_static/float_fusing_example/after.png)
## How is it done in concretefhe?
The first step consists in detecting where we go from floating point computation back to integers. This allows to identify the potential terminal node of the float subgraph we are going to fuse.
From the terminal node, we go back up through the nodes until we find nodes that go from integers to floats. If we can guarantee the identified float subgraph has a single variable integer input then we can replace it by an equivalent ArbitraryFunction node.
An example of a non fusable computation with that technique is:
```python
import numpy
def non_fusable(x, y):
x_1 = x + 1.5 # x_1 is now float
y_1 = y + 3.4 # y_1 is now float
add = x_1 + y_1
add_int = add.astype(numpy.int32)
return add_int
```
From `add_int` you will find two `Add` nodes going from int to float (`x_1` and `y_1`) which we cannot represent with a single input table look-up.
## Possible improvements
This technique is not perfect because you could try to go further back in the graph to find a single variable input.
Firstly, it does not cover optimizing the graph, so you can end up with multiple operations, like additions with constants, or two look-up tables in a row, that can trivially be fused but that are not fused, as the optimization work is left to the compiler backend with MLIR and LLVM tooling. This first limitation does not impact the kind of programs that are compilable.
Secondly, the current approach fails to handle some programs that in practice could be compiled. The following example could be covered by pushing the search to find a single integer input:
```python
def theoretically_fusable(x):
x_1 = x + 1.5
x_2 = x + 3.4
add = x_1 + x_2
add_int = add.astype(numpy.int32)
return add_int
```
Here the whole function is a single int giving a single int (i.e. representable by a table look-up), but the current implementation of fusing is going to find `x_1` and `x_2` as the starting nodes of the float subgraph and say it cannot fuse it. Room for improvement. It is probably a graph coloring problem where we would have to list for all graph's inputs which nodes depend on them.
At some point having a proper optimization system with patterns and rules to rewrite them could become more interesting than having this ad-hoc system.

4
docs/dev/explanation/MLIR.md Normal file
View File

@@ -0,0 +1,4 @@
# MLIR
to be done

30
docs/dev/explanation/TERMINOLOGY_AND_STRUCTURE.md Normal file
View File

@@ -0,0 +1,30 @@
# Terminology and Structure
## Terminology
In this section we will go over some terms that we use throughout the project.
- intermediate representation
- a data structure to represent a calculation
- basically a computation graph where nodes are either inputs or operations on other nodes
- tracing
- it is our technique to take directly a plain numpy function from a user and deduce its intermediate representation in a painless way for the user
- bounds
- before intermediate representation is sent to the compiler, we need to know which node will output which type (e.g., uint3 vs uint5)
- there are several ways to do this but the simplest one is to evaluate the intermediate representation with all combinations of inputs and remember the maximum and the minimum values for each node, which is what we call bounds, and bounds can be used to determine the appropriate type for each node
## Module Structure
In this section, we will discuss the module structure of `concretefhe` briefly. You are encouraged to check individual `.py` files to learn more!
- concrete
- common: types and utilities that can be used by multiple frontends (e.g., numpy, torch)
- bounds_measurement: utilities for determining bounds of intermediate representation
- compilation: type definitions related to compilation (e.g., compilation config, compilation artifacts)
- data_types: type definitions of typing information of intermediate representation
- debugging: utilities for printing/displaying intermediate representation
- extensions: utilities that provide special functionality to our users
- representation: type definitions of intermediate representation
- tracing: utilities for generic function tracing used during intermediate representation creation
- numpy: numpy frontend of the package