concrete-protocol
This commit: + Adds support for a protocol which enables inter-op between concrete, tfhe-rs and potentially other contributors to the fhe ecosystem. + Gets rid of hand-made serialization in the compiler, and client/server libs. + Refactors client/server libs to allow more pre/post processing of circuit inputs/outputs. The protocol is supported by a definition in the shape of a capnp file, which defines different types of objects among which: + ProgramInfo object, which is a precise description of a set of fhe circuit coming from the same compilation (understand function type information), and the associated key set. + *Key objects, which represent secret/public keys used to encrypt/execute fhe circuits. + Value object, which represent values that can be transferred between client and server to support calls to fhe circuits. The hand-rolled serialization that was previously used is completely dropped in favor of capnp in the whole codebase. The client/server libs, are refactored to introduce a modular design for pre-post processing. Reading the ProgramInfo file associated with a compilation, the client and server libs assemble a pipeline of transformers (functions) for pre and post processing of values coming in and out of a circuit. This design properly decouples various aspects of the processing, and allows these capabilities to be safely extended. In practice this commit includes the following: + Defines the specification in a concreteprotocol package + Integrate the compilation of this package as a compiler dependency via cmake + Modify the compiler to use the Encodings objects defined in the protocol + Modify the compiler to emit ProgramInfo files as compilation artifact, and gets rid of the bloated ClientParameters. + Introduces a new Common library containing the functionalities shared between the compiler and the client/server libs. + Introduces a functional pre-post processing pipeline to this common library + Modify the client/server libs to support loading ProgramInfo objects, and calling circuits using Value messages. + Drops support of JIT. + Drops support of C-api. + Drops support of Rust bindings. Co-authored-by: Nikita Frolov <nf@mkmks.org>
📒 Read documentation | 💛 Community support
⚠️ Starting from v1, Concrete Rust Libraries are now deprecated and replaced by TFHE-rs, Concrete is now, exclusively, Zama TFHE Compiler. Read full announcement here
Concrete is an open-source FHE Compiler which simplifies the use of fully homomorphic encryption (FHE).
FHE is a powerful cryptographic tool, which allows computation to be performed directly on encrypted data without needing to decrypt it first. With FHE, you can build services that preserve privacy for all users. FHE is also great against data breaches as everything is done on encrypted data. Even if the server is compromised, in the end no sensitive data is leaked.
Since writing FHE programs can be difficult, Concrete, based on LLVM, make this process easier for developers.
Main features
- Ability to compile Python functions (that may include NumPy) to their FHE equivalents, to operate on encrypted data
- Support for large collection of operators
- Partial support for floating points
- Support for table lookups on integers
- Support for integration with Client / Server architectures
Installation
| OS / HW | Available on Docker | Available on PyPI |
|---|---|---|
| Linux | Yes | Yes |
| Windows | Yes | No |
| Windows Subsystem for Linux | Yes | Yes |
| macOS (Intel) | Yes | Yes |
| macOS (Apple Silicon) | Yes | Yes |
The preferred way to install Concrete is through PyPI:
pip install concrete-python
You can get the concrete-python docker image by pulling the latest docker image:
docker pull zamafhe/concrete-python:v2.0.0
You can find more detailed installation instructions in installing.md
Getting started
from concrete import fhe
def add(x, y):
return x + y
compiler = fhe.Compiler(add, {"x": "encrypted", "y": "encrypted"})
inputset = [(2, 3), (0, 0), (1, 6), (7, 7), (7, 1), (3, 2), (6, 1), (1, 7), (4, 5), (5, 4)]
print(f"Compiling...")
circuit = compiler.compile(inputset)
print(f"Generating keys...")
circuit.keygen()
examples = [(3, 4), (1, 2), (7, 7), (0, 0)]
for example in examples:
encrypted_example = circuit.encrypt(*example)
encrypted_result = circuit.run(encrypted_example)
result = circuit.decrypt(encrypted_result)
print(f"Evaluation of {' + '.join(map(str, example))} homomorphically = {result}")
or if you have a simple function that you can decorate, and you don't care about explicit steps of key generation, encryption, evaluation and decryption:
from concrete import fhe
@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def add(x, y):
return x + y
inputset = [(2, 3), (0, 0), (1, 6), (7, 7), (7, 1), (3, 2), (6, 1), (1, 7), (4, 5), (5, 4)]
print(f"Compiling...")
circuit = add.compile(inputset)
examples = [(3, 4), (1, 2), (7, 7), (0, 0)]
for example in examples:
result = circuit.encrypt_run_decrypt(*example)
print(f"Evaluation of {' + '.join(map(str, example))} homomorphically = {result}")
Documentation
Full, comprehensive documentation is available at https://docs.zama.ai/concrete.
Target users
Concrete is a generic library that supports a variety of use cases. Because of this flexibility, it doesn't provide primitives for specific use cases.
If you have a specific use case, or a specific field of computation, you may want to build abstractions on top of Concrete.
One such example is Concrete ML, which is built on top of Concrete to simplify Machine Learning oriented use cases.
Tutorials
Various tutorials are proposed in the documentation to help you start writing homomorphic programs:
- How to use Concrete with Decorators
- Partial support of Floating Points
- How to perform Table Lookup
If you have built awesome projects using Concrete, feel free to let us know and we'll link to it.
Project layout
concrete project is a set of several modules which are high-level frontends, compilers, backends and side tools.
frontendsdirectory contains apythonfrontend.compilersdirectory contains theconcrete-compilerandconcrete-optimizermodules.concrete-compileris a compiler that:- synthetize a FHE computation dag expressed as a MLIR dialect
- compile to a set of artifacts
- and provide tools to manipulate those artifacts at runtime.
concrete-optimizeris a specific module used by the compiler to find the best, secure and accurate set of cryptographic parameters for a given dag.
- The
backendsdirectory contains implementations of cryptographic primitives on different computation unit, used byconcrete-compilerruntime.concrete-cpumodule provides CPU implementation, whileconcrete-cudamodule provides GPU implementation using the CUDA platform. - The
toolsdirectory contains side tools used by the rest of the project.
Need support?
Citing Concrete
To cite Concrete in academic papers, please use the following entry:
@Misc{Concrete,
title={{Concrete: TFHE Compiler that converts python programs into FHE equivalent}},
author={Zama},
year={2022},
note={\url{https://github.com/zama-ai/concrete}},
}
License
This software is distributed under the BSD-3-Clause-Clear license. If you have any questions, please contact us at hello@zama.ai.
