Alexandre Péré e8ef48ffd8 feat(compiler): introduce concrete-protocol
This commit:
 + Adds support for a protocol which enables inter-op between concrete,
   tfhe-rs and potentially other contributors to the fhe ecosystem.
 + Gets rid of hand-made serialization in the compiler, and
   client/server libs.
 + Refactors client/server libs to allow more pre/post processing of
   circuit inputs/outputs.

The protocol is supported by a definition in the shape of a capnp file,
which defines different types of objects among which:
 + ProgramInfo object, which is a precise description of a set of fhe
   circuit coming from the same compilation (understand function type
   information), and the associated key set.
 + *Key objects, which represent secret/public keys used to
   encrypt/execute fhe circuits.
 + Value object, which represent values that can be transferred between
   client and server to support calls to fhe circuits.

The hand-rolled serialization that was previously used is completely
dropped in favor of capnp in the whole codebase.

The client/server libs, are refactored to introduce a modular design for
pre-post processing. Reading the ProgramInfo file associated with a
compilation, the client and server libs assemble a pipeline of
transformers (functions) for pre and post processing of values coming in
and out of a circuit. This design properly decouples various aspects of
the processing, and allows these capabilities to be safely extended.

In practice this commit includes the following:
 + Defines the specification in a concreteprotocol package
 + Integrate the compilation of this package as a compiler dependency
   via cmake
 + Modify the compiler to use the Encodings objects defined in the
   protocol
 + Modify the compiler to emit ProgramInfo files as compilation
   artifact, and gets rid of the bloated ClientParameters.
 + Introduces a new Common library containing the functionalities shared
   between the compiler and the client/server libs.
 + Introduces a functional pre-post processing pipeline to this common
   library
 + Modify the client/server libs to support loading ProgramInfo objects,
   and calling circuits using Value messages.
 + Drops support of JIT.
 + Drops support of C-api.
 + Drops support of Rust bindings.

Co-authored-by: Nikita Frolov <nf@mkmks.org>
2023-11-09 17:09:04 +01:00
2022-12-05 12:35:26 +01:00


📒 Read documentation | 💛 Community support


⚠️ Starting from v1, Concrete Rust Libraries are now deprecated and replaced by TFHE-rs, Concrete is now, exclusively, Zama TFHE Compiler. Read full announcement here

Concrete is an open-source FHE Compiler which simplifies the use of fully homomorphic encryption (FHE).

FHE is a powerful cryptographic tool, which allows computation to be performed directly on encrypted data without needing to decrypt it first. With FHE, you can build services that preserve privacy for all users. FHE is also great against data breaches as everything is done on encrypted data. Even if the server is compromised, in the end no sensitive data is leaked.

Since writing FHE programs can be difficult, Concrete, based on LLVM, make this process easier for developers.

Main features

  • Ability to compile Python functions (that may include NumPy) to their FHE equivalents, to operate on encrypted data
  • Support for large collection of operators
  • Partial support for floating points
  • Support for table lookups on integers
  • Support for integration with Client / Server architectures

Installation

OS / HW Available on Docker Available on PyPI
Linux Yes Yes
Windows Yes No
Windows Subsystem for Linux Yes Yes
macOS (Intel) Yes Yes
macOS (Apple Silicon) Yes Yes

The preferred way to install Concrete is through PyPI:

pip install concrete-python

You can get the concrete-python docker image by pulling the latest docker image:

docker pull zamafhe/concrete-python:v2.0.0

You can find more detailed installation instructions in installing.md

Getting started

from concrete import fhe

def add(x, y):
    return x + y

compiler = fhe.Compiler(add, {"x": "encrypted", "y": "encrypted"})
inputset = [(2, 3), (0, 0), (1, 6), (7, 7), (7, 1), (3, 2), (6, 1), (1, 7), (4, 5), (5, 4)]

print(f"Compiling...")
circuit = compiler.compile(inputset)

print(f"Generating keys...")
circuit.keygen()

examples = [(3, 4), (1, 2), (7, 7), (0, 0)]
for example in examples:
    encrypted_example = circuit.encrypt(*example)
    encrypted_result = circuit.run(encrypted_example)
    result = circuit.decrypt(encrypted_result)
    print(f"Evaluation of {' + '.join(map(str, example))} homomorphically = {result}")

or if you have a simple function that you can decorate, and you don't care about explicit steps of key generation, encryption, evaluation and decryption:

from concrete import fhe

@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def add(x, y):
    return x + y

inputset = [(2, 3), (0, 0), (1, 6), (7, 7), (7, 1), (3, 2), (6, 1), (1, 7), (4, 5), (5, 4)]

print(f"Compiling...")
circuit = add.compile(inputset)

examples = [(3, 4), (1, 2), (7, 7), (0, 0)]
for example in examples:
    result = circuit.encrypt_run_decrypt(*example)
    print(f"Evaluation of {' + '.join(map(str, example))} homomorphically = {result}")

Documentation

Full, comprehensive documentation is available at https://docs.zama.ai/concrete.

Target users

Concrete is a generic library that supports a variety of use cases. Because of this flexibility, it doesn't provide primitives for specific use cases.

If you have a specific use case, or a specific field of computation, you may want to build abstractions on top of Concrete.

One such example is Concrete ML, which is built on top of Concrete to simplify Machine Learning oriented use cases.

Tutorials

Various tutorials are proposed in the documentation to help you start writing homomorphic programs:

If you have built awesome projects using Concrete, feel free to let us know and we'll link to it.

Project layout

concrete project is a set of several modules which are high-level frontends, compilers, backends and side tools.

  • frontends directory contains a python frontend.
  • compilers directory contains the concrete-compiler and concrete-optimizer modules. concrete-compiler is a compiler that:
    • synthetize a FHE computation dag expressed as a MLIR dialect
    • compile to a set of artifacts
    • and provide tools to manipulate those artifacts at runtime. concrete-optimizer is a specific module used by the compiler to find the best, secure and accurate set of cryptographic parameters for a given dag.
  • The backends directory contains implementations of cryptographic primitives on different computation unit, used by concrete-compiler runtime. concrete-cpu module provides CPU implementation, while concrete-cuda module provides GPU implementation using the CUDA platform.
  • The tools directory contains side tools used by the rest of the project.

Need support?

Citing Concrete

To cite Concrete in academic papers, please use the following entry:

@Misc{Concrete,
  title={{Concrete: TFHE Compiler that converts python programs into FHE equivalent}},
  author={Zama},
  year={2022},
  note={\url{https://github.com/zama-ai/concrete}},
}

License

This software is distributed under the BSD-3-Clause-Clear license. If you have any questions, please contact us at hello@zama.ai.

Description
No description provided
Readme 148 MiB
Languages
C++ 34.3%
Python 23.1%
MLIR 22.9%
Rust 14.6%
C 2.2%
Other 2.8%