mirror of
https://github.com/zama-ai/concrete.git
synced 2026-02-17 16:11:26 -05:00
e8ef48ffd8
This commit: + Adds support for a protocol which enables inter-op between concrete, tfhe-rs and potentially other contributors to the fhe ecosystem. + Gets rid of hand-made serialization in the compiler, and client/server libs. + Refactors client/server libs to allow more pre/post processing of circuit inputs/outputs. The protocol is supported by a definition in the shape of a capnp file, which defines different types of objects among which: + ProgramInfo object, which is a precise description of a set of fhe circuit coming from the same compilation (understand function type information), and the associated key set. + *Key objects, which represent secret/public keys used to encrypt/execute fhe circuits. + Value object, which represent values that can be transferred between client and server to support calls to fhe circuits. The hand-rolled serialization that was previously used is completely dropped in favor of capnp in the whole codebase. The client/server libs, are refactored to introduce a modular design for pre-post processing. Reading the ProgramInfo file associated with a compilation, the client and server libs assemble a pipeline of transformers (functions) for pre and post processing of values coming in and out of a circuit. This design properly decouples various aspects of the processing, and allows these capabilities to be safely extended. In practice this commit includes the following: + Defines the specification in a concreteprotocol package + Integrate the compilation of this package as a compiler dependency via cmake + Modify the compiler to use the Encodings objects defined in the protocol + Modify the compiler to emit ProgramInfo files as compilation artifact, and gets rid of the bloated ClientParameters. + Introduces a new Common library containing the functionalities shared between the compiler and the client/server libs. + Introduces a functional pre-post processing pipeline to this common library + Modify the client/server libs to support loading ProgramInfo objects, and calling circuits using Value messages. + Drops support of JIT. + Drops support of C-api. + Drops support of Rust bindings. Co-authored-by: Nikita Frolov <nf@mkmks.org>
Python Frontend
Installation for end-users
End-users should install concrete-python using pip:
pip install concrete-python
Setup for development
Developers that want to contribute to the Concrete-Python project can use the following approach to setup their environment.
# clone the repository
git clone https://github.com/zama-ai/concrete.git --recursive
cd concrete
# create virtual environment
cd frontends/concrete-python
make venv
# activate virtual environment
source .venv/bin/activate
# build the compiler bindings
cd ../../compilers/concrete-compiler/compiler
make python-bindings
# set bindings build directory as an environment variable
# *** NOTE ***: You must use the Release build of the compiler!
# For now, the Debug build is not compatible with concrete-python
export COMPILER_BUILD_DIRECTORY=$(pwd)/build
echo "export COMPILER_BUILD_DIRECTORY=$(pwd)/build" >> ~/.bashrc
# run tests
cd ../../../frontends/concrete-python
make pytest
VSCode setup
Alternatively you can use VSCode to develop Concrete-Python:
Suppose the compiler bindings were built in /home/zama/concrete/compilers/concrete-compiler/compiler/build:
- Create a
.envfile in the concrete-python root directory - Determine the absolute path of the local compiler repository, e.g.
/home/zama/concrete. Replace this with your path in the following two lines - Add to it
PYTHONPATH=$(PYTHON_PATH):/home/zama/concrete/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/ - Add to it
LD_PRELOAD=/home/zama/concrete/compilers/concrete-compiler/compiler/build/lib/libConcretelangRuntime.so
You can now configure pytest in VScode and run the tests using the graphical interface.