refactor(strings): move fhe_strings from examples to strings module

This commit is contained in:
Mayeul@Zama
2024-10-21 16:38:13 +02:00
committed by mayeul-zama
parent daf57f5665
commit 8680e1de0a
21 changed files with 16 additions and 68 deletions

View File

@@ -779,6 +779,13 @@ test_high_level_api_gpu: install_rs_build_toolchain install_cargo_nextest
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,gpu -p $(TFHE_SPEC) \
-E "test(/high_level_api::.*gpu.*/)"
.PHONY: test_strings # Run the tests for strings ci
test_strings: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,strings -p $(TFHE_SPEC) \
-- strings::
.PHONY: test_user_doc # Run tests from the .md documentation
test_user_doc: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) --doc \
@@ -792,11 +799,7 @@ test_user_doc_gpu: install_rs_build_toolchain
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache,gpu,zk-pok -p $(TFHE_SPEC) \
-- test_user_docs::
.PHONY: test_fhe_strings # Run tests for fhe_strings example
test_fhe_strings: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--example fhe_strings \
--features=$(TARGET_ARCH_FEATURE),integer
.PHONY: test_regex_engine # Run tests for regex_engine example
test_regex_engine: install_rs_build_toolchain

View File

@@ -10,11 +10,9 @@ const DIR_TO_IGNORE: [&str; 3] = [
"tfhe/tfhe-backward-compat-data",
];
const FILES_TO_IGNORE: [&str; 4] = [
const FILES_TO_IGNORE: [&str; 3] = [
// This contains fragments of code that are unrelated to TFHE-rs
"tfhe/docs/tutorials/sha256_bool.md",
// This contains fragments of code coming from the tutorial that cannot be run as a doctest
"tfhe/examples/fhe_strings/README.md",
// TODO: This contains code that could be executed as a trivium docstring
"apps/trivium/README.md",
// TODO: should we test this ?

View File

@@ -93,6 +93,7 @@ bytemuck = "1.14.3"
boolean = []
shortint = ["dep:sha3"]
integer = ["shortint"]
strings = ["integer"]
internal-keycache = ["dep:lazy_static", "dep:fs2"]
gpu = ["dep:tfhe-cuda-backend"]
zk-pok = ["dep:tfhe-zk-pok"]
@@ -308,10 +309,6 @@ required-features = ["shortint", "internal-keycache"]
# Real use-case examples
[[example]]
name = "fhe_strings"
required-features = ["integer"]
[[example]]
name = "dark_market"
required-features = ["integer", "internal-keycache"]

View File

@@ -1,53 +0,0 @@
# FHE Strings
This example contains the implementation of a str API in FHE, featuring 30 methods. This API allows the user to:
* Encrypt the `str` with or without padding nulls (i.e. encrypted `0u8`s at the end of the string), which serve to obfuscate the length but are ignored by algorithms
* Encrypt any kind of pattern (`pat`, `from`, `to`, `rhs`) with or without padding nulls
* Encrypt the number of repetitions `n`, allowing to provide a clear `max` to restrict the range of the encrypted `n`
* Provide a cleartext pattern when algorithms can run faster. Otherwise, it's possible to trivially encrypt the pattern with `FheString::trivial`
Encrypted strings contain a flag indicating whether they have padding nulls or not. Algorithms are optimized to differentiate between the two kind of strings. For instance, in some cases we can skip entirely the FHE computations if we know the true lengths of the string or pattern.
Just like the clear str API, any encrypted string returned by a function can be used as input to other functions. For instance when `trim_start` is executed, or a `Split` iterator instance is advanced with `next`, the result will only have nulls at the end. The decryption function `decrypt_ascii` will panic if it encounters with malformed encrypted strings, including padding inconsistencies.
### Example
```rust
let (ck, sk) = gen_keys();
let s = "Zama ";
let padding = Some(2);
let enc_s = FheString::new(&ck, &s, padding);
let clear_count = UIntArg::Clear(3);
// All the nulls are shifted to the right end
let result_repeat = sk.repeat(&enc_s, &clear_count);
let result_trim_end = sk.trim_end(&result_repeat);
let result_uppercase = sk.to_uppercase(&result_trim_end);
let clear = ck.decrypt_ascii(&result_uppercase);
assert_eq!(clear, "ZAMA ZAMA ZAMA");
```
## Technical Details
We have implemented conversions between encrypted strings (`FheString`) and UInts (`RadixCiphertext`). This is useful for:
- Speeding up comparisons and pattern matching: We perform a _single comparison_ between two numbers. This is more efficient than many u8 comparisons.
- Shifting by an encrypted number of characters: By treating the string as a `RadixCiphertext` we can use the tfhe-rs shifting operations, and then convert back to `FheString`.
Similarly, when a pattern is provided in the clear (`ClearString`) we convert it to `StaticUnsignedBigInt<N>`. This type requires a constant `N` for the u64 length of the clear UInt, and we have set it to 4, allowing for up to 32 characters in `ClearString`.
`N` can be increased in `main.rs` to enable longer clear patterns, or reduced to improve performance (if we know that we will work with smaller clear patterns).
## Test Cases
We have handled corner cases like empty strings and empty patterns (with and without padding), the number of repetitions `n` (clear and encrypted) being zero, etc. A complete list of tests can be found at `assert_functions/test_vectors.rs`.
## Usage
To run all the functions and see the comparison with the clear Rust API you can specify the following arguments:
```--str <"your str"> --pat <"your pattern"> --to <"argument used in replace"> --rhs <"used in comparisons and concat"> --n <number of repetitions> --max <clear max n>```
To optionally specify a number of padding nulls for any argument you can also use: ``--str_pad``, ``--pat_pad``, ``--to_pad`` and ``--rhs_pad``.

View File

@@ -119,6 +119,9 @@ mod js_on_wasm_api;
))]
mod test_user_docs;
#[cfg(feature = "strings")]
pub mod strings;
#[cfg(feature = "integer")]
/// cbindgen:ignore
pub(crate) mod high_level_api;

View File

@@ -4,9 +4,9 @@ use crate::server_key::{gen_keys, FheStringIsEmpty, FheStringIterator, FheString
use clap::{value_parser, Arg, Command};
use std::time::Instant;
mod ciphertext;
mod client_key;
mod server_key;
pub mod ciphertext;
pub mod client_key;
pub mod server_key;
mod assert_functions;