Compare commits

...

602 Commits

Author SHA1 Message Date
David Testé
c42dee5e51 test(core_crypto): improve coverage for various files 2023-12-20 18:43:32 +01:00
David Testé
37d25a659f chore(ci): improve coverage by ignoring assertion statements
Those statements, if they contain a custom string to display,
are not covered in the test even when implementing tests that
trigger the assert macro.
2023-12-20 18:34:58 +01:00
tmontaigu
526a53f3d4 feat(integer): add signed_overflowing_scalar_add/sub 2023-12-15 10:57:21 +01:00
Mayeul@Zama
7d17b71740 fix(shortint): fix smart_add/sub 2023-12-13 09:55:20 +01:00
Mayeul@Zama
8ecb85e4dd refactor(shortint): is_possible ops take CtNoiseDegree 2023-12-13 09:55:20 +01:00
Mayeul@Zama
cf30db7a30 fix(shortint): rename smart_apply_lookup_table_bivariate 2023-12-13 09:55:20 +01:00
Mayeul@Zama
2599f7d5ea fix(shortint): fix ciphertexts_can_be_packed_without_exceeding_space_or_noise 2023-12-13 09:55:20 +01:00
Mayeul@Zama
ae88bb3264 fix(shortint): fix smart_evaluate_bivariate_function 2023-12-13 09:55:20 +01:00
Mayeul@Zama
6fb898db66 refactor(shortint): move bivariate_pbs to its own module 2023-12-13 09:55:20 +01:00
Mayeul@Zama
e750d2cd92 refactor(shortint): use smart_evaluate_bivariate_function and evaluate_univariate_function 2023-12-13 09:55:20 +01:00
Arthur Meyre
d8586080da chore(integer): simplify an API to create a ServerKey from a shortint one
- the API required passing the client_key to get access to the message and
carry modulus, as the shortint ServerKey already has that information drop
the requirement to pass the ClientKey

BREAKING CHANGE:
new_radix_server_key_from_shortint, new_crt_server_key_from_shortint APIs
have changed and no longer require a ClienKey, the max degree methods now
only take a MessageModulus and CarryModulus instead of the full parameter
set
2023-12-13 09:41:59 +01:00
tmontaigu
4cc2e85556 feat(integer): add unsigned_overflowing_scalar_sub 2023-12-12 16:08:06 +01:00
tmontaigu
303a65c88d feat(integer): add unsigned_overflowing_scalar_add 2023-12-12 16:08:06 +01:00
Arthur Meyre
18c01e74d6 chore(core_crypto): remove legacy new types 2023-12-12 11:06:06 +01:00
dependabot[bot]
a8b6c72910 chore(deps): bump tj-actions/changed-files from 40.2.0 to 40.2.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 40.2.0 to 40.2.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](da093c1609...1c938490c8)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-11 10:21:53 +01:00
Arthur Meyre
c2b21ed709 chore(integer): remove unused RadixDecomposition struct and code 2023-12-08 10:38:21 +01:00
tmontaigu
ad41fdf5a5 feat(integer): expose default and smart ciphertext sum
This expose the ciphertext sum default and smart variants

This also removes the par_seq_op functions as they are less optimal
2023-12-06 10:50:45 +01:00
Arthur Meyre
eeae19f35f chore(core_crypto): disable seeded entities for non power of 2 moduli
- as random uniform generation has rejection sampling for non native moduli
the seeded decompression currently does not work as it allocates just
enough bytes for a native integer and not for the various retries which may
be needed
- follow-up issue: https://github.com/zama-ai/tfhe-rs-internal/issues/358
2023-12-05 15:25:30 +01:00
Mayeul@Zama
798572e58c style(shortint): rename MaxNoiseLevel::valid validate 2023-12-04 18:16:46 +01:00
Mayeul@Zama
36d375943c refactor(shortint): encapsulate Degree and MaxDegree 2023-12-04 18:16:46 +01:00
Mayeul@Zama
a1488b10d5 style(shortint): move MaxDegree 2023-12-04 18:16:46 +01:00
Mayeul@Zama
b153641280 style(shortint): scalar_sub use scalar_add 2023-12-04 18:16:46 +01:00
tmontaigu
48405959a4 feat(integer): add decrypt_trivial
This adds a decrypt_trivial method to all ciphertext types of
shortint and integer.

This functions tries to "decrypt" the ciphertext if it is a
trivial one, otherwise it return an error.

This is meant to be a debugging 'tool':

To debug a function / circuit, users can call the function
on trivial ciphertexts intead of real ciphertext, that way,
computations are faster _and_ they will now be able to see intermediate
values via these decrypt_trivial, to do some print-debugging or use a
debugger.
2023-12-04 15:50:53 +01:00
Arthur Meyre
d39e73be91 chore(core): freshen up native decomposition tests
- the classic native decomposer results are exact and there aren't that
many cases to test so tests are changed to be exhaustive
- non native currently still has some work in progress parts so won't be
made exhaustive right away, additionally the next PR for the prime Q effort
will make some changes to it, so this is why the current code has not been
touched for the non native decomposer
2023-12-04 14:56:17 +01:00
dependabot[bot]
71447d845f chore(deps): bump tj-actions/changed-files from 40.1.1 to 40.2.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 40.1.1 to 40.2.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](25ef3926d1...da093c1609)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-04 09:22:17 +01:00
Arthur Meyre
837df59b44 fix(shortint): programmable_bootstrapping_native_crt could alter its input
- the alteration is a trick to be able to perform the wopbs in the native
crt mode but it modifies the input ciphertext and leaves it modified
meaning that the input no longer represents the same data/encryption
- applying the functon several times likely would lead to incorrect results
starting from the second call with increasing error/divergence from the
original encrypted data
- clone locally instead, should be negligible given wopbs runtime

BREAKING CHANGE:
programmable_bootstrapping_native_crt signature has changed
2023-12-01 14:46:09 +01:00
Arthur Meyre
303cac2092 chore(ci): re-enable release profile for doctest
- following merge of 17.0.4 in rust stable the bug uncovered by lto on
aarch64 has been fixed https://github.com/rust-lang/rust/issues/116941 so
we remove the hard coded override
2023-12-01 12:34:26 +01:00
Arthur Meyre
bb1a969c34 chore(ci): update nightly toolchain to have fixed LLVM as well
- fix lints linked to latest nightly
2023-12-01 12:34:26 +01:00
tmontaigu
9dac9242be feat(integer): add boolean_ops to work on BooleanBlock
This adds boolean_bitand/or/xor/not to work on BooleanBlock
This is meant to improve API of BooleanBlock.
2023-12-01 11:13:31 +01:00
tmontaigu
e97ac815eb feat(shortint): add trivial pbs
When we detect that the ciphertext to be bootstrapped
is a trivial ciphertext, we can apply the lookup table
in clear thus saving computation time.

This allows shortint,integer,hlapi to have
way faster computations when all inputs are trivial
allowing to more rapidly check and debug a circuit
2023-11-30 22:01:28 +01:00
Arthur Meyre
62feb59722 chore(ci): fix doctest by using parameters with enough precision 2023-11-30 14:57:07 +01:00
Mayeul@Zama
ac8916a30f refactor(shortint): define woppbs on server key instead of engine 2023-11-30 14:54:33 +01:00
Mayeul@Zama
069ea98ad6 style(shortint): fix remaining unused_self 2023-11-30 14:54:33 +01:00
Mayeul@Zama
e587d1835e refactor(shortint): define operations on server key instead of engine 2023-11-30 14:54:33 +01:00
Mayeul@Zama
16121e7487 style(core): fix some unused self 2023-11-30 14:54:33 +01:00
Mayeul@Zama
3ab566de7b style(boolean): fix some unused self 2023-11-30 14:54:33 +01:00
David Testé
2309b07703 test(integer): add smart tests for radix_parallel min/max
This also refactor the code to use macro to parametrize tests to
make code smaller.
2023-11-30 14:44:17 +01:00
David Testé
8755094c38 test(integer): fix tests for unsigned scalar min/max operations 2023-11-30 14:44:17 +01:00
David Testé
4da10e9dd5 chore(ci): add aws ec2 fallback profile for cpu tests
This is done to mitigate resource shortages in our base AWS region
(eu-west-3) due to the high number of instances that are launched
in parallel in our Pull Requests.
2023-11-30 13:02:42 +01:00
Arthur Meyre
cdda260063 chore(ci): .gitignore was ignoring all files/directories names keys
- this was hiding some source files in vscode search and could likely have
been very annoying when commiting stuff
2023-11-29 18:14:33 +01:00
Arthur Meyre
be413fff50 chore(shortint): remove doctest as tests as it is confirmed they fail
- doctest were also failing as tests and so it is not linked to doctests
- still unclear what is causing the issue, the results are sometimes way
off
- the concentration of failed tests can indicate a miscompile as those
tests never fail on the M1 CI, some alignment is causing issues from time
to time
2023-11-29 17:43:25 +01:00
Arthur Meyre
3ed960d255 chore(ci): fix a command naming issue for the CI 2023-11-29 16:07:31 +01:00
Arthur Meyre
bdadd39a34 chore(ci): add some missing spec indicators for cargo commands 2023-11-29 15:29:06 +01:00
Arthur Meyre
f03f2f9c6d chore(ci): fix clippy trivium target which also triggered for tfhe 2023-11-29 15:29:06 +01:00
Arthur Meyre
f03ec9bbed chore(tfhe): re-allow semicolon_if_nothing_returned
- create a section for lints that have been considered to be disallowed but
are kept allowed, either because they bring too little value or because
they don't help with readability in certain circumstances
2023-11-29 15:28:43 +01:00
Arthur Meyre
5137751dd2 chore(ci): fix pedantic lint with missing trailing ; on () return 2023-11-29 15:28:43 +01:00
tmontaigu
edc3449dbf feat(integer): add signed overflowing_add 2023-11-29 13:02:58 +01:00
Arthur Meyre
6068c509de feat(tfhe): add LWE encryption and linalg with non power of 2 moduli 2023-11-29 09:54:55 +01:00
David Testé
30a4348e3a test(integer): use keycache for comparisons and wopbs 2023-11-28 14:11:53 +01:00
Mayeul@Zama
3013e02d90 fix(core): fix typo in comment 2023-11-28 13:10:23 +01:00
Mayeul@Zama
c029917c5c style(all): rename NB_TEST NB_TESTS 2023-11-28 13:10:23 +01:00
Mayeul@Zama
d23d04021b style(core): fix clippy::inconsistent_struct_constructor 2023-11-28 13:10:23 +01:00
Mayeul@Zama
5a5e9e0ac1 style(core): fix clippy::ptr_as_ptr 2023-11-28 13:10:23 +01:00
Mayeul@Zama
cc0a3bad8d style(core): fix clippy::iter_without_into_iter 2023-11-28 13:10:23 +01:00
Mayeul@Zama
1d12f60849 style(core): fix clippy::default_trait_access 2023-11-28 13:10:23 +01:00
Mayeul@Zama
a0db39c86e style(core): fix clippy::redundant_closure_for_method_calls 2023-11-28 13:10:23 +01:00
Mayeul@Zama
ef4558ac13 style(core): fix clippy::trivially_copy_pass_by_ref 2023-11-28 13:10:23 +01:00
Mayeul@Zama
bfb22b4531 style(core): fix clippy::needless_pass_by_value 2023-11-28 13:10:23 +01:00
Mayeul@Zama
88025010e1 style(core): fix clippy::unnecessary_wraps 2023-11-28 13:10:23 +01:00
Mayeul@Zama
b1f4f3b330 style(core): fix clippy::semicolon_if_nothing_returned 2023-11-28 13:10:23 +01:00
Mayeul@Zama
7575a426ab style(core): fix clippy::used_underscore_binding 2023-11-28 13:10:23 +01:00
Mayeul@Zama
1ac57218b1 style(core): fix clippy::manual_let_else 2023-11-28 13:10:23 +01:00
Mayeul@Zama
b7c3f16e24 style(core): fix clippy::implicit_clone 2023-11-28 13:10:23 +01:00
Mayeul@Zama
bf4f9198fb style(core): fix clippy::if_not_else 2023-11-28 13:10:23 +01:00
Mayeul@Zama
e618e1d05d style(core): fix clippy::uninlined_format_args 2023-11-28 13:10:23 +01:00
Mayeul@Zama
000428d688 style(core): replace assert by unwrap 2023-11-28 13:10:23 +01:00
Arthur Meyre
937c90666b chore(core): change the test LUT to apply the identity function
- the identity function more easily detects errors in the PBS as each mega
case contains a different value compared to its neighbours
2023-11-28 11:27:45 +01:00
sarah el kazdadi
b6a6f1b098 feat(core): specialize keyswitch implementation for small scalar values 2023-11-27 09:57:37 +01:00
David Testé
c2d7f1748c chore(ci): add core_crypto layer to code coverage 2023-11-22 10:21:17 +01:00
Mayeul@Zama
e8cd55dee6 feat(shortint): add degree information in CheckError::CarryFull 2023-11-21 19:52:26 +01:00
Mayeul@Zama
95aea9dbe8 feat(shortint): add noise checks 2023-11-21 19:52:26 +01:00
Mayeul@Zama
89f701d307 refactor(shortint): refactor CheckError 2023-11-21 19:52:26 +01:00
Mayeul@Zama
224146686f feat(shortint): add max_noise_level 2023-11-21 19:52:26 +01:00
Mayeul@Zama
b6b5f92220 fix(integer): update noise_level manually in direct calls to core_crypto 2023-11-21 19:52:26 +01:00
David Testé
0fec9e252b chore(ci): change benchmark aws ec2 machine type
This instance type hpc7a.96xlarge yields better performances for
nearly the same hourly cost.
2023-11-21 14:34:08 +01:00
Arthur Meyre
53c9b82824 chore(shortint): fix typo for NoiseLevel variant UNKNOWN 2023-11-20 18:54:14 +01:00
tmontaigu
f670a950d6 fix(integer): fix inner index computation in sum
In the function that sums a vec of ciphertexts,
we track trivial zeros to avoid un-needed PBSes.

One of this tracker is `last_block_where_addition_happened`
however it was not properly computed.
It was initialized to `num_blocks - 1`, and then got applied
a bunch of `max(current, new)` where 0 <= new <= num_block - 1
which means last_block_where_addition_happened was always num_blocks - 1.

The correct initial value is 0.
2023-11-20 16:40:21 +01:00
tmontaigu
a44970a9a3 feat(integer): avoid un-necessary computations in mul params 1_X
When the parameters have 1 bit of message (message modulus == 2)
then the multiplication of 2 blocks does not create a result that can
go into the carry space.

We use that fact to avoid doig un-necessary computations
when multiplying integers encrypted under parameters with 1 bit.
2023-11-20 15:21:59 +01:00
Arthur Meyre
55775b8e02 fix(shortint): fix overflow behavior of NoiseLevel
- we will need to use a MAX/UNKNOWN level for forward compatibility with
old serialized ciphertexts, this patch ensures the add/mul behavior
saturates properly to usize::MAX to force a refresh in operations which
do it automatically
2023-11-17 18:34:02 +01:00
Arthur Meyre
523d561de6 chore(ci): add _ci_run_filter to standalone tests in shortint
- those tests were likely ignored, this is no longer the case
2023-11-17 18:34:02 +01:00
tmontaigu
61a50d0bcc chore(integer): make oveflowing_add/sub return BooleanBlock 2023-11-17 16:22:20 +01:00
Arthur Meyre
ee57f5658b chore(ci): refactor integer script and skip div and rem preferring div_rem 2023-11-17 15:00:50 +01:00
tmontaigu
9362965f50 feat(integer): add accessors to inner shortint sks
Users can access blocks from an integer but they don't have
the ability to use the inner shortint server key to process
individual blocks.

This adds an AsRef impl on integer ServerKey to allow that.

This also adds shortcuts to the integer ServerKey to get
the MessageModulus/CarryModulus (these are shorticuts
because users could do `integer_key.as_ref().message_modulus`.
2023-11-16 16:25:27 +01:00
Arthur Meyre
00fb60451d chore(ci): group signed and unsigned integer for better runtime homogeneity 2023-11-16 14:18:30 +01:00
Arthur Meyre
18b9fd4464 chore(ci): re-enable mistakenly disabled AVX512 for integers 2023-11-16 14:18:30 +01:00
Arthur Meyre
eace0bfb85 chore(ci): spread tests between two CI machines/workflow for faster runtime 2023-11-16 14:18:30 +01:00
Arthur Meyre
af1be5ebca chore(core): fix noise generation which could overflow the custom modulus
- updated some function name (for modulus checking) to be clearer on what
they do and when to use them
2023-11-16 08:58:40 +01:00
tmontaigu
916bd8a09f feat(hlapi): move if_then_else/cmux to FheBool
- This makes FheBool use integer::BooleanBlock internally.
- It makes comparisons (eq, ne, le, etc) return a FheBool instead of
  FheUint/FheInt.
- It also moves the if_then_else and cmux methods to FheBool.
- Adds casting from FheBool to FheUint/FheInt (but not from
  FheUint/FheInt to FheBool as we expect users to do `a.ne(0)`
  as its matches Rust)

BREAKING CHANGE:
    - Comparisons now return FheBool
    - if_then_else/cmux are now methods of FheBool.
2023-11-15 23:22:30 +01:00
tmontaigu
20cb0642ce refactor(hlapi): implement CastFrom for GenericInteger
And add the trait to the prelude so that users can use
it.
2023-11-15 23:22:30 +01:00
Arthur Meyre
151f9f6d82 chore(ci): fix build on main following several big merges 2023-11-15 13:29:08 +01:00
Arthur Meyre
8db8cb49e4 chore(shortint): add some flaky/failing doctests as actual tests
- check that those are actually failing or that they are a doctest bug
- add _ci_run_filter so that we can easily make sure tests run in CI even
if they don't have the "parameter format"
2023-11-15 11:10:44 +01:00
Arthur Meyre
b4583976a2 chore(tfhe): fix .gitignore for key cache
- this was not properly ignoring the keycache if a file had a specific
extension
2023-11-15 11:10:30 +01:00
Arthur Meyre
b450375da1 chore(integer): restore assert after using 3_3 params for CRT doctests
- fix max degree for CRT keys which don't need to propagate carries

BREAKING CHANGE:
pub API removed from pub interface
2023-11-15 11:10:30 +01:00
tmontaigu
f02f1fb297 feat(integer): add unsigned_oveflowing_add 2023-11-14 18:57:09 +01:00
Mayeul@Zama
17642fa703 refactor(shortint): remove unused EngineResult 2023-11-14 16:30:09 +01:00
Mayeul@Zama
23fa9b24bd refactor(shortint): separate lut generation from ShortintEngine 2023-11-14 16:30:09 +01:00
tmontaigu
0453b9bd60 fix(integer): fix signed_overflowing_sub using trivial 0 2023-11-13 15:43:33 +01:00
Arthur Meyre
9b2cf67911 chore(tfhe): fix required features for the generate_test_keys util 2023-11-13 10:05:17 +01:00
dependabot[bot]
36a7656048 chore(deps): bump tj-actions/changed-files from 40.1.0 to 40.1.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 40.1.0 to 40.1.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](18c8a4eceb...25ef3926d1)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-13 09:58:27 +01:00
Arthur Meyre
61c8eadd58 chore(ci): update Makefile for semver trick compatibility
- adding the tfhe package as a dependency is currently causing issues with
Cargo because of unified feature resolution it seems, it needs an
additional version specifier to disambiguate which package we are referring
to, an issue exists on their end but I don't think a fix is to be expected
soon https://github.com/rust-lang/cargo/issues/12891
- commiting this to main and then backporting the relevant pieces to 0.4.x
2023-11-10 15:35:38 +01:00
Arthur Meyre
fdd4d9d1cc chore(c_api): add more comments in the build.rs file and cbindgen.toml 2023-11-10 15:35:38 +01:00
Arthur Meyre
62700ab853 chore(tfhe): clarify dependency vs feature selection 2023-11-10 15:35:38 +01:00
Arthur Meyre
27445645e7 chore(c_api): have a way to skip cbindgen in a semver trick setting 2023-11-10 15:35:38 +01:00
tmontaigu
ea0cd26c0b chore(tfhe): fix builds on main 2023-11-10 15:15:31 +01:00
David Testé
ff48582679 test(core_crypto): silence dead code warnings on test utils 2023-11-10 09:35:16 +01:00
tmontaigu
a77c87ff12 refactor(hlapi): make GenericInteger generic over the Id 2023-11-09 20:33:53 +01:00
tmontaigu
6d143f1edc refactor(hlapi): remove unused FromParameters trait 2023-11-09 20:33:53 +01:00
Arthur Meyre
216e6b443a chore(tfhe): fix pedantic lints 2023-11-09 17:12:00 +01:00
Arthur Meyre
1400ae946c test(tfhe): add uniform random test
- use DKW test, it is e.g. used in
https://github.com/wch/r-source/blob/trunk/tests/p-r-random-tests.R

See Wikipedia DKW inequality
2023-11-09 17:12:00 +01:00
Arthur Meyre
c332902a05 feat(core): add support for non power of 2 moduli for random generation
- add convenience function to get truncated f64 value of an integer modulus
- update trait bounds for random generation for clearer diagnostics
2023-11-09 17:12:00 +01:00
Arthur Meyre
cf7a7f132d chore(doc): update a slightly wrong docstring 2023-11-09 14:38:43 +01:00
tmontaigu
6e0a3b9ad7 feat(integer): add BooleanBlock wrapper type
The BooleanBlock wrapper type is meant to convey the fact that
the ciphertext encrypts a 0 or 1.

Since its meant to be a simple wrapper, the goal for is to be flexible
and not add more burden than usefulness.

Hopefully this implementation somehow achieves that

Breaking Changes:
 - This changes the return type of comparisons from a T to
   a BooleanBlock. Requiring existing code to explicitely convert
   using `.into_radix`.
 - This makes the cmux/if_then_else functions take a BooleanValue
   as the input type  Requiring existing code to wrap their condition
   ciphertext in a new BooleanValue
2023-11-08 19:40:21 +01:00
Arthur Meyre
1f825dde08 chore(tfhe): bump version to 0.5.0 2023-11-08 15:55:22 +01:00
tmontaigu
f9222de47c feat(integer): add signed_overflowing_sub 2023-11-08 15:11:05 +01:00
Mayeul@Zama
5732e8dd7a test(hlapi): test base and compressed integer conformance 2023-11-08 09:25:55 +01:00
Mayeul@Zama
9db35c5474 chore(clippy): remove useless #[allow(warning)] 2023-11-07 16:47:04 +01:00
Mayeul@Zama
b69f73e8e6 chore(clippy): fix use_self warnings 2023-11-07 16:47:04 +01:00
Mayeul@Zama
90bdf75147 chore(clippy): enable nursery lints 2023-11-07 16:47:04 +01:00
Mayeul@Zama
233ea17adf chore(clippy): enable pedantic lints 2023-11-07 16:47:04 +01:00
David Testé
df6ee79841 chore(ci): test examples and apps in the ci 2023-11-07 10:58:03 +01:00
Mayeul@Zama
6497fb9a15 feat(shortint): update noise level in operations 2023-11-06 11:33:24 +01:00
Mayeul@Zama
d8894e3b69 feat(shortint): add noise level to ciphertexts 2023-11-06 11:33:24 +01:00
dependabot[bot]
42636bab13 chore(deps): bump tj-actions/changed-files from 40.0.0 to 40.1.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 40.0.0 to 40.1.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](af292f1e84...18c8a4eceb)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-06 09:42:04 +01:00
tmontaigu
ec27d3dc6f refactor(hlapi): remove wrapping of booleans
This commit removes the wrapping of the `tfhe::boolean`
that was done in the HLAPI, effectively making the HLAPI
only wrapping `tfhe::integer`.

FheBool is now reused to be a single shortint block
compatible with other type FheUint8,16,etc (previously they were not).

In the future, `tfhe::boolean` could be re-wrapped in hlapi, but
this time, to be used as a base for all integers and not just
FheBool.

BREAKING CHANGE:
- hlapi no longer wraps tfhe::boolean API.
- tfhe::ConfigBuilder::enable_bool/disable_bool/all_disabled/all_enabled
  removed. Now default configuration should be done using
  `tfhe::ConfigBuilder::default()`.
- `tfhe::ConfigBuilder::use_default_small_integer` removed
  use `tfhe::CondifBuilder::default_with_small_encryption()`
- Uninitialied{ClientKey, PublicKey, CompressedPublicKey} error types
  removed as these erros are no longer possible
2023-11-04 00:18:16 +01:00
Mayeul@Zama
5272c95de4 fix(shortint): fix modulus on LUT output in test 2023-11-03 09:45:22 +01:00
Mayeul@Zama
27d7ace3ef feat(shortint): fix keyswitching wrapping behavior 2023-11-03 09:45:22 +01:00
Mayeul@Zama
d80ab231a8 fix(shortint): add LUT generation without carry 2023-11-03 09:45:22 +01:00
tmontaigu
fe3fa531f9 refactor(hlapi): Remove shortint support from HLAPI
This removes the wrapping of shortints from the HLAPI,
the reasons are:

Contrary to integers for which we have different bit size
by combining different number of blocks from the _same_ key.
shortints had different bit size, but also different keys
which lead to:

- Not being able to cast between 2 different shortint type
  and between 1 shortint and 1 integer. Technically these casts
  are possible, but requires a keyswitch (and likely a PBS).
  But the keyswitch requires parameters, which may not always exists.

- Due to each shortint having different keys, the internal code to
  manage that made heavy use of macros to avoid having thousands of
  repeated lines. However, this made the code harder to follow / modify
  especially for people that were not familiar with that.

- In practive to really benefit from shortints, proper management of
  carry space is needed, however the HLAPI completely hides that,
  resulting in less optimal performances. In short, shortints
  are better used as a low level construct.

- Building a FheUint4 with two block of message_2_carry_2
  is likely to be faster the one message_4_carry_4 for most use
  cases.

So removing the wrapping of shortints will simplify the code, and
allow for more simplification later.
Also, it will allow us to expose Fhe{Ui/I}nt{2, 4, 6} types
which are compatible (cast_from/into) with Fhe{Ui/I}nt{8, 16, 32, etc}.

BREAKING CHANGE:
    - FheUint{2,3,4} removed from HLAPI
    - All HLAPI functions thied to shortints are removed
2023-10-31 09:32:05 +01:00
tmontaigu
5c1573c266 fix(integer): fix worst case noise growth in encrypted shifts
In encrypted shifts we pack 3 bits from 3 different blocks into the same
blocks by doing `b0 * 4 + 2 * b1 + b2`, and then do a PBS to simulate a
hardware mux gate.

If the inputs of shift (ie, in lhs << rhs, lhs != rhs, ie we don't do
lhs << lhs) this is fine regarding the norm2 noise.

However if we do things like `a << a` or `a >> a`, which is probably a
very rare thing but not impossible, the norm2 noise would go above the
limit that guarantees our error probability.

To fix that, we extract the bits that tells shift amount, so that they
are already properly aligned to their mux input position.
The packing becomes `b0 + 2 * b1 + b2` and so,
the noise growth is ok even in the worst case of doind `a << a`.
2023-10-30 15:02:02 +01:00
dependabot[bot]
7772e8112d chore(deps): bump tj-actions/changed-files from 39.2.3 to 40.0.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.2.3 to 40.0.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](95690f9ece...af292f1e84)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-30 12:07:44 +01:00
dependabot[bot]
5e92cb1475 chore(deps): bump JS-DevTools/npm-publish from 3.0.0 to 3.0.1
Bumps [JS-DevTools/npm-publish](https://github.com/js-devtools/npm-publish) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/js-devtools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](6fd3bc8dad...4b07b26a2f)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-30 12:07:24 +01:00
dependabot[bot]
f51e19b071 chore(deps): bump actions/checkout from 4.1.0 to 4.1.1
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4.1.0...b4ffde65f46336ab88eb53be808477a3936bae11)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-30 12:06:53 +01:00
tmontaigu
aeb00ae584 chore(integer): use Arc<ServerKey> for executor
The goal is to avoid holding the key twice in memory
when both the executor and the test case needs the key
2023-10-27 18:01:55 +02:00
Arthur Meyre
ce5e9c1bdb chore(integer): more CRT tests and related fixes
- add remaining tests
- fix unchecked scalar mul for small carries
2023-10-27 11:30:00 +02:00
Arthur Meyre
4d4e124e94 chore(integer): add crt 32 bits tests with 5_1 params
- remove buggy unchecked_scalar_add_assign and replace by the proper
implementation which had a different name

BREAKING CHANGE:
removed an API entry point which was not required
2023-10-27 11:30:00 +02:00
Arthur Meyre
ca6d37e06f feat(integer): better handle trivial 0 blocks from LHS
- currently the filter only applied to the RHS but LHS can also benefit
from the filter
2023-10-27 10:31:24 +02:00
Mayeul@Zama
e3143315f3 fix(integer): disable broken assert in smart_crt_sub_assign 2023-10-27 09:43:51 +02:00
Mayeul@Zama
f8636fe814 feat(integer): add asserts in smart ops 2023-10-27 09:43:51 +02:00
tmontaigu
7e72400321 chore(doc): replace some ^ which could be interpreted as xor not pow 2023-10-26 23:42:58 +02:00
tmontaigu
728b409256 chore(integer): move comparator test out of it
Move the comparisons test (eq, ne, ge, gt, etc)
that were in the comparator module out of the comparator module.

This is so that in later commits will create test cases out
of these tests so they can, like other unsigned tests be
used to test other implementations of ServerKey
2023-10-25 10:31:55 +02:00
Arthur Meyre
d91404e567 chore(integer): remove empty where clause 2023-10-25 09:41:37 +02:00
David Testé
e11c3d7b7c chore(ci): add signed integer benchmarks to the CI 2023-10-25 09:14:00 +02:00
David Testé
6f8eeb043c chore(bench): add default ops for singed integers benchmarks 2023-10-25 09:14:00 +02:00
Arthur Meyre
00d55182b4 chore(ci): update examples to have a tmp dir to avoid rights issues in /tmp
- on machines where multiple users can log in, some files used for
serialization doctests would cause rights access issues and crash doctests
2023-10-23 15:03:18 +02:00
dependabot[bot]
6f6ce106c3 chore(deps): bump tj-actions/changed-files from 39.2.2 to 39.2.3
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.2.2 to 39.2.3.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](408093d9ff...95690f9ece)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-23 10:28:00 +02:00
dependabot[bot]
68fcbb5280 chore(deps): bump JS-DevTools/npm-publish from 2.2.2 to 3.0.0
Bumps [JS-DevTools/npm-publish](https://github.com/js-devtools/npm-publish) from 2.2.2 to 3.0.0.
- [Release notes](https://github.com/js-devtools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](fe72237be0...6fd3bc8dad)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-23 10:27:35 +02:00
dependabot[bot]
3f46389cc8 chore(deps): bump actions/checkout from 4.1.0 to 4.1.1
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](8ade135a41...b4ffde65f4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-23 10:27:10 +02:00
Arthur Meyre
9e8dd01cb9 chore(ci): enable integer multi bit tests on M1 2023-10-20 17:55:38 +02:00
tmontaigu
0085ceb97b chore(ci): set node version 2023-10-20 10:24:50 +02:00
tmontaigu
be9a4d2d9c chore(wasm): update dependencies of wasm tests 2023-10-20 10:24:50 +02:00
Arthur Meyre
87421e8307 chore(ci): update M1 workflow to not explode the 6h GitHub limit
- run doc tests for CI with LTO off following M1 investigation
- LTO fat may be a cause of the wopbs flaky tests, disabling to check
2023-10-19 14:18:05 +02:00
Arthur Meyre
0c3919628f refactor(core): use avx512 intrinsics when available for data conversions
- we use inline assembly for now as rust does not propose those in the std
or core arch crates at the moment
- add tests for avx512 conversion
2023-10-19 13:21:19 +02:00
Arthur Meyre
f1c21888a7 chore(doc): encourage users to use dedicated keys to Radix or CRT 2023-10-19 09:52:22 +02:00
tmontaigu
2624beb7fa fix(integer): fix unsigned_overflowing_sub on trivials
unsigned_overflowing_sub does an independant subtraction
on each blocks with a correcting term being added to avoid
trashing the padding bit (lhs - rhs + correction).

The correction depended on rhs's degree.
e.g. if rhs's degree was in range 1..(msg_mod-1) -> correction =
     msg_mod

However if rhs's degree was zero (so rhs is a trivial 0), the correction
was also 0, however the borrow propagation rely on that correction to
always be added.
2023-10-18 19:26:01 +02:00
tmontaigu
e44c38a102 chore(ci): tell nvm to use node version 20 in wasm parallel tests 2023-10-18 19:04:27 +02:00
Arthur Meyre
4535230874 refactor(core): rename pbs_modulus_switch to fast_pbs_modulus_switch
- update docstring to reflect the change that has been done

BREAKING CHANGE:
pbs_modulus_switch is currently part of the public API and the rename is
therefore a breaking change
2023-10-17 16:53:19 +02:00
Arthur Meyre
a7b2d9b228 chore(ci): update check toolchain to latest nightly
- no new lints
2023-10-17 16:13:26 +02:00
Arthur Meyre
ab923a3ebc fix(crt): fix mul for non symmetrical parameters
- add non reg test for 32 bits mul with 5_1 parameters
2023-10-17 14:22:00 +02:00
Arthur Meyre
a0e85fb355 feat(core): add more custom moduli primitives to UnsignedInteger
As always for now the objective is to have functional custom modulus
implementations, not efficient ones

- add multiplication
- add leading_zeros
- add neg
2023-10-17 13:31:35 +02:00
Arthur Meyre
ecee305340 chore(core): change prelude algorithms imports 2023-10-17 13:31:35 +02:00
Mayeul@Zama
f08ea8cf85 fix(integer): fix max_degree formula 2023-10-17 11:35:08 +02:00
Mayeul@Zama
096e320b97 fix(crt): use 3_3 parameters for crt tests 2023-10-17 11:35:08 +02:00
Mayeul@Zama
95aac64c1c style(crt): compute modulus from base in tests 2023-10-17 11:35:08 +02:00
Mayeul@Zama
76aaa56691 fix(integer): fix small mul test 2023-10-17 11:35:08 +02:00
Mayeul@Zama
a40489bdd2 style(shortint): do not use assign ops on a cloned input 2023-10-17 11:35:08 +02:00
Mayeul@Zama
4bf617eb10 feat(shortint): cleanup input if necessary in ops 2023-10-17 11:35:08 +02:00
Mayeul@Zama
070073d229 feat(shortint): cleanup input if necessary in apply_lookup_table_bivariate 2023-10-17 11:35:08 +02:00
Arthur Meyre
6c1ca8e32b chore(core): use modular_distance instead of abs_diff in fft tests
- we are doing backwards conversions to the torus, so values could wrap
around near 0 or u64::MAX, take the modular distance which represents the
distance on the torus
2023-10-17 10:29:24 +02:00
Arthur Meyre
6523610ca4 refactor(core): refactor conversion code from f64 to i64
- observed that the subnormal case is already handled by the shift logic so
the special handling was not required
- add test for avx512 conversion
2023-10-17 10:29:24 +02:00
Arthur Meyre
41c20e22f5 chore(ci): enable AVX512 for integer and multi bit integer tests 2023-10-17 10:28:14 +02:00
J-B Orfila
4a00d25cb1 doc: updating doc for v0.4 2023-10-16 17:56:17 +02:00
tmontaigu
8c9ee64612 fix(integer): better estimate which algorithm to choose 2023-10-16 16:19:00 +02:00
tmontaigu
bfdfbfac0f chore(integer): add tests for default signed rotations/shifts 2023-10-16 16:16:07 +02:00
tmontaigu
dbe7bdcd5c feat(integer): map cmux to if_then_else 2023-10-16 16:15:49 +02:00
tmontaigu
6d77ff18ad chore(integer): add full_propagate test 2023-10-16 14:11:44 +02:00
tmontaigu
7d4d0e0b16 fix(integer): fix is_scalar_add_possible 2023-10-16 14:11:44 +02:00
Mayeul@Zama
b27762232c feat(wasm): add integers safe deserialization 2023-10-16 10:19:09 +02:00
Mayeul@Zama
f597d0f06f feat(c_api): add base and compress integers safe deserialization 2023-10-16 10:19:09 +02:00
dependabot[bot]
ee188448f3 chore(deps): bump tj-actions/changed-files from 39.2.1 to 39.2.2
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.2.1 to 39.2.2.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](db153baf73...408093d9ff)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-16 09:51:26 +02:00
Mayeul@Zama
ee49f048c7 style(integer): rename num_blocks_per_integer 2023-10-13 14:18:44 +02:00
Mayeul@Zama
a9b09ecc45 feat(c_api): add compact integer safe deserialization 2023-10-13 14:18:44 +02:00
Mayeul@Zama
efc243edc9 feat(global): refactor ciphertext conformance 2023-10-13 14:18:44 +02:00
tmontaigu
bc34411d3f feat(integer): speed-up division by using overflowing_sub
using overflowing sub allows to remove the comparison used
in the algorithm, giving significant performance boost.

before
             8        16       32      40       64       128        256
hpc7a:    `981 ms` `2.53 s` `6.41 s` `9.04 s` `16.1 s` `39.3 s` `1.55 min`
m6i:      `1.10 s` `2.97 s` `7.17 s` `10.5 s` `19.7 s` `50.2 s` `2.11 min`

afer:
             8        16       32      40       64       128        256
hpc7a:   `604 ms` `1.6 s`  `3.8 s`  `5.14 s` `9.4 s`  `22.4 s`  `54.613 s`
m6i:     `659 ms` `1.77 s` `4.4 s`  `5.9 s`  `11.5 s` `29.8 s`  `87.95 s`
2023-10-12 14:35:36 +02:00
J-B Orfila
c7923ff3ed refactor(shortint): update compact parameters 2023-10-12 11:56:50 +02:00
Arthur Meyre
7534b68e5c test(core): use polynomial tests from NTT PR
- initial work done in https://github.com/zama-ai/tfhe-rs/pull/394
- useful reworks of the tests have been waiting in that PR, this is to
have those tests while NTT usage gets validated

co-authored-by: sarah-ek <sarah.elkazdadi@zama.ai>
2023-10-12 10:40:15 +02:00
tmontaigu
655f7e6214 chore(hlapi): improve scalar type convertion 2023-10-10 17:18:32 +02:00
tmontaigu
b8556ddbd4 feat(hlapi): add C API support for FheInt 2023-10-10 17:18:32 +02:00
tmontaigu
cab7439064 fix(integer): handle trivial ct in if_then_else
if_then_else uses two calls to zero_out_if.

In zero_out_if, if the condition block given has a degree of 0
then it would return 0, without calling the predicate function.

This is not correct, as its the predicate function that
gives whether the output should be 0 or the original ciphertext.

Which meant that if if_then_else received a condition with a
degree of 0, it would always return 0.
2023-10-10 17:18:12 +02:00
tmontaigu
f8a8780651 fix(integer): remove remove if_then_else assert
unchecked_if_then_else had an assert that required
that the condition value looked like it encrypts a boolean.
This check was made using the degree.

However, the only cases where a value looks like it encrypts a boolean
value is when they are the result of a comparison (lt, le, eq, etc).

But there are other cases were the value holds a boolean value but
due to how degree works, it's not possible to know thus limiting the
use of if_then_else.

So we remove that assert, and rely on the developper knowing
its condition is 0 or 1.
2023-10-09 18:35:26 +02:00
tmontaigu
bb3c8e7d5d feat(integer): add unsigned_overflowing_sub 2023-10-09 15:39:41 +02:00
Arthur Meyre
69536960c3 chore: fix typos 2023-10-09 14:49:13 +02:00
dependabot[bot]
52a7c52a49 chore(deps): bump tj-actions/changed-files from 39.2.0 to 39.2.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.2.0 to 39.2.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](8238a41032...db153baf73)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-09 10:21:04 +02:00
tmontaigu
751c407ba5 feat(wasm): add FheInt support 2023-10-05 15:52:45 +02:00
Mayeul@Zama
492d348138 test(serialization): run tests in CI 2023-10-05 09:15:58 +02:00
Mayeul@Zama
e7df7eb5ef test(serialization): add serialization test 2023-10-05 09:15:58 +02:00
Mayeul@Zama
380ee52986 test(hlapi): test compact integer conformance 2023-10-05 09:15:58 +02:00
Mayeul@Zama
439a28f68b feat(global): impl ParameterSetConformant for ciphertexts 2023-10-05 09:15:58 +02:00
Mayeul@Zama
2eb1e37ca7 feat(global): add safe deserialization 2023-10-05 09:15:58 +02:00
Mayeul@Zama
eb1b136c45 feat(core): add to_equivalent_lwe_dimension 2023-10-05 09:15:58 +02:00
Mayeul@Zama
1376bcba7c chore(test): add type hint for rust-analyzer 2023-10-05 09:15:58 +02:00
tmontaigu
b5b4e54b9b feat(hlapi): add FheInt{8,16,32,64,128,256} 2023-10-04 20:41:19 +02:00
Arthur Meyre
23c2bd790a chore(test): fix incorrect memory buffer size in wopbs core_crypto tests 2023-10-04 14:17:33 +02:00
tmontaigu
251ee9aa0e chore(hlapi): add InnerCiphertext type to integer wrapper
Make the GenericInteger struct have a generic `InnerCiphertext`
instead of always RadixCiphertext.

This is to prepare the addition of signed types which will use a
SignedRadixCiphertext.
2023-10-03 16:26:09 +02:00
Arthur Meyre
fad066a996 refactor(core): remove a copy in the external product
- add an fft backward primitive that can use the input fourier buffer as
output as well
- gains 0.6 ms on 2_2 m6i.metal
2023-10-03 13:10:01 +02:00
tmontaigu
6ef1f22b33 feat(hlapi): tie scalar ops with corresponding clear type
Operations that used a scalar as right operand where generically
implemented meaning a user could, for example, add a u32 to a FheUint8.

Rust only allows operations between matching types, so we do the same
thing.

BREAKING CHANGE: This is a breaking change on the Rust API, but
for the better I believe. On the C API it is not a breaking change
as we already made that association as it was simpler to implement
2023-10-02 23:17:30 +02:00
tmontaigu
8cc8dba1ab feat(integer): add encryption of signed radix via compressed pk 2023-10-02 16:02:36 +02:00
tmontaigu
082328c91a feat(integer): add default signed_scalar div/rem/div_rem 2023-10-02 16:02:18 +02:00
tmontaigu
fdb6faa0a8 fix(integer): clean output quotient of division
The quotient was slowly computed by
getting a resut bit, shifting it to its position then adding it
to a quotient block, i.e quotient += bit << pos;

This meant that the output quotient was noisy, too noisy for
parameters like param_message_4_carry_4, and so the signed division
would then negate and cmux this quotient and due to the high noise,
some computations would fail, on param_message_4_carry_4.

To fix this we clean the quotient's noise before returning it.
2023-10-02 08:48:45 +02:00
Arthur Meyre
856440386f chore(csprng): the stabilized aarch64 intrisics were in Rust 1.72
- update the version accordingly
2023-09-29 18:33:39 +02:00
tmontaigu
2e8189514c feat(integer): make compact ciphertext compatible with signed 2023-09-28 20:41:38 +02:00
tmontaigu
29b2454cce feat(integer): add sign extend fn for SignedRadixCiphertext 2023-09-28 17:48:41 +02:00
tmontaigu
9ed2589c7a chore(integer): impl RecomposableSignedInteger for StaticSignedBigInt 2023-09-28 14:01:14 +02:00
tmontaigu
36b71529e6 chore(integer): make tests work with different ServerKey
This is a first step, a second step would be
to plug the non parallel radix tests so that
they are testing the same things.
2023-09-28 10:50:18 +02:00
Arthur Meyre
b738946d72 chore(core): add utils to test noise distribution for power of 2 q 2023-09-28 09:49:30 +02:00
David Testé
62f1425257 chore(bench): add missing unsigned integer operations 2023-09-28 08:47:39 +02:00
David Testé
44e491b93f style(integer): rename absolute_value functions to abs
Also add _parallelized suffix since the implementation is located in
radix_parallel directory.
2023-09-28 08:47:39 +02:00
tmontaigu
a470b26672 fix(integer): StaticSignedBigInt right shift 2023-09-27 18:37:25 +02:00
tmontaigu
015409424c chore(hlapi): remove unused keychain_member from macro 2023-09-27 14:33:24 +02:00
tmontaigu
37be751188 fix(integer): is_neg/sub/add possible
The way we did the is_neg/add/sub possible at the integer level was
incorrect in two ways.

1) We simply called the is_neg/add/sub_possible from
   the shortint impl on each block as if the were independant.
   However that is not the case, and to the check did not reflect
   actual computation.

2) We checked that we did not go beyond max degree on each block,
   However, a more correct approach would be to check that adding
   the potential carry from preceding block would not exceeding the
   current block max capacity.
2023-09-26 16:02:15 +02:00
sarah el kazdadi
2580a834af feat(core): optimize monic polynomial operations in pbs 2023-09-26 15:02:33 +02:00
David Testé
a029bd878e chore(ci): fix file exclusion for coverage reports 2023-09-26 08:58:36 +02:00
David Testé
400e7930b6 chore(ci): fix options typos for new tarpaulin version 2023-09-26 08:58:36 +02:00
David Testé
40d07c6bc3 chore(ci): speed-up boolean coverage
This is done by reducing the number of parameters set run in tests.
Using the keycache for the key switching key and public key tests also
help to reduce total run duration.
2023-09-26 08:58:36 +02:00
Mayeul@Zama
9dd2d39f1c style(global): fix typos 2023-09-25 17:27:29 +02:00
dependabot[bot]
4045a3bc2f chore(deps): bump tj-actions/changed-files from 39.0.2 to 39.2.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.0.2 to 39.2.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](6ee9cdc581...8238a41032)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-25 10:30:28 +02:00
dependabot[bot]
b4ffeccd46 chore(deps): bump actions/checkout from 4.0.0 to 4.1.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](3df4ab11eb...8ade135a41)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-25 10:30:22 +02:00
tmontaigu
7fe3ad3b6e feat(integer): improve scalar_mul
This changes the algorithm for scalar_mul.
The new algorithm allows to remove a lot of work.

For small precisions (16, 32, 64) the gains are in range 5%-10%
for higher precisions the gains are 25%-50%.

This also changes the mul to use the functions that sums many
clean ciphertexts in parallel. For mul, there is only a 5%-10%
improvements for 128bits and 256bits mul.
2023-09-22 15:45:07 +02:00
tmontaigu
7fdd4f9532 chore(integer): add default signed bitand/or/xor tests 2023-09-22 14:50:27 +02:00
Arthur Meyre
81eef39ddb feat(core): add parallel variant of extract_lwe_sample_from_glwe
- allows to quickly extract all coefficients packed in a GLWE cipehrtext
2023-09-22 10:55:02 +02:00
tmontaigu
b6459e3cda fix(integer): fix signed div_rem test for 0/0 2023-09-21 21:38:16 +02:00
Arthur Meyre
f2ef78c348 refactor(core): simplify closest_representable and pbs_modulus_switch
- both code were selecting the bit below the last representable bit,
extracted it and then added it to the bit above, the same effect can be
achieved by adding a 1 at the bit below the last representable bit
- update closest_representable to use an approach more like
pbs_modulus_switch yielding assembly with 42% less instructions (12 -> 7)
2023-09-21 15:54:53 +02:00
Mayeul@Zama
aef8f31621 chore(deps): update cargo dependencies 2023-09-21 15:11:13 +02:00
sarah el kazdadi
df78d178da fix(integer): replace unnecessary unsafe code in integer shift/add 2023-09-21 11:02:41 +02:00
Arthur Meyre
9297a886a4 chore(docs): fix docstring about encryption key choice 2023-09-20 16:02:55 +02:00
tmontaigu
28b4f91a32 fix(integer): only propagate if necessary after trimming
By unconditionally propagating carries after trimming
we would sometimes do work for nothing, and as propagating
carries is not cheap at all it would degrade performances.

So only propagate when necessary
2023-09-20 15:57:33 +02:00
David Testé
04fb46e41b chore(ci): print security level in parameters check
The devo profile is used to speed up the compilation phase.
2023-09-20 15:33:39 +02:00
David Testé
53da809f37 chore(ci): reduce max dimension threshold in lattice estimator 2023-09-20 09:39:50 +02:00
David Testé
723910c669 chore(ci): fix end-of-file newlines 2023-09-20 09:39:50 +02:00
David Testé
8ecf8879fb chore(ci): add end-of-file newline checks recipe 2023-09-20 09:39:50 +02:00
tmontaigu
2427f744f8 feat(integer): add unchecked implementation of signed ciphertext 2023-09-20 08:50:15 +02:00
Arthur Meyre
422e1f23d5 feat(core): add GLWE linear algebra primitives
- add appropriate tests and doctest
2023-09-19 11:41:16 +02:00
sarah el kazdadi
30a5ade17f fix(csprng): enable target_feature attributes for functions using simd intrinsics 2023-09-19 09:19:47 +02:00
tmontaigu
6cdd41c22f fix(integer): fix is_neg_possible
shortint's is_neg_possible did not check the degree on the correct value.
It check the degree on the value that should be added to the next block
not on the value that actually becomes the degree.

integer's is neg possible had the same problem so we also fix it
and also check the next block can 'receive' the value that should be added to it.

Our tests did not catch that as they were not testing non empty carry case
2023-09-18 17:19:48 +02:00
Mayeul@Zama
f369bec394 feat(core): add par_convert_standard_lwe_multi_bit_bootstrap_key_to_fourier 2023-09-18 14:35:06 +02:00
Mayeul@Zama
df4e9c69c7 feat(core): add par_convert_bootstrap_key_fourier 2023-09-18 14:35:06 +02:00
Mayeul@Zama
0e3d129906 feat(core): add par_fill_with_forward_fourier 2023-09-18 14:35:06 +02:00
Mayeul@Zama
682e455c94 feat(core): add par_convert_polynomials_list_to_fourier 2023-09-18 14:35:06 +02:00
Mayeul@Zama
b553a68fa9 chore(docs): simplify improved formula in dark market 2023-09-18 09:58:13 +02:00
Mayeul@Zama
be95eadf79 chore(docs): remove fallible directory change 2023-09-18 09:58:13 +02:00
Mayeul@Zama
0213a11a0c chore(docs): refactor dark market 2023-09-18 09:58:13 +02:00
Mayeul@Zama
413fde3b3b chore(docs): doc fixes and improvements 2023-09-18 09:58:13 +02:00
sarah el kazdadi
40f8ac9adf feat(core): replace unsafe simd intrinsics 2023-09-18 09:30:17 +02:00
dependabot[bot]
2ab25c1084 chore(deps): bump rtCamp/action-slack-notify from 2.2.0 to 2.2.1
Bumps [rtCamp/action-slack-notify](https://github.com/rtcamp/action-slack-notify) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/rtcamp/action-slack-notify/releases)
- [Commits](https://github.com/rtcamp/action-slack-notify/compare/v2.2.0...b24d75fe0e728a4bf9fc42ee217caa686d141ee8)

---
updated-dependencies:
- dependency-name: rtCamp/action-slack-notify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 09:25:56 +02:00
dependabot[bot]
86c62b70e5 chore(deps): bump JS-DevTools/npm-publish from 2.2.1 to 2.2.2
Bumps [JS-DevTools/npm-publish](https://github.com/js-devtools/npm-publish) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/js-devtools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](5a85faf05d...fe72237be0)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 09:25:49 +02:00
dependabot[bot]
18d790fc26 chore(deps): bump actions-ecosystem/action-add-labels
Bumps [actions-ecosystem/action-add-labels](https://github.com/actions-ecosystem/action-add-labels) from 1.1.0 to 1.1.3.
- [Release notes](https://github.com/actions-ecosystem/action-add-labels/releases)
- [Commits](bd52874380...18f1af5e35)

---
updated-dependencies:
- dependency-name: actions-ecosystem/action-add-labels
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 09:25:38 +02:00
dependabot[bot]
e9e3dae786 chore(deps): bump actions/checkout from 3.5.3 to 4.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.3 to 4.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3.5.3...3df4ab11eba7bda6032a0b82a6bb43b11571feac)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 09:25:33 +02:00
dependabot[bot]
9b1dccbcb4 chore(deps): bump tj-actions/changed-files from 39.0.1 to 39.0.2
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39.0.1 to 39.0.2.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](246636f5fa...6ee9cdc581)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-18 09:25:28 +02:00
David Testé
cef011dd91 chore(ci): add compact keys to parameters security checks 2023-09-18 09:12:51 +02:00
Arthur Meyre
19f7d5af5c chore(test): remove some unused imports 2023-09-15 18:27:57 +02:00
Arthur Meyre
95ca5a80dc feat(tfhe): plug parallel decompression in higher level APIs
- boolean, shortint and integer have been updated to benefit from paralle
decompression
2023-09-15 18:27:57 +02:00
Arthur Meyre
b5fded34d1 feat(core): add multi bit BSK parallel decompression
- added decompression equivalence test
2023-09-15 18:27:57 +02:00
Arthur Meyre
0c3b09c83d chore(core): update multi bit BSK decompression to match encryption
- test passes
2023-09-15 18:27:57 +02:00
Arthur Meyre
85a19d30a9 feat(core): add KSK parallel decompression
- update to check decompression equivalence
2023-09-15 18:27:57 +02:00
Arthur Meyre
f58132c391 feat(core): add GGSW par decompression, add LWE BSK par decompression
- add par decompression equivalency test
2023-09-15 18:27:57 +02:00
Arthur Meyre
099bff84aa refactor(core): use forking to decompress GGSW ciphertext list
- existing BSK equivalency test passes which means the change is compatible
2023-09-15 18:27:57 +02:00
Arthur Meyre
42ad474a46 feat(core): add parallel decompression to GGSW ciphertext
- added equivalence test for parallel decompression
2023-09-15 18:27:57 +02:00
Arthur Meyre
9f6827b803 chore(core): update code to use the newly introduced MaskRandomGenerator 2023-09-15 18:27:57 +02:00
Arthur Meyre
d23c0df449 refactor(core): update decompression code for LweCiphertextList
- update related algorithms
2023-09-15 18:27:57 +02:00
Arthur Meyre
229bfeebe4 chore(core): remove unsafe new_unchecked on CiphertextModulus
- the function has been renamed to new and is now generally available
2023-09-15 18:27:16 +02:00
David Testé
48aab9d494 chore(ci): add boolean layer to code coverage 2023-09-15 11:00:23 +02:00
David Testé
e4769a8212 chore(ci): do not trigger code coverage on pr sync
Automatic code coverage will be enable again, once all the layers of the
library have coverage implemented.
2023-09-15 08:33:10 +02:00
David Testé
79bdaaba20 chore(ci): disable codecov patch status
This is done to avoid noisy reports in GitHub since coverage in all the
library layers haven't been implemented yet.
2023-09-15 08:33:10 +02:00
Arthur Meyre
02a14fff7c feat(core): add parallel LWE packing keyswitch
- update test to check equivalence of parallel and serial algorithm
2023-09-14 14:45:35 +02:00
Arthur Meyre
72cce4c5b2 chore(core): move thread_count computation before buffer allocations
- for parallel LWE KS and LWE PFPKS
- remove useless type annotation as well
2023-09-14 14:45:35 +02:00
David Testé
a317c4b9dd chore(ci): run code coverage workflow on aws ec2 instance 2023-09-14 13:32:09 +02:00
David Testé
2e2bd5ba29 chore(ci): use aws ami with missing packages installed
libssl-dev and pkg-config packages were missing to be able to install
cargo tarpaulin.
2023-09-14 13:32:09 +02:00
David Testé
827d8d8708 chore(ci): run coverage only if source files have changed 2023-09-14 13:32:09 +02:00
David Testé
bf434be347 chore(ci): exclude unwanted files from coverage 2023-09-14 13:32:09 +02:00
Arthur Meyre
ed83fbb460 chore(tfhe): remove unused deps, drop once_cell, enable paste when needed
- bump to 1.72 for std lib OnceLock and stabilized ARM intrisics
2023-09-14 10:52:34 +02:00
David Testé
0aad2e669b chore(ci): notify slack about coverage only in case of failure 2023-09-13 15:25:32 +02:00
David Testé
cd68a3bd1c chore(ci): execute clippy all-targets on benchmark code
The use of internal-keycache feature is mandatory to ensure clippy
is building against benchmark code.
2023-09-13 11:49:33 +02:00
Arthur Meyre
b77286bcbc chore(bench): fix pbs bench code which had issues with name type
- clippy will be hardened in a subsequent commit
2023-09-13 11:49:33 +02:00
Mayeul@Zama
609f83bbff style(shortint): smart always take mut references 2023-09-13 09:44:46 +02:00
David Testé
2a8ebb81d8 chore(ci): fix trivium test recipes 2023-09-13 09:09:39 +02:00
David Testé
1a2a17a6ab chore(ci): avoid running full suite several times on approval
If two or more reviewers approve a Pull Request successively with
no new commits in between, the full test suite would have been run
for each approval. With this commit, the full test suite would be
run again upon approval only if a push has occurred.
2023-09-13 09:09:20 +02:00
David Testé
0080caf95d chore(ci): add code coverage workflow
Coverage is performed and then data are sent to Codecov to handle
reports.
2023-09-13 09:09:03 +02:00
David Testé
c26238533b chore(ci): add key cache keys generation for coverage 2023-09-13 09:09:03 +02:00
David Testé
b29936d844 test(shortint): add key switching key handling in keycache
This mainly done to speed up coverage test by avoiding generating
key on each tests.
2023-09-13 09:09:03 +02:00
David Testé
25914cc727 test(shortint): make tests suite lighter for coverage target 2023-09-13 09:09:03 +02:00
David Testé
ca229e369b chore(ci): add make recipe for shortint test coverage
Tarpaulin is the cargo module used to test coverage.
2023-09-13 09:09:03 +02:00
Asher
4a99e54c0d chore(docs): typo in readme 2023-09-12 18:34:06 +02:00
tmontaigu
2383591351 chore(hlapi): move integrations tests into src dir 2023-09-12 10:32:42 +02:00
tmontaigu
dc464f398d chore(trivium): fix module inception 2023-09-12 10:32:26 +02:00
Mayeul@Zama
ce70b5758a refactor(shortint): replace apply_msg_identity_lut_assign by message_extract 2023-09-11 17:38:15 +02:00
Mayeul@Zama
1c76a08373 refactor(shortint): replace clear_carry by message_extract 2023-09-11 17:38:15 +02:00
Mayeul@Zama
9b19bd1e8b refactor(shortint): remove comp_assign operations 2023-09-11 17:38:15 +02:00
Arthur Meyre
a3dde21240 refactor(core): add NoiseRandomGenerator
- update EncryptionRandomGenerator to make use of that generator
2023-09-11 17:29:47 +02:00
Arthur Meyre
005e1afe2f refactor(core): add MaskRandomGenerator
- update EncryptionRandomGenerator to make use of that generator
- remove fork_n and par_fork_n on EncryptionRandomGenerator which were lazy
shortcuts to get multi bit PBS working faster
- MaskRandomGenerator will be usable for parallel decompression, which will
be done in a subsequent commit
2023-09-11 17:29:47 +02:00
Arthur Meyre
17c404b77d chore(core): re-organize encryption.rs to have smaller files
- preparatory work for refactor around mask and noise generators to allow
parallel decompression
2023-09-11 17:29:47 +02:00
Arthur Meyre
1403971d15 feat(core): add parallel LWE PFPKS and LWE KS
- add utility algorithms where applicable
2023-09-11 13:25:43 +02:00
Arthur Meyre
94ad69bfa3 chore(docs): update toolchain requirements 2023-09-11 13:05:33 +02:00
Arthur Meyre
bc129ba0ed chore(csprng): add dieharder test suite 2023-09-11 13:05:33 +02:00
Arthur Meyre
462834a12e chore(tfhe): use concrete-csprng 0.4.0 allows to use stable for M1/M2 macs 2023-09-11 13:05:33 +02:00
Arthur Meyre
ebeee1d6f8 chore(ci): add concrete-csprng to the CI 2023-09-11 13:05:33 +02:00
Arthur Meyre
d0e1a582e1 chore(csprng): add code base taken from concrete-core repo 2023-09-11 13:05:33 +02:00
Arthur Meyre
546cb369a8 chore(c_api): mark some functions manipulating pointers as unsafe
- restrict visibility to the c_api

BREAKING CHANGE:
change in visibility specifier is a breaking change though those functions
were not meant to be used by external users
2023-09-11 12:49:57 +02:00
dependabot[bot]
445af7ab97 chore(deps): bump tj-actions/changed-files from 38.2.1 to 39.0.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 38.2.1 to 39.0.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](2f7246cb26...246636f5fa)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:05:38 +02:00
dependabot[bot]
23f8c69bae chore(deps): bump actions/upload-artifact from 3.1.2 to 3.1.3
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](0b7f8abb15...a8a3f3ad30)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:05:29 +02:00
dependabot[bot]
b8df207b68 chore(deps): bump actions/checkout from 3.6.0 to 4.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.6.0 to 4.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](f43a0e5ff2...3df4ab11eb)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-11 10:05:17 +02:00
sarah el kazdadi
03688aee4c feat(fft): use monomial fft for multibit pbs 2023-09-08 18:27:43 +02:00
David Testé
5a3652f398 chore(ci): add clippy target for trivium application 2023-09-08 16:49:27 +02:00
David Testé
ae3c261d1e chore(bench): add 128 and 256 bit sizes for multi-bit benchmarks 2023-09-08 13:25:28 +02:00
tmontaigu
95dcf95e88 fix(integer): make trim_radix_msb handle dirty inputs 2023-09-08 10:29:25 +02:00
J-B Orfila
b92d6400f4 chore(bench): add new parameter sets to bench 2023-09-07 14:49:20 +02:00
David Testé
9ba27b4082 chore(ci): add multiplier to get effective pbs throughput value 2023-09-06 11:02:47 +02:00
dependabot[bot]
b164c90d75 chore(deps): bump tj-actions/changed-files from 38.1.3 to 38.2.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 38.1.3 to 38.2.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](c860b5c47f...2f7246cb26)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 16:11:54 +02:00
Arthur Meyre
ff893ca6ef chore(wopbs): rework the inner part of WopLUTBase to be easier to use
BREAKING CHANGE: Wopbs LUT structs for shortint and integer have changed
APIs
2023-09-04 13:19:37 +02:00
tmontaigu
2dd1e13dad fix(integer): add assert on scalar bits in scalar_div
To work the scalar div requires that the scalar type (u32, u64, etc)
has at least as many bits of precision that what the ciphertext
encrypts (e.g. if the ciphertext encrypt 32bits, the scalar type must
have at least 32 bits)

We were missing the assert which lead to misuse, so we add it.
2023-09-01 18:04:51 +02:00
tmontaigu
4393fce861 feat(integer): be smarter about parallel carry propagation usage
The parallel carry propagation algorithm is a lot faster
than the non parallel one, however to actually be faster
it requires the CPU to have enough threads.

This adds checks so that we use the parallel algorithm
only if this condition is met.

This should improve performance on smaller CPUs
typically found in standard laptops/desktops.
2023-09-01 18:04:34 +02:00
Arthur Meyre
4f10cfa6dd chore(tfhe): bump version to 0.4.0 2023-08-31 14:29:17 +02:00
Mayeul@Zama
a6e4488de2 feat(js_test): add warning for expected error lor 2023-08-31 09:28:28 +02:00
Mayeul@Zama
878f3fa448 style(shortint): allow needless_pass_by_ref_mut on smart ops 2023-08-31 09:28:28 +02:00
Mayeul@Zama
90b887a56f style(tfhe): rename module 2023-08-31 09:28:28 +02:00
Mayeul@Zama
7dc52cf4ef chore(toolchain): fix clippy lints 2023-08-31 09:28:28 +02:00
Mayeul@Zama
a69333ed37 chore(toolchain): format rust 2023-08-31 09:28:28 +02:00
Mayeul@Zama
ffad25449e chore(toolchain): update toolchain 2023-08-31 09:28:28 +02:00
tmontaigu
6d471856c7 chore(docs): add section about writing trait bounds
This how to shows how to write the trait bounds of function
for the high level API.
2023-08-30 17:14:45 +02:00
David Testé
65749cb39b chore(bench): benchmark shortint ops against multi-bit parameters 2023-08-30 13:31:46 +02:00
David Testé
bf36316c12 chore(bench): benchmark server key creation from compressed one 2023-08-30 13:31:46 +02:00
David Testé
d98bb0eb86 chore(bench): measure bootstrap compressed key sizes 2023-08-30 13:31:46 +02:00
Arthur Meyre
5747af6dce feat(core): add LwePackingKeyswitchKey entity and algorithms
BREAKING CHANGE:
- more stringent checks for ciphertext moduli on some operations
- uniformized order of certain parameters for keyswitching primitives
- uniformized some primitives name
2023-08-30 10:54:35 +02:00
David Testé
cf08436c7d chore(bench): add pbs_ks parameters to pbs benchmarks 2023-08-29 10:02:01 +02:00
dependabot[bot]
241bddccaf chore(deps): bump rtCamp/action-slack-notify from 2.2.0 to 2.2.1
Bumps [rtCamp/action-slack-notify](https://github.com/rtcamp/action-slack-notify) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/rtcamp/action-slack-notify/releases)
- [Commits](12e36fc18b...b24d75fe0e)

---
updated-dependencies:
- dependency-name: rtCamp/action-slack-notify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-28 13:39:57 +02:00
dependabot[bot]
8ce1984214 chore(deps): bump tj-actions/changed-files from 37.6.1 to 38.1.3
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.6.1 to 38.1.3.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](a0585ff990...c860b5c47f)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-28 10:17:42 +02:00
dependabot[bot]
82ef430dfa chore(deps): bump actions/checkout from 3.5.3 to 3.6.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.3 to 3.6.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](c85c95e3d7...f43a0e5ff2)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-28 10:17:31 +02:00
tmontaigu
2348303b26 chore(ci): pin clap dependency 2023-08-25 21:03:42 +02:00
tmontaigu
a35386f740 chore(ci): remove ability for tests to be retried 2023-08-24 20:38:07 +02:00
tmontaigu
3df542c5f8 chore(bench): bench integer scalar ops up to 256 bits 2023-08-24 16:53:41 +02:00
David Testé
37623eedf3 chore(ci): add shortint pbs_ks parameters to security check 2023-08-23 09:05:45 +02:00
David Testé
fa8cf73d57 chore(ci): run parameters security check job unconditionally
Even if the parameters haven't changed in a commit, the lattice
estimator could have been updated. Hence the reason to run this
quick check on every push on main.
2023-08-23 09:05:45 +02:00
David Testé
872b20a4a1 chore(bench): add division and modulo ops to integer benchmarks 2023-08-23 09:05:16 +02:00
dependabot[bot]
4920e3b4df chore(deps): bump tj-actions/changed-files from 37.6.0 to 37.6.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.6.0 to 37.6.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](87697c0dca...a0585ff990)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-21 23:18:38 +02:00
David Testé
75a0881e9d fix(boolean): increase noise for tfhe_lib parameters set
This is done to match the values exposed in TFHE_LIB project.
2023-08-21 09:37:56 +02:00
dependabot[bot]
f67effc359 chore(deps): bump tj-actions/changed-files from 37.5.1 to 37.6.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.5.1 to 37.6.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](a96679dfee...87697c0dca)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-11 22:13:39 +02:00
David Testé
e8eb82f7ae chore(bench): use clean inputs for unchecked ops benchmarks
Previousely, unchecked_eq benchmark would fail at each run. Since
unchecked operations require clean inputs, we change the way we
instantiate benchmark functions.
2023-08-11 13:22:59 +02:00
David Testé
53018ddc36 chore(ci): ignore tfhe_lib_parameters check in lattice esitmator
This third-party parameters set is known to be less secure, so
there is no need to make the CI fail because of this set.
2023-08-11 11:08:59 +02:00
David Testé
9ef62acff1 chore(ci): change full suite benchmarks worflow description
This is done to avoid having the same name than the other
full benchmarks workflow.
2023-08-11 09:18:47 +02:00
aquint-zama
12220b2a18 chore(docs): design upgrade 2023-08-10 18:46:29 +02:00
tmontaigu
625c150dc1 feat(integer): start adding signed radix ciphertext
This introduces the SignedRadixCiphertext type
which encrypts signed values (i8, i16, etc).

Encryption and decryption functions are added.

This also makes the addition work with signed
values as the algorithm is the same.
2023-08-10 17:47:13 +02:00
David Testé
304932a861 chore(tfhe): refactor keycache in its own crate
The general parts of shortint keycache has been moved to its own
crate. This enable boolean layer to get access to traits without
having to import shortint::keycache module.
2023-08-10 16:42:40 +02:00
David Testé
59181d4717 chore(ci): add workflow to check security of parameters sets
Cryptographic parameters sets security is checked automatically
with a lattice estimator. The first step is to collect all the
parameters that need to be checked in in a file with a format
understable by Sagemath. Second, a lattice estimator is run in
a Sage script. Each parameters set is run against two attacks and
then security level is estimated from that.
These steps have been put into a GitHub workflow to perform
automatic checks.

Co-authored-by: Ben <ben.curtis@zama.ai>
2023-08-10 16:42:40 +02:00
David Testé
c5d93f4b38 chore(bench): Put integer comparisons ops in different groups
This is done to avoid hitting GitHub's hard limit of 6hours
maximum execution per job.
This commit also enable the cron job for the full benchmarks suite.
2023-08-10 11:16:43 +02:00
tmontaigu
7a465cd258 feat(integer): make full_propagate_parallelized more parallel
Using the functions that were introduced recently,
it is possible to make the full_propagate_parallelized method
more parallel than it was, resulting in faster computations.

The new carry propapagation should now be the cost of
a default add + one PBS, so ~400ms in 256 instead of ~3s.

However its probably slower for smaller number of blocks (eg 4 blocks)

This is done by extracting carry and messages in parallel,
then adding the carries to the correct message, the final step
is to use the single carry propapagation function.
2023-08-08 17:04:46 +02:00
tmontaigu
a0946ac509 feat(hlapi): add if_then_else 2023-08-08 11:35:37 +02:00
tmontaigu
5521f2a4a4 feat(integer): add if_then_else
This adds if_then_else (aka cmux / select)
to the integer API.

This also makes the min/max implementation use that
cmux instead of their own version of it, and allows
to save one pbs.
2023-08-08 11:35:37 +02:00
J-B Orfila
1a9e40c860 feat(boolean): add KS-PBS pattern choice to boolean
Co-authored-by: tmontaigu<thomas.montaigu@laposte.net>
2023-08-08 11:35:06 +02:00
David Testé
bc3e3e46a0 chore(bench): remove scalar prefix to default operations
This was making benchmark results parsing error-prone since scalar
operations must have the same name. It's the operand_type field
in the record parameters that acts as identifier (CipherText or
PlainText).
2023-08-04 08:45:46 +02:00
Arthur Meyre
60c87b6d95 chore(ci): the workflow was seen as ill formed by github
- it was sending error messages via email, silence it by having it well
formed
2023-08-02 15:14:29 +02:00
Arthur Meyre
df4c9f511d chore(ci): add placeholder workflow to be able to experiment with actions
- most of the time the workflow file needs to exist in main, with this it's
possible to experiment directly on ones branch as the file already exists
in main
2023-08-02 13:31:04 +02:00
David Testé
69bfd6556f chore(bench): prefix benchmarked function by its related layer
Replace ServerKey::[...] benchmark identifier by shortint::[...]
and integer::[...] to ease resutls parsing.
2023-08-02 10:26:06 +02:00
David Testé
7fad91e429 chore(ci): fix full benchmarks workflows
Workspace is cleaned between jobs so we can't factorize parts
of the benchmarks.
2023-08-02 10:26:06 +02:00
David Testé
0da30d5e58 chore(bench): benchmark with only one bit size for scalar 2023-08-02 10:26:06 +02:00
David Testé
97ce5b617a chore(bench): handle scalar bit size on integer results parsing 2023-08-02 10:26:06 +02:00
David Testé
a4723b03f3 chore(ci): install job dependencies right before running benchs
Since full benchmarks use a matrix to run jobs, we need to install
job dependencies (rustup, checkout repositories, ...) each time
because at the end of the job the workspace will be cleaned.
2023-08-02 10:26:06 +02:00
Arthur Meyre
d24d484bcf chore(bench): apply fix to circumvent rust's weird shift behavior 2023-08-02 10:26:06 +02:00
David Testé
9945cdd9b2 chore(ci): bench multiple scalar bit sizes and add operators
Scalar operations are benchmarked against scalar bit sizes.
Scalar division and modulo are added to the benchmark set.
2023-08-02 10:26:06 +02:00
tmontaigu
04a0aa0c31 feat(hlapi): allow scalar ops on values up to U256
This enables to use u128 and U256 as operands to
operations in the high level api.

BREAKING CHANGE: a breaking change in the C API for scalar operations
for FheUint128 and FheUint256 as they previously required
a u64 and now a U218 / U256 respectively.
2023-08-01 15:12:10 +02:00
Arthur Meyre
898c305acd refactor(keycache): refactor the named_params_impl macro
- allows to easily create named parameters from non standard shortint
parameter structs
- allows to reate such named parameters easily outside of the keycache mod
the way the macro was written before expanded to more macro calls that may
not have been in scope and rendered the macro virtually useless
2023-08-01 14:56:58 +02:00
dependabot[bot]
f5854db0b1 chore(deps): bump tj-actions/changed-files from 37.4.0 to 37.5.1
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.4.0 to 37.5.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](de0eba3279...a96679dfee)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-31 18:32:08 +02:00
Arthur Meyre
6a56af0b07 chore(ci): add a command to just build docs to see how it looks like
- add a target to lint doc building to replace the previous doc target
- adapt the top lib configuration for broken links to go from deny to warn
2023-07-28 11:52:01 +02:00
J-B Orfila
b7d830c57f docs: update the README for v0.3 2023-07-27 15:16:26 +02:00
David Testé
9b983745da chore(ci): create benchmarks workflows that runs the full suite
It will allow to runs benchmarks all the operations in shortint and
integer layers.
2023-07-27 12:52:42 +02:00
David Testé
10b1305e66 chore(ci): fast integer benchmarks to run on each push in main 2023-07-27 12:52:31 +02:00
Arthur Meyre
58b4089524 chore(docs): add remarks on smart operations taking mutable inputs 2023-07-26 11:57:53 +02:00
Arthur Meyre
98db328de2 fix(integer): set proper MaxDegree for CompressedServerKey
- add shortint API to generate a CompressedServerKey with MaxDegree
- add non regression test based on the user issue
- factorize MaxDegree computation for integer server keys
2023-07-26 10:00:24 +02:00
David Testé
f5fab4db99 chore(bench): run groups of benchmarks using env variable 2023-07-26 09:36:29 +02:00
Arthur Meyre
95f2eef94f chore(doc): fix multiplication typo 2023-07-25 20:51:15 +02:00
David Testé
2c10a792a5 chore(ci): trigger benchmarks only if layers have changed
For example, if only shortint layer related files have changed,
only the shortint benchmarks would be run on push.
However, if any files changed in the common_benches group then
all the benchmarks would be run.
2023-07-25 09:13:41 +02:00
dependabot[bot]
96689443ef chore(deps): bump tj-actions/changed-files from 37.1.2 to 37.4.0
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.1.2 to 37.4.0.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](2a968ff601...de0eba3279)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-24 13:04:38 +02:00
Arthur Meyre
514cb9e6af feat(core): add concrete-cpu tests for wopbs
- manage luts a bit differently to match TFHE-rs wopbs implementation
2023-07-24 10:56:49 +02:00
Pakorn Nathong
c79da46bb2 feat(integer): expose scalar mul and sub trait 2023-07-24 10:56:14 +02:00
tmontaigu
a8449f1ded feat(integer): allow scalar shift/rotate with more unsigned types
This is mainly for convenience.

Also, rust implements shift by u8, u16..u128 for each types.
(even shift by i8...i128 are implemented).
2023-07-21 13:31:29 +02:00
tmontaigu
11517703e6 fix(integer): remove incorrect bounds
In 35c6aea84b the bounds for
the scalar_div family of functions were changed.

However, the a few bounds `u64: From<T>` were
not removed meaning the functions which still
had these were still stuck with u64 as the max scalar value.

This commit removes the leftover bounds.
2023-07-20 17:10:26 +02:00
tmontaigu
35c6aea84b feat(integer): allow scalar_div/rem up to 256 bits 2023-07-20 13:55:21 +02:00
J-B Orfila
4e37f7e5bf docs(all): TFHE-rs v0.3.0 doc update 2023-07-19 11:08:36 +02:00
tmontaigu
a69a9c727b feat(integer): allow scalar_mul with U256
Same as scalar_sub, the UnsignedInteger bound was to strict, so we create a
`ScalarMultiplier` trait to allow using U256 as a scalar.
2023-07-17 16:30:17 +02:00
tmontaigu
bafb4f9e17 feat(integer): allow scalar sub with 256 bit scalar
The scalar (T) in scalar_sub could be at most a u128
because the bounds were `T: UnsignedInteger` and our
U256 does not implement this trait yet.

To make scalar_sub accept a U256 we create
a smaller trait.
2023-07-17 16:30:17 +02:00
Arthur Meyre
0663d7ca0e chore(ci): use aws ami with the latest updates
- Ubuntu 22.04 based
2023-07-17 15:57:58 +02:00
Arthur Meyre
8112684aae chore(ci): yet another bench fix 2023-07-17 15:51:07 +02:00
Arthur Meyre
f2f4cb7937 chore(ci): select benches to run 2023-07-17 14:16:28 +02:00
Arthur Meyre
e0e6aa845a chore(core): track caller on CiphertextModulus methods that can fail 2023-07-17 14:16:12 +02:00
Arthur Meyre
1455da273d chore(ci): remove auto retry for wasm tests
- the pipe was masking the potential error test
2023-07-17 14:16:00 +02:00
Arthur Meyre
763ad60ff9 chore(ci): fix m1 workflow run conidition 2023-07-17 11:07:32 +02:00
Arthur Meyre
6f4f923951 chore(ci): avoid removing labels when we are not on a PR 2023-07-17 11:07:32 +02:00
dependabot[bot]
5de984f7d6 chore(deps): bump tj-actions/changed-files from 37.0.5 to 37.1.2
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 37.0.5 to 37.1.2.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](54849deb96...2a968ff601)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-17 10:32:28 +02:00
dependabot[bot]
640b849d4b chore(deps): bump JS-DevTools/npm-publish from 2.2.0 to 2.2.1
Bumps [JS-DevTools/npm-publish](https://github.com/js-devtools/npm-publish) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/js-devtools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](a25b4180b7...5a85faf05d)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-17 10:32:07 +02:00
Arthur Meyre
7d484583ff chore(ci): add more triggers for re-running benches
- the bench "dependency" tree is bigger than first assumed
2023-07-17 09:36:54 +02:00
Arthur Meyre
68aa2ba25a chore(ci): fix workflow dispatch not triggering bench start 2023-07-17 09:36:54 +02:00
Arthur Meyre
228f85d843 chore(tfhe): remove dbg! macro calls and add a Makefile check for it 2023-07-13 19:45:15 +02:00
Arthur Meyre
f982c58538 chore(shortint): make shortint div behavior match integer on div by zero 2023-07-13 13:22:43 +02:00
Arthur Meyre
e2e901c220 chore(ci): fix usage of changed files 2023-07-12 17:27:06 +02:00
Arthur Meyre
507c569eee chore(shortint): add more convenience parameter aliases 2023-07-12 17:26:52 +02:00
Arthur Meyre
c37d9c590b chore(hlapi): remove leftover empty file 2023-07-12 14:55:12 +02:00
Arthur Meyre
549e9e70da chore(benches): need to checkout repo to check changed files for benchmarks 2023-07-10 13:05:17 +02:00
Arthur Meyre
d56afcd8c3 chore(integer): disable most sequence default add tests
- those are too slow and not the most optimized option to perform those
operations
2023-07-10 09:34:10 +02:00
Arthur Meyre
2019cd1708 chore(ci): M1 don't run multibit integer tests (too slow) 2023-07-10 09:34:10 +02:00
Arthur Meyre
3cfee104cb chore(ci): forward profile to shortint and integer test scripts 2023-07-10 09:34:10 +02:00
Arthur Meyre
4b174d552a chore(ci): run all M1 tests in FAST_TESTS=TRUE mode for better coverage 2023-07-10 09:34:10 +02:00
Arthur Meyre
1764c88de0 chore(ci): run schedule build only on public repo 2023-07-10 09:34:10 +02:00
Arthur Meyre
e4af2bad0f chore(test): fix wopbs only test which was using a wrong set of parameters 2023-07-10 09:34:10 +02:00
Arthur Meyre
59ef915095 chore(ci): fix C API build system to manage profiles other than release 2023-07-10 09:34:10 +02:00
Arthur Meyre
10f034171f chore(ci): LTO is causing issues in M1 CI tests use LTO off instead 2023-07-10 09:34:10 +02:00
Arthur Meyre
5e0aff616e chore(ci): run tests on M1 without integer as those are too long
- add a nightly trigger
2023-07-10 09:34:10 +02:00
Arthur Meyre
9687c55eb6 chore(ci): fix c_api_tests.sh to use threads on M1 properly 2023-07-10 09:34:10 +02:00
Arthur Meyre
222c5e1c19 chore(tfhe): misc fixes for error messages 2023-07-10 09:34:10 +02:00
Arthur Meyre
ea47265f15 chore(tfhe): remove unwarranted uses of unsafe when the code is not unsafe
- marking functions unsafe because the computations may be wrong due to a
bad choice of crypto parameters is not in line with the meaning of unsafe
in rust, so remove those uses
2023-07-10 09:33:18 +02:00
David Testé
465b79f42d chore(ci): trigger benchmarks only on specific file changes 2023-07-10 09:30:15 +02:00
tmontaigu
2557b29230 fix(shortint): Ciphertext::copy_from
Ciphertext::copy_from did not copy the degree
resulting in potential bad results for some operations.

This fixes it, and rewrites to use destructuring
in order to prevent such thing from happenning again
(with destructuring, if a member is not destructured,
a compile error is emited)

Also we move the implementation of copy_from into
clone_from.
2023-07-09 20:15:07 +02:00
tmontaigu
490bdaea30 fix(integer): fix U256::copy_to_be_byte_slice
There was a bug in to_be_bytes_slice, it was missing a
`slice.reverse()` (from_be_bytes correctly has it)

The from/to functions have been refactored to used
from_be_bytes / to_be_bytes, etc from the stdlib to
only have one layer of endianess to manage.

The test value used did not catch that so we change the value
used to expose the problem.
2023-07-07 15:29:35 +02:00
tmontaigu
936ac05e51 chore(core): fix typo in SignedInteger trait doc 2023-07-07 10:10:12 +02:00
tmontaigu
d496cfa431 feat(hlapi): bind scalar_bitwise/div/rem operations 2023-07-06 17:57:58 +02:00
Arthur Meyre
16be1c1c1d chore(bench): enable auto integer multi bit bench launch 2023-07-06 17:06:43 +02:00
Arthur Meyre
f2f4e397f1 chore(tfhe): bump version to 0.3.0 2023-06-30 23:10:26 +02:00
David Testé
facc2a162f test(integer): add unit and doc test for bitnot operator 2023-06-30 19:42:18 +02:00
Arthur Meyre
5981a886fd chore(bench): add multi bit key size measurements 2023-06-30 18:37:52 +02:00
tmontaigu
e98315fa60 feat(integer): add division by encrypted value
Adds a simple and slow algorithms for division/remainder
but at least it enables to use of this operators.

This also adds the same implementation in clear
so we will now be able to have u256 div.
2023-06-30 16:26:22 +02:00
Arthur Meyre
6b235f6fef chore(bench): fix issue due to overlapping merge 2023-06-30 13:15:36 +02:00
Arthur Meyre
4d376eea39 chore(bench): proper param name fix for WASM bench 2023-06-30 11:16:31 +02:00
tmontaigu
d93ddbe897 feat(integer): add scalar division/remainder 2023-06-30 09:46:47 +02:00
tmontaigu
189018ed05 feat(hlapi): allow use of multibit for integers 2023-06-30 09:45:14 +02:00
David Testé
fdae4e958c chore(ci): add bitnot operators to integer benchmarks 2023-06-30 09:32:42 +02:00
David Testé
d5ef359a04 chore(ci): use multi-bit params in shortint for pbs benchmarks
Use up-to-date crypto parameters for PBS benchmarks with multi-bit
instead of hardcoded ones.
2023-06-30 09:31:56 +02:00
J-B Orfila
a52cd6454d feat(shortint): add encrypt_message_and_carry 2023-06-29 17:34:36 +02:00
Arthur Meyre
142851792a chore(bench): fix param names 2023-06-29 16:20:58 +02:00
David Testé
e52bc09db5 chore(ci): add integer benchmarks with multi-bit parameters 2023-06-29 15:30:19 +02:00
Arthur Meyre
5bea1e0bc0 chore(ci): fix fast tests launching too many multi bit parameters 2023-06-28 19:14:20 +02:00
Arthur Meyre
224d81378a chore(docs): add information about the KS_PBS/PBS_KS naming "spec" 2023-06-28 19:14:20 +02:00
Arthur Meyre
011cb48ded chore(shortint): update exposed parameters 2023-06-28 19:14:20 +02:00
Arthur Meyre
da05f16c10 chore(shortint): add aliases for "old" parameter sets
- wopbs not included as it's due a heavy rewok
2023-06-28 19:14:20 +02:00
Arthur Meyre
ffc2472c95 chore(shortint): update keycache for CPK params, remove unusable params 2023-06-28 19:14:20 +02:00
Arthur Meyre
c0b82c77fb chore(shortint): plug cpk tests in scripts 2023-06-28 19:14:20 +02:00
Arthur Meyre
b09dc1f3ca chore(tfhe): rename params 2023-06-28 19:14:20 +02:00
J-B Orfila
a8e8a2e555 chore(shortint): update param compact key 2023-06-28 19:14:20 +02:00
David Testé
61819b2cea chore(ci): add ciphertext modulus for boolean crypto parameters 2023-06-28 18:14:27 +02:00
David Testé
1ee4440c0a chore(ci): put casting benchmarks into group to parse results
Fix usage of generics for crypto parameters in utilities.
2023-06-28 12:01:06 +02:00
David Testé
cbfaf63964 chore(ci): add pbs throughput benchmarks
This implies to add a conversion method to CiphertextModulus in
order to create the CryptoParametersRecord struct used as utils.
2023-06-27 18:16:27 +02:00
Arthur Meyre
6ac96bb46a chore(tfhe): dump non deterministic key and use deterministic when required 2023-06-27 16:10:22 +02:00
David Testé
f9b49eeb39 chore(ci): add feature gate for shortint benchmarks in utilities 2023-06-27 16:00:04 +02:00
Arthur Meyre
fdda5c56f2 feat(multibit): give the possibility to select deterministic execution
BREAKING CHANGE:
shortint ServerKey serialization has changed due to the additional info for
deterministic execution carried by the MultiBit variant
2023-06-27 13:21:23 +02:00
tmontaigu
cb20b4ad3a fix(integer): fix strict assert in add_paralellized 2023-06-27 12:54:50 +02:00
David Testé
fb653ef9b2 chore(ci): write shortint casting benchmarks to json file 2023-06-27 12:33:54 +02:00
Arthur Meyre
2e58fe36a4 test(core): add test on noise variance for lwe encryption 2023-06-26 14:27:09 +02:00
tmontaigu
2cbd8c9fd5 feat(integer): implement more U256 operators
This implements the following operators for U256
- BitXor
- Mul
- is_power_of_two
- ilog2
- SubAssign
2023-06-23 18:59:41 +02:00
twiby
11ac8e6cb9 feat(trivium): add bench for casting and packing 2023-06-23 16:01:40 +02:00
twiby
5f635e97fa feat(apps): add Trivium application of TFHE 2023-06-23 16:01:40 +02:00
twiby
7426e441ba feat(hlapi): keys can be derefed into their underlying keys 2023-06-23 16:01:40 +02:00
twiby
8ae799c477 feat(hlapi): impl TryFrom opeartors for GenericInteger: RadixCiphertext and Vec<Ciphertext> 2023-06-23 16:01:40 +02:00
tmontaigu
ee232ed81e feat(integer): add scalar bitwise operations
Nothing much interesting in terms of performance,
we only use the fact that we can 'inspect' the scalar
to avoid unnecessary work.
2023-06-23 15:57:21 +02:00
tmontaigu
16f4c721ab chore(wasm): re-enable tests which were wrongly disabled
Also fix a small typo, in HLAPI error message
2023-06-23 15:02:32 +02:00
Arthur Meyre
7ea13715ee chore(ci): run example tests 2023-06-22 11:57:27 +02:00
Arthur Meyre
a924b6ebde chore(ci): fix actions URL in a few workflows 2023-06-22 10:31:35 +02:00
Arthur Meyre
2b5d39c927 chore(ci): make release idiot proof 2023-06-22 10:31:35 +02:00
Morten Dahl
aafcbd0a3f chore(docs): fix typo 2023-06-21 18:09:12 +02:00
Arthur Meyre
e810b42eb6 chore(tfhe): remove wildcard deps 2023-06-21 16:24:22 +02:00
tmontaigu
352b282149 feat(hlapi): bind not operator (!) 2023-06-20 21:16:40 +02:00
tmontaigu
bc5f648c35 feat(hlapi): bind scalar comparisons 2023-06-20 21:16:40 +02:00
tmontaigu
7f83761fde feat(hlapi): bind not equal 2023-06-20 21:16:40 +02:00
tmontaigu
8c3993def2 feat(hlapi): bind bit rotations 2023-06-20 15:52:47 +02:00
tmontaigu
a361ad339d feat(hlapi): bind shift by encrypted amount
To keep things easy we have to drop the part
where the macro generated the docs.
2023-06-20 15:52:47 +02:00
sarah el kazdadi
aa82d9f19c feat(multibit): implement deterministic multibit pbs 2023-06-20 13:23:44 +02:00
tmontaigu
eae2d8137b refactor(shortint): rename generate_accumulator into generate_lookup_table
This renames the generate_accumulator* family of
functions into `generate_lookup_table`.

The reasoning is that we have `generate_accumulator`
which returns a `LookupTable` type, and you use
that with the `apply_lookup_table` function which is not
coherent.

Accumulator was the name we had originaly and consistently
for these, however lookup_table is probably easier to
understand / guess what it is about.
Also, it is the term used in concrete, the other
Zama product.
2023-06-19 18:09:28 +02:00
tmontaigu
ca127b2878 refactor(all): remove PBSOrder generic marker
The motivation for this change is that having the Big/Small
being a generic parameter of types, makes the code more complex
than it should be.

Also, It was not complete, not all structs had this generic parameter
making the whole thing a bit more clunky.
2023-06-19 15:16:09 +02:00
Arthur Meyre
80b5ce7f63 chore(core): remove Deref and DerefMut for Polynomial
- add indexing and iteration primitives
2023-06-19 13:21:39 +02:00
dependabot[bot]
2469aa0c2a chore(deps): bump actions/checkout from 3.5.2 to 3.5.3
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3.5.2...c85c95e3d7251135ab7dc9ce3241c5835cc595a9)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-19 11:47:07 +02:00
Arthur Meyre
48b307c627 chore(ci): fix wasm test docker 2023-06-19 11:46:52 +02:00
Arthur Meyre
8100b2d0de chore(ci): skip super slow integer tests in CI 2023-06-19 09:25:29 +02:00
Arthur Meyre
aab390470c chore(ci): add PR template 2023-06-19 09:25:16 +02:00
aquint-zama
f8f723f42d chore(readme): add citing tfhe-rs section 2023-06-16 14:14:13 +02:00
Arthur Meyre
1afdc71689 chore(tfhe): bump version for pre-release 2023-06-16 14:12:39 +02:00
tmontaigu
120f6b0304 feat(integer): add scalar comparisons
This adds scalar comparisons and min/max

- eq (==) and ne (!=) have similar performances
  to the non scalar version, only gaining a few milliseconds
  when the scalar value is smaller than the encrypted value

- orderings (<, >, <= ,>=) have more interesting
  performances gain the more the scalar value
  is small than the number of encrypted bits in the
  integer ciphertext.

  e.g:
  comparing an encrypted U256 to a value <= u128::MAX
  brings the comparison time from 234 ms to 194 ms

  comparing an encrypted U256 to a value <= u64::MAX
  brings the comparison time from 234 ms to 169 ms
2023-06-16 10:41:39 +02:00
Petar Ivanov
1d817c45d5 chore(makefile): add experimental deterministic FFT target for the C API
`Duration` is only needed when `experimental-force_fft_algo_dif4` is
set. We add a cfg directive to avoid a compiler warning.
2023-06-15 16:14:11 +03:00
David Testé
5690796da4 chore(ci): write boolean keys results file at the correct location 2023-06-14 18:47:32 +02:00
twiby
00c4eb417b fix(shortint): key switching key doctests 2023-06-14 18:45:54 +02:00
David Testé
42374db7cb chore(ci): use correct path to wasm pk generation result file 2023-06-14 15:57:59 +02:00
twiby
f98127498e feat(integer): add CastingKey struct to allow users to switch between integer parameter sets.
Possibilities are limited: you can only cast to a parameter set with the exact same representation: same message and carry size, same number of blocks in a radix representation, and same basis in crt representation.
2023-06-14 13:53:12 +02:00
twiby
d70729668e feat(shortint): add CastingKey struct to allow users to change server_key during a server circuit 2023-06-14 13:53:12 +02:00
twiby
8d339f2fbf feat(boolean): add CastingKey struct to allow users to change server_key during a server circuit 2023-06-14 13:53:12 +02:00
David Testé
c4a73f4f44 chore(ci): use correct relative path to wasm bench directory
Parsing program is using tfhe/ as working directory. Thus providing
a relative path starting with tfhe/ would result to an error while
trying to walk the directory.
2023-06-14 09:32:36 +02:00
tmontaigu
bd061dc85c refactor(hlpai): remove ServerKeyOp trait and macros
Since 8b3d31ae8a
integers use the same serve key, so
the `GenericIntegerServerKey<P>` type was removed.

Since `GenericIntegrServerKey<P>` does not exist,
there is much less need for the collection
of server key traits (ServerKeyAdd, SeverKeySub).

This commit removes them, as well as the macro layer
that was used to implement it
2023-06-13 12:57:03 +02:00
tmontaigu
2defb5a669 feat(c_api): safer destroys
This adds a null check in the different destroy function
of the C API. The performance impact of this should
be negligible / inexistant but should make usage more
ergonomic and safer.

Also this renames the detroy functions from being named
`destroy_shortint_*` | `destroy_boolean_*` to
`shortint_destroy_*` | `boolean_destroy_*` as it is more
coherent with the rest of the API where functions starts with `shortint`
or `boolean`.
2023-06-13 09:38:56 +02:00
David Testé
1b0f3631d4 chore(ci): fix wasm benchmarks and boolean keys measurement
Now use the CI version of the make recipe to run WASM client
benchmarks. In addition, boolean keys and wasm parsing is fixed
so that benchmarks_parameters directory is created and populated
under tfhe directory.
2023-06-12 18:30:36 +02:00
David Testé
06ddfe893a chore(ci): notify slack channel in case of benchmarks failure 2023-06-12 15:20:42 +02:00
David Testé
b69c9e7e7a chore(ci): create profile for wasm client bench and add in workflow 2023-06-12 15:20:13 +02:00
David Testé
18ed2e29a1 chore(ci): create fast feedback unit test profile
This is done to get quick feedback to developpers in a Pull Request.
It tests shorint level with only three sets of parameters. Integer
level is tested with only the default operations with two sets of
parameters.
This profile will be automatically triggered on each push in a
pull request. Conversely the full suite of test will also be
triggered automatically but once the review is approved.
2023-06-12 15:19:56 +02:00
Arthur Meyre
3a17ebd2fa feat(c_api): add entry point to generate LWE multi bit BSK 2023-06-12 14:18:13 +02:00
Agnes Leroy
dd15fd1b05 fix(core): fix multi bit bsk number of GGSWs check 2023-06-12 14:18:13 +02:00
dependabot[bot]
097ea6500c chore(deps): bump actions/checkout from 3.5.2 to 3.5.3
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](8e5e7e5ab8...c85c95e3d7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-12 13:51:34 +02:00
Arthur Meyre
9e307a8945 chore(hlapi): add example to measure CPK and CCTL sizes
This also includes key generation time in WASM web client side
2023-06-12 11:41:21 +02:00
Arthur Meyre
f8b497a4b8 chore(ci): fix integer bench workflow uploads 2023-06-12 10:23:13 +02:00
tmontaigu
189f02b696 refactor(hlapi): simplify wrapping of booleans
The way the boolean type was
wrapped was done the same way
shortints and integers were wrapped.

This was so that the internal code was
consistent despite not needing the same complexity.

Now that integers are wrapped differently it make
sense to remove the consistency constraint and
simplify the way booleans are wrapped in the HLAPI
2023-06-09 17:04:29 +02:00
tmontaigu
5654fe7981 feat(integer): scalar_mul generic over UnsignedInteger
This makes the scalar_mul family of operation
accepts any scalar of type T that implements
the UnsignedInteger traits.

This unlocks scalar_multiplication with
scalar being a u128.
2023-06-09 15:21:24 +02:00
Arthur Meyre
2b83a1fec0 chore(ci): add a parser to output csv files for integer benchmarks
- will simplify "just to see" benchmarks output parsing to share when
iterating on performance work
2023-06-09 14:27:49 +02:00
tmontaigu
efac3c842f feat(c_api): add #[repr(C)] for boolean parameters
The same thing was done in 3508019cd2
for shortint.

This does it for booleans
2023-06-09 11:53:22 +02:00
Arthur Meyre
7dbb4485bc chore(shortint): use the right noise at encryption time 2023-06-09 10:49:17 +02:00
aquint-zama
a5906bb7cb chore(tfhe): add a Code of Conduct 2023-06-08 14:06:29 +02:00
Jeremy Shulman
90b7494acd chore(doc): attach tutorials to doc 2023-06-08 14:05:46 +02:00
Arthur Meyre
3508019cd2 feat(core): Add Compact Public Key
- Based on "TFHE Public-Key Encryption Revisited "
  https://eprint.iacr.org/2023/603.pdf

Co-authored-by: tmontaigu <thomas.montaigu@laposte.net>
2023-06-07 19:47:50 +02:00
Arthur Meyre
200c8a177a feat(core): add std multi-bit bootstrapping 2023-06-07 16:12:37 +02:00
Arthur Meyre
2f6c1cf0b5 chore(ci): add docs alias make target for doc 2023-06-07 14:18:49 +02:00
tmontaigu
b96027f417 feat(integer): improve default sub latency 2023-06-07 11:04:11 +02:00
tmontaigu
90c850ca0d feat(integer): improve scalar add,sub and negation
- scalar_add now uses the same parallel carry propagation algorithm
  as the add function.

- scalar_sub now uses the same parallel carry propagation algorithm
  as the sub function.

- the 'default' negation function uses the now improved scalar_add
  to be faster

- unchecked_scalar_add, smart_scalar_add, checked_scalar_add, scalar_add
  have been updated to work on generic scalar type so it should work
  on u32, u64, u128, U256, etc

- unchecked_scalar_sub, smart_scalar_sub, checked_scalar_sub, scalar_sub
  have been updated to work on generic scalar type so it should work
  on u32, u64, u128.
  As U256 does not yet implement the UnsignedInteger trait, its not
  usable yet as a scalar type for the sub operation.

- The HLAPI is still locked to u64 scalars, it will be updated
  when most / all scalar ops are ready
2023-06-06 19:56:56 +02:00
Arthur Meyre
c8d3008a8d chore(shortint): proper ThreadCount serialization for bootstrapping key
- skip thread_count on serialization, deserialize using the function to
properly populate thread_count
2023-06-06 16:58:23 +02:00
David Testé
08c264f193 chore(ci): put wasm tests in their own workflow
This is mostly done to avoid failure on AWS tests (core, boolean,
shortint, ...) workflow due to flaky tests in WASM.
2023-06-06 14:02:52 +02:00
twiby
4ae202d8a4 refactor(tfhe): provide CiphertextBase with functions to convert from a generic type OpOrder to a specific struct.
This allows removing all calls to std::mem::transmute in shortint/engine/server_side/mod.rs, isolating unsafe blocks in the conversion functions. This makes the code safer and more likely to panic! in case of an error.
2023-06-06 12:19:56 +02:00
dependabot[bot]
7eb8601540 chore(deps): bump JS-DevTools/npm-publish from 2.1.0 to 2.2.0
Bumps [JS-DevTools/npm-publish](https://github.com/JS-DevTools/npm-publish) from 2.1.0 to 2.2.0.
- [Release notes](https://github.com/JS-DevTools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](541aa6b21b...a25b4180b7)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-06 10:23:30 +02:00
tmontaigu
8a1691c536 chore(wasm): remove serialization in web test
In the web wasm test we serialize the public key
to print its size (38_931_6265 bytes) this
means we hold the public key twice in ram.

I suspect this causes frequent out of
memory errors which then result in the
test timing out.

So we remove that hoping it has a positive impact
2023-06-02 17:19:04 +02:00
Arthur Meyre
d1cb55ba24 chore(tfhe): add multi bit shortint and integer tests
- default tests do not run multi bit PBS as it's not yet deterministic
- only radix parallel currently use multi bit pbs in integer
- remove determinism checks for some unchecked ops
- 4_4 multi bit parameters are disabled for now as they seem to introduce
too much noise
2023-06-02 16:00:28 +02:00
Arthur Meyre
2b9a49db87 chore(tfhe): switch to using Into for PBS parameters conversion
- it seems generally better for some "Self conversion" i.e. Into<A> for A
seems to work better than From<A> for A
2023-06-02 16:00:28 +02:00
Arthur Meyre
62ddb24f00 chore(ci): add multibit to key cache generation 2023-06-02 16:00:28 +02:00
Arthur Meyre
c6ae463b41 feat(shortint): add the possibility to use multi bit PBS 2023-06-02 16:00:28 +02:00
tmontaigu
4947eefad4 fix(u256): align with rust for shift behaviours 2023-06-02 12:00:42 +02:00
tmontaigu
71209e3927 feat(integer): make scalar shift match rust when shift >= bit size
When the scalar value denoting the shift was bigger or equal to
the total  number of bits in the ciphertext we would return zeros.

To match more the rust behaviour as well as the behaviour of
non scalar shift / rotate, the scalar shift will now remove
any higher bits of the clear shift value
2023-06-02 11:35:54 +02:00
tmontaigu
2a66ea3d16 feat(intger): add shifts and rotates on encrypted values
This implemantation is base on barrel shifters
which are used un hardware
2023-06-02 11:35:54 +02:00
tmontaigu
d4ff1f5595 feat(wasm): add parralellism in wasm API and add wasm for HLAPI
Co-authored-by: David Testé <david.teste@zama.ai>
2023-06-02 11:13:12 +02:00
Arthur Meyre
8ae92a960d chore(ci): add multibit workflow 2023-06-02 08:55:42 +02:00
tmontaigu
b042c2f7d6 refactor(integer): improve decomposition/recomposition into blocks
This new implementation should hopefully be a little bit easier to understand.

But more importantly it is more general/generic,
the previous implementation required the input type to be able to be described as u64 words,
the new one works for any type (as long as needed trait are implemented)

Also the new implementation is separated from the encryption code,
meaning it will be usable by scalar operation, which will allow us
to deduplicate code and start making scalar ops support scalar values
that are on more than 64-bits.
2023-06-01 18:13:34 +02:00
tmontaigu
e307da5c7f feat(integer): make eq (==) faster and add ne (!=) 2023-05-31 19:03:02 +02:00
Arthur Meyre
3d5b88d608 chore(core): encode the proper expectation wrt to ciphertext modulus
- we don't manage any non native moduli but rather native-compatible moduli
so update the asserts accordingly
2023-05-30 15:39:14 +02:00
Arthur Meyre
4fbf0691c5 chore(core): rename get_scaling_to_native_torus
- function now named get_power_of_two_scaling_to_native_torus to emphasize
it's reserved to power of 2 moduli
2023-05-30 15:39:14 +02:00
Arthur Meyre
5d277e85b9 feat(core): add non native decomposer 2023-05-30 15:39:14 +02:00
Arthur Meyre
778eea30e9 chore(tfhe): remove anyhow, just use Box<dyn std::error::Error> 2023-05-30 11:55:43 +02:00
tmontaigu
63247fa227 chore(sha256_example): use array_fn 2023-05-25 00:22:01 +02:00
David Testé
799291a1f0 docs(tfhe): format sha256_bool and add make recipes to run it 2023-05-25 00:22:01 +02:00
Sexosexosexo
509fe7a63e docs(tfhe): add boolean sha256 tutorial
Clap dev dependency added
2023-05-25 00:22:01 +02:00
tmontaigu
4eac45f0c6 fix(dark_market): fix change cwd logic 2023-05-24 23:30:26 +02:00
David Testé
ddb3451087 docs(tfhe): format dark market example add make recipe to run it 2023-05-24 23:30:26 +02:00
Yagiz Senal
e66a329e33 docs(tfhe): add dark market tutorial 2023-05-24 23:30:26 +02:00
David Testé
d79b1d9b19 docs(tfhe): format regex_engine and add make recipes to run it 2023-05-24 22:11:53 +02:00
Rick Klomp
b501cc078a docs(tfhe): add FHE Regex Pattern Matching Engine
this includes a tutorial and an example implementation for the regex bounty
2023-05-24 22:11:53 +02:00
tmontaigu
800878d89e feat(hlapi): add CompressedPublicKey decompression 2023-05-23 14:19:35 +02:00
tmontaigu
20d0e81bae feat(boolean): add CompressedPublicKey 2023-05-19 19:07:16 +02:00
tmontaigu
d3dbf4ecc9 feat(integer): allow decompressing CompressedPublicKey 2023-05-19 15:32:25 +02:00
tmontaigu
c20ca07cd3 chore(ci): reduce number of test-threads
Reduce number of test-threads being spawned
to reduce propability if tests getting killed due
to out of memory
2023-05-17 15:58:27 +02:00
tmontaigu
9f6c7e9139 feat(hlapi): add CompressedServerKey
Now that WopPBS key are optional in the hlapi
we can have a CompressedServerKey.
If a user tries to create a CompressedServerKey
but has enabled function evaluation on integers
(WopPBS) then it will panic as WopPBS are not yet compressible.
And 'stuffing' the non-compressed wopbs-key in the
compressed server key, would defeat the purpose of
compressed server key, as WopPBS key makes of for
the vast majority of the space used.

Also having CompressedServerKey is required to
be able to have wasm API of the hlapi
as wasm cannot generate normal server key.
2023-05-17 11:15:37 +02:00
David Testé
3c8d6a6f8b chore(ci): handle aws tests in pull request from forked repository 2023-05-17 08:42:19 +02:00
Arthur Meyre
1c837fa6f0 test(core): add normality test based on Shapiro-Francia 2023-05-16 10:12:28 +02:00
tmontaigu
1ec7e4762a feat(integer): make wopbs compile on wasm
The goal here is just to make the code compile
and not allow js api to generate wopbs key yet.
2023-05-15 22:06:36 +02:00
tmontaigu
20fb697d57 refactor(hlapi): disable WopPBS by default in hlapi
In the HLAPI, the WopPBS is enabled by default,
meaning the WopPBS key is generated when integers
are enabled.

This is not really good as the wopbs key is huge
(~700MB with PARAM_2_2) and only used for function evaluation
which does not scale for all types exposed by the halpi
and is still a bit experimental so not really advertised in the docs.

Also keys for wopbs are not compressible yet
(that is why the HLAPI does not yet have a CompressedServerKey).

So disabling wopbs by default will enable to have a compressed server
key that actually compresse things.
2023-05-15 19:01:53 +02:00
tmontaigu
0429d56cf3 chore(U256): add small tests 2023-05-15 11:40:44 +02:00
tmontaigu
509bf3e284 docs(bench): update results of benchmarks in the docs 2023-05-12 21:58:47 +02:00
Arthur Meyre
b2fc1d5266 refactor(shortint): make a difference between PBS and Wopbs parameters
- preparatory work to manage several PBS implementations and harmonize
parameters management

BREAKING CHANGE:
- parameters structures changed
- gen_keys for integer now takes parameters by value to uniformize with
shortint
2023-05-12 17:20:05 +02:00
Arthur Meyre
62d94dbee8 chore(tfhe): fix double Example heading in docstring 2023-05-12 17:20:05 +02:00
Agnes Leroy
fbe911d7db chore(tfhe): hard set number of threads to 10 for the multi-bit PBS
It's the optimal value measured on an m6i.metal instance where we run the benchmarks
2023-05-12 15:12:11 +02:00
tmontaigu
ba72faf828 chore(readme): remove non-needed mut in boolean example 2023-05-11 22:25:12 +02:00
tmontaigu
c387b9340f feat(integer): improve mul and scalar mul
This improves the mul and scalar_mul algorithms
to be faster

The improvement is made within the code
that was responsible for summing up all
the terms by making better use of carries
and avoiding uncessary propagations.

The scalar mul forwards the call to a right shift
when the scalar is a power of two as it just cost one
PBS so it will always be faster.

For 64-bits, target-cpu=native + avx512:
- mul before: 3.4s
- mul after: 900ms
2023-05-11 14:07:11 +02:00
Arthur Meyre
cbb7d30fb8 chore(core): avoid having branching depending on secret values in PKE 2023-05-11 11:12:43 +02:00
David Testé
6e4a707eff chore(ci): compute throughput as operations per second
Since most of the operations are over 1 ms, there is no point
to compute the number of operation per millisecond.
2023-05-11 08:58:45 +02:00
tmontaigu
06b700f904 feat(integer): improve parallel algorithms for add/sub
This adds fully parallel algorithms for the addition and subtraction

These algorithms take ciphertexts with clean carries and
return creates a sum ciphertext that also has clean carries.

The carries are propagated in parallel, using
parallel algorithms for prefix sum / cumulative sum.

The is one based on Hillis and Steele, it the fastest
but uses a lot of threads.

The other on Blelloch, it requires less threads but is a
bit slower.

256bits addition using param_2_2 goes down from ~2.7s to:
- 364ms using Hillis and Steele
- 474ms using Blelloch

The commit also adds bitwise not, as it it necessary for the
subtraction.
2023-05-10 14:43:40 +02:00
dependabot[bot]
cfbabf7480 chore(deps): bump JS-DevTools/npm-publish from 2.0.0 to 2.1.0
Bumps [JS-DevTools/npm-publish](https://github.com/JS-DevTools/npm-publish) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/JS-DevTools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](0be441d808...541aa6b21b)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-09 11:15:16 +02:00
tmontaigu
291ed9026f feat(hlapi): add casting between integer types
This adds the casting of integer types.

Downcasting truncates blocks.
Upcasting appends 0s

Casting is done via the introduced `cast_from` associated
function and the `cast_into` method. They are the equivalent of
the `From` and `Into` traits.

It was not possible to implement casting by implementing the
standard `From` and `Into` traits as initially planned because:

```rust
impl<P1, P2> From<GenericInteger<P1>> for GenericInteger<P2>
where P1: IntegerParameter,
      P2: Integer Parameter, {
    fn from(_: GenericInteger<P1>) -> Self {
        todo!()
    }
}
```

As it conflicts the blanket impl found in the stdlib
`impl<T> From<T> for T;` as P1, P2 may be the same and we have no way
of telling the compiler to consider this impl only when P1 != P2.
So we had to create our own stuff
2023-05-05 21:43:10 +02:00
tmontaigu
610f0010b8 refactor(hlapi): remove RefCell used to wrap integers
Our integer types are based on tfhe::integer.

Originally, operators (+, -, *, <<, etc) were mapped to "smart" operations
until commit "ee96a0ff185fedb9c4467a5b0c8195798c30b19f" where we swapped
to "default" ops.

The motivation of swapping from smart to default was that default ops
timing was always the same, so its easier to predict and reason about when
comparing with available benchmarks.
However, they may give worse performance depending
on the computations being done (addition/subtraction heavy or not).

In the High level API, we overloaded operators on const ref e.g. `&a + &b`
but as we initially mapped to smart operations we needed interior mutability.
RefCell was chosen, mainly because using Mutexes would have allowed
users to write "fake" parallel code, that is, the code compiles but is not
truly parallel due to the mutexes. RefCell makes writting parallel code
harder but If you manage to do it, its truly parallel.

After moving to default ops for the hlapi, we kept the inner RefCells
to allow time to decide to retract the change.

The final choice is to keep default ops as the default, so that means
we can remove the RefCells.

Entry points to smart operation will be added later to enable
'power users' to explicitely try them and see if they bring
improvement(s) for them.
Letting advanced user explicitely handle mutability of smart
operations and let them choose their synchronisation is likely a better
choice than making it for them as its an important choice.
2023-05-05 18:21:43 +02:00
tmontaigu
8b3d31ae8a refactor(hlapi): use one unique key for integers
This refactor of the inner workings of the High Level API
makes it so that all integer types share and use the same key
to encrypt their blocks

Before this commit, users that wanted to use integers via the
hlapi needed to select the type amongst the ones available
and enable it (eg i want to use 16 bits integers so I call
enable_default_uint16).

This meant that if users wanted to use many integer types
they would have to manually enable them. Since each type had its
own key, it meant they were completely separate (no casting possible)
and the different key type `ClientKey`, `ServerKey` would become very
big. So in practice you would stick to one and only integer type in your
program.

With this changes, users that wishes to use integers, will just need
to enable them (enable_default_integers()), and will get access to
all statically difined integer types (FheUint8, FheUint16, etc)
at the cost of one key for of these types.

This reduces complexity, memory footprint and,
will enable to introduce in commits later the ability to cast between
integer types.

BREAKING CHANGES:
 - Serialized keys are not backward compatible
 - enable_default_uint[8,16,etc] become enable_default_integers
2023-05-05 16:05:58 +02:00
sarah el kazdadi
98539aaa61 fix(pbs): fix bug in rounding code in f128 pbs 2023-05-05 15:32:59 +02:00
Arthur Meyre
9a80a01dc3 feat(integer): add trim/extend APIs for radix ciphertexts 2023-05-04 09:46:06 +02:00
tmontaigu
ecf9d50058 feat(integer): add parallelized scalar rotate_left/right
Like shifts, rotates are implemented by combining
a rotation of the block and bivariate PBSs in case
the rotation number `n` is not a multiple of the number
of bits in a block.

Since the behaviour of rotations is to 'cycle' bits
back the end/beginning of the 'bit slice' (i.e. no bits is ever lost like
it can with shifts), the performance is always the same when
(n % nb_bits_in block) != 0. However the implementation is simpler.

So assuming a machine where the number of threads
is >= to the ciphertext's number of block, the operation
cost one bivariate PBS.
2023-05-02 17:52:44 +02:00
Arthur Meyre
65e4aab38d chore(shortint): fix docstrings which were mixing big and small key params 2023-05-02 17:21:39 +02:00
Arthur Meyre
ac348870ba refactor(shortint): add encryption key choice in parameters
BREAKING CHANGE:
- Parameters layout change
- C API removal of SHORTINT_NATIVE_MODULUS which was a leftover from
a refactor
2023-05-02 17:21:39 +02:00
Arthur Meyre
6adfcaa5f7 chore(tfhe): bump version to 0.3.0 2023-05-02 11:10:14 +02:00
Arthur Meyre
bc6bbe66d9 chore(shortint): fix some wopbs function signature 2023-05-02 11:10:04 +02:00
Arthur Meyre
871d4aea17 refactor(core): refactor CiphertextModulus to be less error prone 2023-04-28 16:40:33 +02:00
Arthur Meyre
f81376b762 chore(ci): start benches only on our repo 2023-04-28 11:17:31 +02:00
Arthur Meyre
64813bae18 chore(tfhe): as seen there are uses of ilog2 which come from rust 1.67 2023-04-28 11:01:06 +02:00
Arthur Meyre
16ce2a8a3f refactor(wopbs): manage LUTs for wopbs to avoid copies 2023-04-28 09:45:11 +02:00
tmontaigu
f018987eac feat(integer): improve scalar shifts performances
This reworks the left/right scalar shift method.

This new implementation takes more advantage of the
radix representation property and use a combo
of shifting/moving blocks via a rotate_left/right
optionnaly followed by in-block shifting and propagation
to other blocks.

This new implementation requires the carries to be empty
(not sure what the preconditions were for the previous implementation)
and the output will also have clean carries.
Requiring empty carries allows to do the shift in a way
that scales well with the number of blocks as we can use truly parallel
operations.

This means that the time required to shift is dependent
in the shift value, which should not be a security problem
as its a clear value. (The previous implementation also
had timing that depended on the shift value)

There are two scenarios possible:
- The shift only require to move blocks -> its fast
- The shift requires moving + in-block shifting -> slower,
  but still faster than the previous implementation.

Worst case is when shift is less than the number of bits
in a block.

This also changes the type of the `shift` parameter from
`usize` to `u64` to be consistent with other scalar operations.

The follwing pseudo bench code on 64 bits
gives the following time ranges:

With changes:
```
unchecked_left: BenchStat {
    min: 16.743µs
    max: 526.518563ms
    mean: 150.77871ms
}
unchecked_left_parallelized: BenchStat {
    min: 17.291µs
    max: 94.408455ms
    mean: 30.092279ms
}
unchecked_right: BenchStat {
    min: 16.723µs
    max: 548.417332ms
    mean: 160.345234ms
}
unchecked_right_parallelized: BenchStat {
    min: 16.978µs
    max: 97.955322ms
    mean: 33.500562ms
}
Measured in 37.890296743s
```

Previous code:
```
unchecked_left: BenchStat {
    min: 1.055401595s
    max: 1.156574075s
    mean: 1.085648592s
}
unchecked_left_parallelized: BenchStat {
    min: 559.636545ms
    max: 630.83338ms
    mean: 584.747893ms
}
unchecked_right: BenchStat {
    min: 1.055041354s
    max: 2.314891255s
    mean: 1.644513996s
}
unchecked_right_parallelized: BenchStat {
    min: 562.017144ms
    max: 1.275945891s
    mean: 894.812286ms
}
Measured in 421.412913883s
```

```rust
use rand::Rng;
use std::time::Instant;
use tfhe::integer::{gen_keys, RadixClientKey};
use tfhe::shortint::parameters::PARAM_MESSAGE_2_CARRY_2;

const NB_CTXT: usize = 32;
const NB_TEST: usize = 100;

struct BenchStat {
    min: Option<std::time::Duration>,
    max: Option<std::time::Duration>,
    sum: std::time::Duration,
    count: u32,
}

impl BenchStat {
    fn update(&mut self, elapsed: std::time::Duration) {
        if self.min.is_none() {
            self.min = Some(elapsed);
        } else {
            self.min = self.min.map(|l| l.min(elapsed));
        }
        if self.max.is_none() {
            self.max = Some(elapsed);
        } else {
            self.max = self.max.map(|l| l.max(elapsed));
        }
        self.sum += elapsed;
        self.count += 1;
    }

    fn print(&self) {
        println!("BenchStat {{");
        println!("    min: {:?}", self.min.unwrap());
        println!("    max: {:?}", self.max.unwrap());
        println!("    mean: {:?}", self.sum / self.count);
        println!("}}");
    }
}

type ShiftType = u64;

fn main() {
    let mut unchecked_left_timing = BenchStat::default();
    let mut unchecked_left_parallelized_timing = BenchStat::default();
    let mut unchecked_right_timing = BenchStat::default();
    let mut unchecked_right_parallelized_timing = BenchStat::default();

    let total = Instant::now();

    let param = PARAM_MESSAGE_2_CARRY_2;
    let (cks, sks) = gen_keys(&param);
    let cks = RadixClientKey::from((cks, NB_CTXT));

    let mut rng = rand::thread_rng();

    //Nb of bits to shift
    let tmp_f64 = param.message_modulus.0 as f64;
    let nb_bits = tmp_f64.log2().floor() as usize * NB_CTXT;
    let modulus = (param.message_modulus.0 as u128).pow(NB_CTXT as u32);
    assert_eq!(nb_bits, 64);

    for i in 0..NB_TEST {
        println!("{} / {NB_TEST}", i + 1);
        let clear = rng.gen::<u128>() % modulus;
        let scalar = rng.gen::<u128>() % nb_bits as u128;

        println!("clear: {clear}, scalar: {scalar}");

        let ct = cks.encrypt(clear);

        {
            let before = Instant::now();
            let ct_res = sks.unchecked_scalar_left_shift(&ct, scalar as ShiftType);
            unchecked_left_timing.update(before.elapsed());
            //assert!(ct_res.block_carries_are_empty());
            let dec_res: u128 = cks.decrypt(&ct_res);
            assert_eq!((clear << scalar) % modulus, dec_res);

            let before = Instant::now();
            let ct_res = sks.unchecked_scalar_left_shift_parallelized(&ct, scalar as ShiftType);
            unchecked_left_parallelized_timing.update(before.elapsed());
            //assert!(ct_res.block_carries_are_empty());
            let dec_res: u128 = cks.decrypt(&ct_res);
            assert_eq!((clear << scalar) % modulus, dec_res);
        }

        {
            let before = Instant::now();
            let ct_res = sks.unchecked_scalar_right_shift(&ct, scalar as ShiftType);
            unchecked_right_timing.update(before.elapsed());
            // assert!(ct_res.block_carries_are_empty());
            let dec_res: u128 = cks.decrypt(&ct_res);
            assert_eq!((clear >> scalar) % modulus, dec_res);

            let before = Instant::now();
            let ct_res = sks.unchecked_scalar_right_shift_parallelized(&ct, scalar as ShiftType);
            unchecked_right_parallelized_timing.update(before.elapsed());
            //assert!(ct_res.block_carries_are_empty());
            let dec_res: u128 = cks.decrypt(&ct_res);
            assert_eq!((clear >> scalar) % modulus, dec_res);
        }
    }

    print!("unchecked_left: ");
    unchecked_left_timing.print();
    print!("unchecked_left_parallelized: ");
    unchecked_left_parallelized_timing.print();

    print!("unchecked_right: ");
    unchecked_right_timing.print();
    print!("unchecked_right_parallelized: ");
    unchecked_right_parallelized_timing.print();

    println!("Measured in {:?}", total.elapsed());
}
```

BREAKING CHANGE: parameter type changed from usize to u64
2023-04-27 17:18:55 +02:00
Arthur Meyre
20f6c5419b chore(core): re-enable split pbs for u128 2023-04-25 17:42:40 +02:00
Arthur Meyre
58b530f40b chore(doc): fix docstring ref 2023-04-25 17:42:40 +02:00
Arthur Meyre
689ad195f3 refactor(integer): remove usage of Mutex for determinism 2023-04-25 15:25:51 +02:00
sarah el kazdadi
4a1eda25d3 fix(split): fix split pbs backward conversion 2023-04-24 16:08:23 +02:00
Arthur Meyre
af936df064 chore(core): change rng tests to better avoid false failures
- we still check we generate non zero values but add retry conditions or
have less stringent checks, to allow some values to be zero for example as
it's a valid value that can be generated
- each test suite (test and doctest) for these tests ran 1000 times without
failure
2023-04-24 13:26:35 +02:00
dependabot[bot]
0233a69ea6 chore(deps): bump JS-DevTools/npm-publish from 1.4.3 to 2.0.0
Bumps [JS-DevTools/npm-publish](https://github.com/JS-DevTools/npm-publish) from 1.4.3 to 2.0.0.
- [Release notes](https://github.com/JS-DevTools/npm-publish/releases)
- [Changelog](https://github.com/JS-DevTools/npm-publish/blob/main/CHANGELOG.md)
- [Commits](0f451a9417...0be441d808)

---
updated-dependencies:
- dependency-name: JS-DevTools/npm-publish
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-24 12:56:59 +02:00
Arthur Meyre
f72a6ec835 chore(doc): fix typo 2023-04-24 09:30:16 +02:00
David Testé
25a2586eae chore(ci): publish tfhe release on-demand
This will perform on-demand release publication.
It will publish on the following channels:
 * crates.io
 * web and node package on npmjs
2023-04-21 14:39:36 +02:00
Arthur Meyre
c112a43a63 chore(core): add more sanity checks on RNG 2023-04-21 14:36:14 +02:00
Arthur Meyre
2813812380 fix(core): fix rng 2023-04-21 14:36:14 +02:00
tmontaigu
84a6036789 feat(boolean): add BooleanEngine::replace_thread_local
This new associated function allows to replace
the engine used in the thread.
2023-04-20 15:16:13 +02:00
David Testé
658368d0b6 chore(ci): create dummy release workflow
This is done to be able to test the effective worklfow
implementation in a development branch.
2023-04-20 10:26:31 +02:00
Arthur Meyre
9368049adf chore(core): disable split pbs128 2023-04-19 18:38:22 +02:00
David Testé
5e8ca0b52c chore(ci): fix decomposition basis and add bit size to params
Decomposition basis wasn't correctly set to handle CRT. Now it uses
a Vec that would be displayed as a string in the database.
In addition the bit size has been added to ease comparison between
various of them in Grafana.
2023-04-19 17:58:22 +02:00
Arthur Meyre
605cd5b3b0 chore(doc): updated benchmarks for min to reflect the fix done to min/max 2023-04-19 16:56:46 +02:00
David Testé
4bfe9c22d4 chore(ci): remove unused env variable in boolean benchmarks 2023-04-19 16:28:13 +02:00
Arthur Meyre
1c0b36c672 chore(bench): only run avx512 benches 2023-04-19 09:23:01 +02:00
Arthur Meyre
7dccb01a8d fix(integer): fix mul correctness
- update benches accordingly
2023-04-19 09:23:01 +02:00
Arthur Meyre
7bff348367 chore(bench): more multi-bit bench params 2023-04-19 09:03:58 +02:00
tmontaigu
74a5a278b6 fix(hlapi): use correct number of blocks for FheUint32
The FheUint32 was wrongly defined as being 32 blocks of 2 bits
when it should have been 16 blocks.
2023-04-18 18:40:39 +02:00
tmontaigu
426ced3295 feat(hlapi): add trivial encryptions 2023-04-18 15:07:06 +02:00
tmontaigu
7af5fcc7eb feat(integer): add trivial encryption 2023-04-18 15:07:06 +02:00
Arthur Meyre
12d9947149 chore(integer): add non regression test for scalar mul fix 2023-04-18 10:46:19 +02:00
dependabot[bot]
7c54896e68 chore(deps): bump actions/checkout from 3.5.0 to 3.5.2
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](8f4b7f8486...8e5e7e5ab8)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-17 11:17:58 +02:00
J-B Orfila
04533dedfe chore(doc): fix typo 2023-04-14 18:30:30 +02:00
J-B Orfila
d01be35557 chore(doc): fix TOML 2023-04-14 18:30:30 +02:00
J-B Orfila
9fc32e2f52 chore(doc): fix dead links 2023-04-14 13:37:03 +02:00
Arthur Meyre
e1e78b8b9d chore(integer): restore empty carry check for default comparator tests
- only extract assign message instead of doing a full propagate as carries
are not supposed to be non zero (though the degree will have grown)
2023-04-14 13:34:35 +02:00
635 changed files with 104836 additions and 29600 deletions

View File

@@ -5,13 +5,3 @@ failure-output = "final"
fail-fast = false
retries = 0
slow-timeout = "5m"
[[profile.ci.overrides]]
filter = 'test(/^.*param_message_1_carry_[567]$/) or test(/^.*param_message_4_carry_4$/)'
retries = 3
[[profile.ci.overrides]]
filter = 'test(/^.*param_message_[23]_carry_[23]$/)'
retries = 1

13
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,13 @@
<!-- Feel free to delete the template if the PR (bumping a version e.g.) does not fit the template -->
closes: _please link all relevant issues_
### PR content/description
### Check-list:
* [ ] Tests for the changes have been added (for bug fixes / features)
* [ ] Docs have been added / updated (for bug fixes / features)
* [ ] Relevant issues are marked as resolved/closed, related issues are linked in the description
* [ ] Check for breaking changes (including serialization changes) and add them to commit message following the conventional commit [specification][conventional-breaking]
[conventional-breaking]: https://www.conventionalcommits.org/en/v1.0.0/#commit-message-with-description-and-breaking-change-footer

View File

@@ -0,0 +1,127 @@
# Run a small subset of shortint and integer tests to ensure quick feedback.
name: Fast AWS Tests on CPU
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
# Allows you to run this workflow manually from the Actions tab as an alternative.
workflow_dispatch:
# All the inputs are provided by Slab
inputs:
instance_id:
description: "AWS instance ID"
type: string
instance_image_id:
description: "AWS instance AMI ID"
type: string
instance_type:
description: "AWS instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
type: string
fork_repo:
description: 'Name of forked repo as user/repo'
type: string
fork_git_sha:
description: 'Git SHA to checkout from fork'
type: string
jobs:
fast-tests:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ inputs.runner_name }}
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: stable
default: true
- name: Run concrete-csprng tests
run: |
make test_concrete_csprng
- name: Run core tests
run: |
AVX512_SUPPORT=ON make test_core_crypto
- name: Run boolean tests
run: |
make test_boolean
- name: Run user docs tests
run: |
make test_user_doc
- name: Run js on wasm API tests
run: |
make test_nodejs_wasm_api_in_docker
- name: Gen Keys if required
run: |
make gen_key_cache
- name: Run shortint tests
run: |
BIG_TESTS_INSTANCE=TRUE FAST_TESTS=TRUE make test_shortint_ci
- name: Run integer tests
run: |
BIG_TESTS_INSTANCE=TRUE FAST_TESTS=TRUE make test_integer_ci
- name: Run shortint multi-bit tests
run: |
BIG_TESTS_INSTANCE=TRUE FAST_TESTS=TRUE make test_shortint_multi_bit_ci
- name: Run integer multi-bit tests
run: |
BIG_TESTS_INSTANCE=TRUE FAST_TESTS=TRUE make test_integer_multi_bit_ci
- name: Run high-level API tests
run: |
make test_high_level_api
- name: Run safe deserialization tests
run: |
make test_safe_deserialization
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Fast AWS tests finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -1,4 +1,4 @@
name: AWS Integer Tests on CPU
name: AWS Unsigned Integer Tests on CPU
env:
CARGO_TERM_COLOR: always
@@ -23,28 +23,37 @@ on:
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
description: "Slab request ID"
type: string
matrix_item:
description: 'Build matrix item'
fork_repo:
description: "Name of forked repo as user/repo"
type: string
fork_git_sha:
description: "Git SHA to checkout from fork"
type: string
jobs:
integer-tests:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ github.event.inputs.instance_image_id }}_${{ github.event.inputs.instance_type }}
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ github.event.inputs.runner_name }}
runs-on: ${{ inputs.runner_name }}
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ github.event.inputs.instance_id }}"
echo "AMI: ${{ github.event.inputs.instance_image_id }}"
echo "Type: ${{ github.event.inputs.instance_type }}"
echo "Request ID: ${{ github.event.inputs.request_id }}"
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
@@ -56,18 +65,26 @@ jobs:
toolchain: stable
default: true
- name: Gen Keys if required
run: |
make GEN_KEY_CACHE_MULTI_BIT_ONLY=TRUE gen_key_cache
- name: Run unsigned integer multi-bit tests
run: |
AVX512_SUPPORT=ON make test_unsigned_integer_multi_bit_ci
- name: Gen Keys if required
run: |
make gen_key_cache
- name: Run integer tests
- name: Run unsigned integer tests
run: |
BIG_TESTS_INSTANCE=TRUE make test_integer_ci
AVX512_SUPPORT=ON BIG_TESTS_INSTANCE=TRUE make test_unsigned_integer_ci
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@12e36fc18b0689399306c2e0b3e0f2978b7f1ee7
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}

View File

@@ -0,0 +1,98 @@
name: AWS Signed Integer Tests on CPU
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
# Allows you to run this workflow manually from the Actions tab as an alternative.
workflow_dispatch:
# All the inputs are provided by Slab
inputs:
instance_id:
description: "AWS instance ID"
type: string
instance_image_id:
description: "AWS instance AMI ID"
type: string
instance_type:
description: "AWS instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
fork_repo:
description: "Name of forked repo as user/repo"
type: string
fork_git_sha:
description: "Git SHA to checkout from fork"
type: string
jobs:
multi-bit-tests:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ inputs.runner_name }}
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: stable
default: true
- name: Gen Keys if required
run: |
make GEN_KEY_CACHE_MULTI_BIT_ONLY=TRUE gen_key_cache
- name: Run shortint multi-bit tests
run: |
make test_shortint_multi_bit_ci
- name: Run signed integer multi-bit tests
run: |
AVX512_SUPPORT=ON make test_signed_integer_multi_bit_ci
- name: Gen Keys if required
run: |
make gen_key_cache
- name: Run signed integer tests
run: |
AVX512_SUPPORT=ON BIG_TESTS_INSTANCE=TRUE make test_signed_integer_ci
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Shortint tests finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -25,26 +25,35 @@ on:
request_id:
description: 'Slab request ID'
type: string
matrix_item:
description: 'Build matrix item'
fork_repo:
description: 'Name of forked repo as user/repo'
type: string
fork_git_sha:
description: 'Git SHA to checkout from fork'
type: string
jobs:
shortint-tests:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ github.event.inputs.instance_image_id }}_${{ github.event.inputs.instance_type }}
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ github.event.inputs.runner_name }}
runs-on: ${{ inputs.runner_name }}
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ github.event.inputs.instance_id }}"
echo "AMI: ${{ github.event.inputs.instance_image_id }}"
echo "Type: ${{ github.event.inputs.instance_type }}"
echo "Request ID: ${{ github.event.inputs.request_id }}"
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
@@ -56,6 +65,10 @@ jobs:
toolchain: stable
default: true
- name: Run concrete-csprng tests
run: |
make test_concrete_csprng
- name: Run core tests
run: |
AVX512_SUPPORT=ON make test_core_crypto
@@ -72,10 +85,6 @@ jobs:
run: |
make test_user_doc
- name: Run js on wasm API tests
run: |
make test_nodejs_wasm_api_in_docker
- name: Gen Keys if required
run: |
make gen_key_cache
@@ -88,10 +97,20 @@ jobs:
run: |
BIG_TESTS_INSTANCE=TRUE make test_high_level_api
- name: Run example tests
run: |
make test_examples
make dark_market
- name: Run apps tests
run: |
make test_trivium
make test_kreyvium
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@12e36fc18b0689399306c2e0b3e0f2978b7f1ee7
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}

View File

@@ -0,0 +1,87 @@
name: AWS WASM Tests on CPU
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
# Allows you to run this workflow manually from the Actions tab as an alternative.
workflow_dispatch:
# All the inputs are provided by Slab
inputs:
instance_id:
description: "AWS instance ID"
type: string
instance_image_id:
description: "AWS instance AMI ID"
type: string
instance_type:
description: "AWS instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
type: string
fork_repo:
description: 'Name of forked repo as user/repo'
type: string
fork_git_sha:
description: 'Git SHA to checkout from fork'
type: string
jobs:
wasm-tests:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ inputs.runner_name }}
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: stable
default: true
- name: Run js on wasm API tests
run: |
make test_nodejs_wasm_api_in_docker
- name: Run parallel wasm tests
run: |
make install_node
make ci_test_web_js_api_parallel
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "WASM tests finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -5,24 +5,33 @@ on:
workflow_dispatch:
inputs:
instance_id:
description: 'Instance ID'
description: "Instance ID"
type: string
instance_image_id:
description: 'Instance AMI ID'
description: "Instance AMI ID"
type: string
instance_type:
description: 'Instance product type'
description: "Instance product type"
type: string
runner_name:
description: 'Action runner name'
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
description: "Slab request ID"
type: string
# This input is not used in this workflow but still mandatory since a calling workflow could
# use it. If a triggering command include a user_inputs field, then the triggered workflow
# must include this very input, otherwise the workflow won't be called.
# See start_full_benchmarks.yml as example.
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-boolean-benchmarks:
@@ -42,7 +51,7 @@ jobs:
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
@@ -57,9 +66,9 @@ jobs:
toolchain: nightly
override: true
- name: Run benchmarks
- name: Run benchmarks with AVX512
run: |
make bench_boolean
make AVX512_SUPPORT=ON bench_boolean
- name: Parse results
run: |
@@ -73,24 +82,8 @@ jobs:
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--throughput
- name: Remove previous raw results
run: |
rm -rf target/criterion
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON bench_boolean
- name: Parse AVX512 results
run: |
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--hardware ${{ inputs.instance_type }} \
--name-suffix avx512 \
--walk-subdirs \
--throughput \
--append-results
--throughput
- name: Measure key sizes
run: |
@@ -101,15 +94,15 @@ jobs:
python3 ./ci/benchmark_parser.py tfhe/boolean_key_sizes.csv ${{ env.RESULTS_FILENAME }} \
--key-sizes \
--append-results
- name: Upload parsed results artifact
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_boolean
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
@@ -117,8 +110,6 @@ jobs:
- name: Send data to Slab
shell: bash
env:
COMPRESSED_RESULTS : ${{ env.RESULTS_FILENAME }}.gz
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
@@ -130,3 +121,15 @@ jobs:
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Boolean benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -21,12 +21,26 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- name: Install and run newline linter checks
if: matrix.os == 'ubuntu-latest'
run: |
wget https://github.com/fernandrone/linelint/releases/download/0.0.6/linelint-linux-amd64
echo "16b70fb7b471d6f95cbdc0b4e5dc2b0ac9e84ba9ecdc488f7bdf13df823aca4b linelint-linux-amd64" > checksum
sha256sum -c checksum || exit 1
chmod +x linelint-linux-amd64
mv linelint-linux-amd64 /usr/local/bin/linelint
make check_newline
- name: Run pcc checks
run: |
make pcc
- name: Build concrete-csprng
run: |
make build_concrete_csprng
- name: Build Release core
run: |
make build_core AVX512_SUPPORT=ON

View File

@@ -10,7 +10,7 @@ jobs:
- name: Check first line
uses: gsactions/commit-message-checker@16fa2d5de096ae0d35626443bcd24f1e756cafee
with:
pattern: '^((feat|fix|chore|refactor|style|test|docs|doc)\(\w+\)\:) .+$'
pattern: '^((feat|fix|chore|refactor|style|test|docs|doc)(\(\w+\))?\:) .+$'
flags: "gs"
error: 'Your first line has to contain a commit type and scope like "feat(my_feature): msg".'
excludeDescription: "true" # optional: this excludes the description body of a pull request

119
.github/workflows/code_coverage.yml vendored Normal file
View File

@@ -0,0 +1,119 @@
name: Code Coverage
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
# Allows you to run this workflow manually from the Actions tab as an alternative.
workflow_dispatch:
# All the inputs are provided by Slab
inputs:
instance_id:
description: "AWS instance ID"
type: string
instance_image_id:
description: "AWS instance AMI ID"
type: string
instance_type:
description: "AWS instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
type: string
fork_repo:
description: 'Name of forked repo as user/repo'
type: string
fork_git_sha:
description: 'Git SHA to checkout from fork'
type: string
jobs:
code-coverage:
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ inputs.runner_name }}
timeout-minutes: 1080
steps:
# Step used for log purpose.
- name: Instance configuration used
run: |
echo "ID: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
echo "Fork repo: ${{ inputs.fork_repo }}"
echo "Fork git sha: ${{ inputs.fork_git_sha }}"
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: stable
default: true
- name: Check for file changes
id: changed-files
uses: tj-actions/changed-files@1c938490c880156b746568a518594309cfb3f66b
with:
files_yaml: |
tfhe:
- tfhe/src/**
concrete_csprng:
- concrete-csprng/src/**
- name: Generate Keys
if: steps.changed-files.outputs.tfhe_any_changed == 'true'
run: |
make GEN_KEY_CACHE_COVERAGE_ONLY=TRUE gen_key_cache
make gen_key_cache_core_crypto
- name: Run coverage for core_crypto
if: steps.changed-files.outputs.tfhe_any_changed == 'true'
run: |
make test_core_crypto_cov AVX512_SUPPORT=ON
- name: Run coverage for boolean
if: steps.changed-files.outputs.tfhe_any_changed == 'true'
run: |
make test_boolean_cov
- name: Run coverage for shortint
if: steps.changed-files.outputs.tfhe_any_changed == 'true'
run: |
make test_shortint_cov
- name: Upload tfhe coverage to Codecov
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d
if: steps.changed-files.outputs.tfhe_any_changed == 'true'
with:
token: ${{ secrets.CODECOV_TOKEN }}
directory: ./coverage/
fail_ci_if_error: true
files: shortint/cobertura.xml,boolean/cobertura.xml,core_crypto/cobertura.xml,core_crypto_avx512/cobertura.xml
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Code coverage finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,74 @@
name: CSPRNG randomness testing Workflow
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
# Allows you to run this workflow manually from the Actions tab as an alternative.
workflow_dispatch:
# All the inputs are provided by Slab
inputs:
instance_id:
description: "AWS instance ID"
type: string
instance_image_id:
description: "AWS instance AMI ID"
type: string
instance_type:
description: "AWS instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
type: string
fork_repo:
description: 'Name of forked repo as user/repo'
type: string
fork_git_sha:
description: 'Git SHA to checkout from fork'
type: string
jobs:
csprng-randomness-teting:
name: CSPRNG randomness testing
concurrency:
group: ${{ github.workflow }}_${{ github.ref }}_${{ inputs.instance_image_id }}_${{ inputs.instance_type }}
cancel-in-progress: true
runs-on: ${{ inputs.runner_name }}
steps:
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: ${{ inputs.fork_repo }}
ref: ${{ inputs.fork_git_sha }}
- name: Set up home
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: stable
default: true
- name: Dieharder randomness test suite
run: |
make dieharder_csprng
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "concrete-csprng randomness check finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -5,24 +5,26 @@ on:
workflow_dispatch:
inputs:
instance_id:
description: 'Instance ID'
description: "Instance ID"
type: string
instance_image_id:
description: 'Instance AMI ID'
description: "Instance AMI ID"
type: string
instance_type:
description: 'Instance product type'
description: "Instance product type"
type: string
runner_name:
description: 'Action runner name'
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
description: "Slab request ID"
type: string
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
PARSE_INTEGER_BENCH_CSV_FILE: tfhe_rs_integer_benches_${{ github.sha }}.csv
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-integer-benchmarks:
@@ -42,7 +44,7 @@ jobs:
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
@@ -57,9 +59,20 @@ jobs:
toolchain: nightly
override: true
- name: Run benchmarks
- name: Run benchmarks with AVX512
run: |
make bench_integer
make AVX512_SUPPORT=ON FAST_BENCH=TRUE bench_integer
- name: Parse benchmarks to csv
run: |
make PARSE_INTEGER_BENCH_CSV_FILE=${{ env.PARSE_INTEGER_BENCH_CSV_FILE }} \
parse_integer_benches
- name: Upload csv results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_csv_integer
path: ${{ env.PARSE_INTEGER_BENCH_CSV_FILE }}
- name: Parse results
run: |
@@ -73,33 +86,17 @@ jobs:
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Remove previous raw results
run: |
rm -rf target/criterion
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON bench_integer
- name: Parse AVX512 results
run: |
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--hardware ${{ inputs.instance_type }} \
--walk-subdirs \
--name-suffix avx512 \
--throughput \
--append-results
- name: Upload parsed results artifact
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_integer
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
@@ -118,3 +115,15 @@ jobs:
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Integer benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,157 @@
# Run all integer benchmarks on an AWS instance and return parsed results to Slab CI bot.
name: Integer full benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
prepare-matrix:
name: Prepare operations matrix
runs-on: ubuntu-latest
outputs:
op_flavor: ${{ steps.set_op_flavor.outputs.op_flavor }}
steps:
- name: Weekly benchmarks
if: ${{ github.event.inputs.user_inputs == 'weekly_benchmarks' }}
run: |
echo "OP_FLAVOR=[\"default\", \"default_comp\", \"default_scalar\", \"default_scalar_comp\"]" >> ${GITHUB_ENV}
- name: Quarterly benchmarks
if: ${{ github.event.inputs.user_inputs == 'quarterly_benchmarks' }}
run: |
echo "OP_FLAVOR=[\"default\", \"default_comp\", \"default_scalar\", \"default_scalar_comp\", \
\"smart\", \"smart_comp\", \"smart_scalar\", \"smart_parallelized\", \"smart_parallelized_comp\", \"smart_scalar_parallelized\", \"smart_scalar_parallelized_comp\", \
\"unchecked\", \"unchecked_comp\", \"unchecked_scalar\", \"unchecked_scalar_comp\", \
\"misc\"]" >> ${GITHUB_ENV}
- name: Set operation flavor output
id: set_op_flavor
run: |
echo "op_flavor=${{ toJSON(env.OP_FLAVOR) }}" >> ${GITHUB_OUTPUT}
integer-benchmarks:
name: Execute integer benchmarks for all operations flavor
needs: prepare-matrix
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
continue-on-error: true
strategy:
max-parallel: 1
matrix:
command: [ integer, integer_multi_bit]
op_flavor: ${{ fromJson(needs.prepare-matrix.outputs.op_flavor) }}
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Get benchmark details
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
echo "COMMIT_DATE=$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})" >> "${GITHUB_ENV}"
echo "COMMIT_HASH=$(git describe --tags --dirty)" >> "${GITHUB_ENV}"
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON BENCH_OP_FLAVOR=${{ matrix.op_flavor }} bench_${{ matrix.command }}
- name: Parse results
run: |
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${{ env.COMMIT_HASH }}" \
--branch ${{ github.ref_name }} \
--commit-date "${{ env.COMMIT_DATE }}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_${{ matrix.command }}_${{ matrix.op_flavor }}
path: ${{ env.RESULTS_FILENAME }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
slack-notification:
name: Slack Notification
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ failure() }}
needs: integer-benchmarks
steps:
- name: Notify
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Integer full benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,129 @@
# Run integer benchmarks with multi-bit cryptographic parameters on an AWS instance and return parsed results to Slab CI bot.
name: Integer Multi-bit benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
PARSE_INTEGER_BENCH_CSV_FILE: tfhe_rs_integer_benches_${{ github.sha }}.csv
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-integer-benchmarks:
name: Execute integer multi-bit benchmarks in EC2
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Get benchmark date
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Run multi-bit benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON FAST_BENCH=TRUE bench_integer_multi_bit
- name: Parse benchmarks to csv
run: |
make PARSE_INTEGER_BENCH_CSV_FILE=${{ env.PARSE_INTEGER_BENCH_CSV_FILE }} \
parse_integer_benches
- name: Upload csv results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_csv_integer
path: ${{ env.PARSE_INTEGER_BENCH_CSV_FILE }}
- name: Parse results
run: |
COMMIT_DATE="$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})"
COMMIT_HASH="$(git describe --tags --dirty)"
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${COMMIT_HASH}" \
--branch ${{ github.ref_name }} \
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_integer
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Integer benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -4,11 +4,18 @@ on:
workflow_dispatch:
pull_request:
types: [labeled]
# Have a nightly build for M1 tests
schedule:
# * is a special character in YAML so you have to quote this string
# At 22:00 every day
# Timezone is UTC, so Paris time is +2 during the summer and +1 during winter
- cron: "0 22 * * *"
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-C target-cpu=native"
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
FAST_TESTS: "TRUE"
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
@@ -16,11 +23,11 @@ concurrency:
jobs:
cargo-builds:
if: "github.event_name != 'pull_request' || contains(github.event.label.name, 'm1_test')"
if: ${{ (github.event_name == 'schedule' && github.repository == 'zama-ai/tfhe-rs') || github.event_name == 'workflow_dispatch' || contains(github.event.label.name, 'm1_test') }}
runs-on: ["self-hosted", "m1mac"]
steps:
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- name: Install latest stable
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
@@ -32,6 +39,10 @@ jobs:
run: |
make pcc
- name: Build concrete-csprng
run: |
make build_concrete_csprng
- name: Build Release core
run: |
make build_core
@@ -56,6 +67,10 @@ jobs:
run: |
make build_c_api
- name: Run concrete-csprng tests
run: |
make test_concrete_csprng
- name: Run core tests
run: |
make test_core_crypto
@@ -87,6 +102,18 @@ jobs:
run: |
make test_integer_ci
- name: Gen Keys if required
run: |
make GEN_KEY_CACHE_MULTI_BIT_ONLY=TRUE gen_key_cache
- name: Run shortint multi bit tests
run: |
make test_shortint_multi_bit_ci
- name: Run integer multi bit tests
run: |
make test_integer_multi_bit_ci
remove_label:
name: Remove m1_test label
runs-on: ubuntu-latest
@@ -95,6 +122,7 @@ jobs:
if: ${{ always() }}
steps:
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0
if: ${{ github.event_name == 'pull_request' }}
with:
labels: m1_test
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -102,7 +130,7 @@ jobs:
- name: Slack Notification
if: ${{ needs.cargo-builds.result != 'skipped' }}
continue-on-error: true
uses: rtCamp/action-slack-notify@12e36fc18b0689399306c2e0b3e0f2978b7f1ee7
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ needs.cargo-builds.result }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}

84
.github/workflows/make_release.yml vendored Normal file
View File

@@ -0,0 +1,84 @@
# Publish new release of tfhe-rs on various platform.
name: Publish release
on:
workflow_dispatch:
inputs:
dry_run:
description: "Dry-run"
type: boolean
default: true
push_to_crates:
description: "Push to crate"
type: boolean
default: true
push_web_package:
description: "Push web js package"
type: boolean
default: true
push_node_package:
description: "Push node js package"
type: boolean
default: true
env:
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
publish_release:
name: Publish Release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Publish crate.io package
if: ${{ inputs.push_to_crates }}
env:
CRATES_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
DRY_RUN: ${{ inputs.dry_run && '--dry-run' || '' }}
run: |
cargo publish -p tfhe --token ${{ env.CRATES_TOKEN }} ${{ env.DRY_RUN }}
- name: Build web package
if: ${{ inputs.push_web_package }}
run: |
make build_web_js_api
- name: Publish web package
if: ${{ inputs.push_web_package }}
uses: JS-DevTools/npm-publish@4b07b26a2f6e0a51846e1870223e545bae91c552
with:
token: ${{ secrets.NPM_TOKEN }}
package: tfhe/pkg/package.json
dry-run: ${{ inputs.dry_run }}
- name: Build Node package
if: ${{ inputs.push_node_package }}
run: |
rm -rf tfhe/pkg
make build_node_js_api
sed -i 's/"tfhe"/"node-tfhe"/g' tfhe/pkg/package.json
- name: Publish Node package
if: ${{ inputs.push_node_package }}
uses: JS-DevTools/npm-publish@4b07b26a2f6e0a51846e1870223e545bae91c552
with:
token: ${{ secrets.NPM_TOKEN }}
package: tfhe/pkg/package.json
dry-run: ${{ inputs.dry_run }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "tfhe release failed: (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,42 @@
# Publish new release of tfhe-rs on various platform.
name: Publish concrete-csprng release
on:
workflow_dispatch:
inputs:
dry_run:
description: "Dry-run"
type: boolean
default: true
env:
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
publish_release:
name: Publish concrete-csprng Release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Publish crate.io package
env:
CRATES_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
DRY_RUN: ${{ inputs.dry_run && '--dry-run' || '' }}
run: |
cargo publish -p concrete-csprng --token ${{ env.CRATES_TOKEN }} ${{ env.DRY_RUN }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "concrete-csprng release failed: (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

51
.github/workflows/parameters_check.yml vendored Normal file
View File

@@ -0,0 +1,51 @@
# Perform a security check on all the cryptographic parameters set
name: Parameters curves security check
env:
CARGO_TERM_COLOR: always
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUSTFLAGS: "-C target-cpu=native"
on:
push:
branches:
- "main"
workflow_dispatch:
jobs:
params-curves-security-check:
runs-on: ubuntu-latest
steps:
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- name: Checkout lattice-estimator
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: malb/lattice-estimator
path: lattice_estimator
- name: Install Sage
run: |
sudo apt update
sudo apt install -y sagemath
- name: Collect parameters
run: |
CARGO_PROFILE=devo make write_params_to_file
- name: Perform security check
run: |
PYTHONPATH=lattice_estimator sage ci/lattice_estimator.sage
- name: Slack Notification
if: ${{ always() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Security check for parameters finished with status: ${{ job.status }}. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -5,25 +5,33 @@ on:
workflow_dispatch:
inputs:
instance_id:
description: 'Instance ID'
description: "Instance ID"
type: string
instance_image_id:
description: 'Instance AMI ID'
description: "Instance AMI ID"
type: string
instance_type:
description: 'Instance product type'
description: "Instance product type"
type: string
runner_name:
description: 'Action runner name'
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
description: "Slab request ID"
type: string
# This input is not used in this workflow but still mandatory since a calling workflow could
# use it. If a triggering command include a user_inputs field, then the triggered workflow
# must include this very input, otherwise the workflow won't be called.
# See start_full_benchmarks.yml as example.
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-pbs-benchmarks:
@@ -43,7 +51,7 @@ jobs:
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
@@ -78,13 +86,13 @@ jobs:
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_pbs
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
@@ -103,3 +111,15 @@ jobs:
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "PBS benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,14 @@
# Placeholder workflow file allowing running it without having to merge to main first
name: Placeholder Workflow
on:
workflow_dispatch:
jobs:
placeholder:
name: Placeholder
runs-on: ubuntu-latest
steps:
- run: |
echo "Hello this is a Placeholder Workflow"

View File

@@ -5,25 +5,25 @@ on:
workflow_dispatch:
inputs:
instance_id:
description: 'Instance ID'
description: "Instance ID"
type: string
instance_image_id:
description: 'Instance AMI ID'
description: "Instance AMI ID"
type: string
instance_type:
description: 'Instance product type'
description: "Instance product type"
type: string
runner_name:
description: 'Action runner name'
description: "Action runner name"
type: string
request_id:
description: 'Slab request ID'
description: "Slab request ID"
type: string
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-shortint-benchmarks:
@@ -43,7 +43,7 @@ jobs:
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
@@ -58,9 +58,9 @@ jobs:
toolchain: nightly
override: true
- name: Run benchmarks
- name: Run benchmarks with AVX512
run: |
make bench_shortint
make AVX512_SUPPORT=ON bench_shortint
- name: Parse results
run: |
@@ -74,24 +74,8 @@ jobs:
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--throughput
- name: Remove previous raw results
run: |
rm -rf target/criterion
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON bench_shortint
- name: Parse AVX512 results
run: |
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--hardware ${{ inputs.instance_type }} \
--walk-subdirs \
--name-suffix avx512 \
--throughput \
--append-results
--throughput
- name: Measure key sizes
run: |
@@ -104,13 +88,13 @@ jobs:
--append-results
- name: Upload parsed results artifact
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_shortint
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
@@ -129,3 +113,15 @@ jobs:
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Shortint benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,149 @@
# Run all shortint benchmarks on an AWS instance and return parsed results to Slab CI bot.
name: Shortint full benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
# This input is not used in this workflow but still mandatory since a calling workflow could
# use it. If a triggering command include a user_inputs field, then the triggered workflow
# must include this very input, otherwise the workflow won't be called.
# See start_full_benchmarks.yml as example.
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
shortint-benchmarks:
name: Execute shortint benchmarks for all operations flavor
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
strategy:
max-parallel: 1
matrix:
op_flavor: [ default, smart, unchecked ]
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Get benchmark details
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
echo "COMMIT_DATE=$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})" >> "${GITHUB_ENV}"
echo "COMMIT_HASH=$(git describe --tags --dirty)" >> "${GITHUB_ENV}"
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON BENCH_OP_FLAVOR=${{ matrix.op_flavor }} bench_shortint
- name: Parse results
run: |
COMMIT_DATE="$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})"
COMMIT_HASH="$(git describe --tags --dirty)"
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${COMMIT_HASH}" \
--branch ${{ github.ref_name }} \
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
# This small benchmark needs to be executed only once.
- name: Measure key sizes
if: matrix.op_flavor == 'default'
run: |
make measure_shortint_key_sizes
- name: Parse key sizes results
if: matrix.op_flavor == 'default'
run: |
python3 ./ci/benchmark_parser.py tfhe/shortint_key_sizes.csv ${{ env.RESULTS_FILENAME }} \
--key-sizes \
--append-results
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_shortint_${{ matrix.op_flavor }}
path: ${{ env.RESULTS_FILENAME }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
slack-notification:
name: Slack Notification
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ failure() }}
needs: shortint-benchmarks
steps:
- name: Notify
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Shortint full benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,129 @@
# Run signed integer benchmarks on an AWS instance and return parsed results to Slab CI bot.
name: Signed Integer benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
PARSE_INTEGER_BENCH_CSV_FILE: tfhe_rs_integer_benches_${{ github.sha }}.csv
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-integer-benchmarks:
name: Execute signed integer benchmarks in EC2
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Get benchmark date
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON FAST_BENCH=TRUE bench_signed_integer
- name: Parse benchmarks to csv
run: |
make PARSE_INTEGER_BENCH_CSV_FILE=${{ env.PARSE_INTEGER_BENCH_CSV_FILE }} \
parse_integer_benches
- name: Upload csv results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_csv_integer
path: ${{ env.PARSE_INTEGER_BENCH_CSV_FILE }}
- name: Parse results
run: |
COMMIT_DATE="$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})"
COMMIT_HASH="$(git describe --tags --dirty)"
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${COMMIT_HASH}" \
--branch ${{ github.ref_name }} \
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_integer
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Signed integer benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,133 @@
# Run all signed integer benchmarks on an AWS instance and return parsed results to Slab CI bot.
name: Signed Integer full benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
integer-benchmarks:
name: Execute signed integer benchmarks for all operations flavor
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
continue-on-error: true
strategy:
max-parallel: 1
matrix:
command: [ integer, integer_multi_bit ]
op_flavor: [ default, default_comp, default_scalar, default_scalar_comp,
unchecked, unchecked_comp, unchecked_scalar, unchecked_scalar_comp ]
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Get benchmark details
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
echo "COMMIT_DATE=$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})" >> "${GITHUB_ENV}"
echo "COMMIT_HASH=$(git describe --tags --dirty)" >> "${GITHUB_ENV}"
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Run benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON BENCH_OP_FLAVOR=${{ matrix.op_flavor }} bench_signed_${{ matrix.command }}
- name: Parse results
run: |
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${{ env.COMMIT_HASH }}" \
--branch ${{ github.ref_name }} \
--commit-date "${{ env.COMMIT_DATE }}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_${{ matrix.command }}_${{ matrix.op_flavor }}
path: ${{ env.RESULTS_FILENAME }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
slack-notification:
name: Slack Notification
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ failure() }}
needs: integer-benchmarks
steps:
- name: Notify
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Signed integer full benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -0,0 +1,129 @@
# Run signed integer benchmarks with multi-bit cryptographic parameters on an AWS instance and return parsed results to Slab CI bot.
name: Signed Integer Multi-bit benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
PARSE_INTEGER_BENCH_CSV_FILE: tfhe_rs_integer_benches_${{ github.sha }}.csv
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-integer-benchmarks:
name: Execute signed integer multi-bit benchmarks in EC2
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Get benchmark date
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Run multi-bit benchmarks with AVX512
run: |
make AVX512_SUPPORT=ON FAST_BENCH=TRUE bench_signed_integer_multi_bit
- name: Parse benchmarks to csv
run: |
make PARSE_INTEGER_BENCH_CSV_FILE=${{ env.PARSE_INTEGER_BENCH_CSV_FILE }} \
parse_integer_benches
- name: Upload csv results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_csv_integer
path: ${{ env.PARSE_INTEGER_BENCH_CSV_FILE }}
- name: Parse results
run: |
COMMIT_DATE="$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})"
COMMIT_HASH="$(git describe --tags --dirty)"
python3 ./ci/benchmark_parser.py target/criterion ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${COMMIT_HASH}" \
--branch ${{ github.ref_name }} \
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--walk-subdirs \
--name-suffix avx512 \
--throughput
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_integer
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "Signed integer benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

View File

@@ -4,24 +4,118 @@ name: Start all benchmarks
on:
push:
branches:
- 'main'
- "main"
workflow_dispatch:
inputs:
# The input name must be the name of the slab command to launch
boolean_bench:
description: "Run Boolean benches"
type: boolean
default: true
shortint_bench:
description: "Run shortint benches"
type: boolean
default: true
integer_bench:
description: "Run integer benches"
type: boolean
default: true
signed_integer_bench:
description: "Run signed integer benches"
type: boolean
default: true
integer_multi_bit_bench:
description: "Run integer multi bit benches"
type: boolean
default: true
signed_integer_multi_bit_bench:
description: "Run signed integer multi bit benches"
type: boolean
default: true
pbs_bench:
description: "Run PBS benches"
type: boolean
default: true
wasm_client_bench:
description: "Run WASM client benches"
type: boolean
default: true
jobs:
start-benchmarks:
if: ${{ (github.event_name == 'push' && github.repository == 'zama-ai/tfhe-rs') || github.event_name == 'workflow_dispatch' }}
strategy:
matrix:
command: [boolean_bench, shortint_bench, integer_bench, pbs_bench]
command: [ boolean_bench, shortint_bench,
integer_bench, integer_multi_bit_bench,
signed_integer_bench, signed_integer_multi_bit_bench,
pbs_bench, wasm_client_bench ]
runs-on: ubuntu-latest
steps:
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Check for file changes
id: changed-files
uses: tj-actions/changed-files@1c938490c880156b746568a518594309cfb3f66b
with:
files_yaml: |
common_benches:
- toolchain.txt
- Makefile
- ci/slab.toml
- tfhe/Cargo.toml
- tfhe/src/core_crypto/**
- .github/workflows/start_benchmarks.yml
boolean_bench:
- tfhe/src/boolean/**
- tfhe/benches/boolean/**
- .github/workflows/boolean_benchmark.yml
shortint_bench:
- tfhe/src/shortint/**
- tfhe/benches/shortint/**
- .github/workflows/shortint_benchmark.yml
integer_bench:
- tfhe/src/shortint/**
- tfhe/src/integer/**
- tfhe/benches/integer/bench.rs
- .github/workflows/integer_benchmark.yml
integer_multi_bit_bench:
- tfhe/src/shortint/**
- tfhe/src/integer/**
- tfhe/benches/integer/bench.rs
- .github/workflows/integer_multi_bit_benchmark.yml
signed_integer_bench:
- tfhe/src/shortint/**
- tfhe/src/integer/**
- tfhe/benches/integer/signed_bench.rs
- .github/workflows/signed_integer_benchmark.yml
signed_integer_multi_bit_bench:
- tfhe/src/shortint/**
- tfhe/src/integer/**
- tfhe/benches/integer/signed_bench.rs
- .github/workflows/signed_integer_multi_bit_benchmark.yml
pbs_bench:
- tfhe/src/core_crypto/**
- tfhe/benches/core_crypto/**
- .github/workflows/pbs_benchmark.yml
wasm_client_bench:
- tfhe/web_wasm_parallel_tests/**
- .github/workflows/wasm_client_benchmark.yml
- name: Checkout Slab repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Start AWS job in Slab
# If manually triggered check that the current bench has been requested
# Otherwise if it's on push check that files relevant to benchmarks have changed
if: (github.event_name == 'workflow_dispatch' && github.event.inputs[matrix.command] == 'true') || (github.event_name == 'push' && (steps.changed-files.outputs.common_benches_any_changed == 'true' || steps.changed-files.outputs[format('{0}_any_changed', matrix.command)] == 'true'))
shell: bash
run: |
echo -n '{"command": "${{ matrix.command }}", "git_ref": "${{ github.ref }}", "sha": "${{ github.sha }}"}' > command.json

View File

@@ -0,0 +1,65 @@
# Start all benchmark jobs, including full shortint and integer, on Slab CI bot.
name: Start full suite benchmarks
on:
schedule:
# Weekly benchmarks will be triggered each Saturday at 1a.m.
- cron: '0 1 * * 6'
# Quarterly benchmarks will be triggered right before end of quarter, the 25th of the current month at 4a.m.
# These benchmarks are far longer to execute hence the reason to run them only four time a year.
- cron: '0 4 25 MAR,JUN,SEP,DEC *'
workflow_dispatch:
inputs:
benchmark_type:
description: 'Benchmark type'
required: true
default: 'weekly'
type: choice
options:
- weekly
- quarterly
jobs:
start-benchmarks:
if: ${{ (github.event_name == 'schedule' && github.repository == 'zama-ai/tfhe-rs') || github.event_name == 'workflow_dispatch' }}
strategy:
matrix:
command: [ boolean_bench, shortint_full_bench, integer_full_bench,
signed_integer_full_bench, pbs_bench, wasm_client_bench ]
runs-on: ubuntu-latest
steps:
- name: Checkout tfhe-rs
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Set benchmarks type as weekly
if: (github.event_name == 'workflow_dispatch' && inputs.benchmark_type == 'weekly') || github.event.schedule == '0 1 * * 6'
run: |
echo "BENCH_TYPE=weekly_benchmarks" >> "${GITHUB_ENV}"
- name: Set benchmarks type as quarterly
if: (github.event_name == 'workflow_dispatch' && inputs.benchmark_type == 'quarterly') || github.event.schedule == '0 4 25 MAR,JUN,SEP,DEC *'
run: |
echo "BENCH_TYPE=quarterly_benchmarks" >> "${GITHUB_ENV}"
- name: Start AWS job in Slab
shell: bash
run: |
echo -n '{"command": "${{ matrix.command }}", "git_ref": "${{ github.ref }}", "sha": "${{ github.sha }}", "user_inputs": "${{ env.BENCH_TYPE }}"}' > command.json
SIGNATURE="$(slab/scripts/hmac_calculator.sh command.json '${{ secrets.JOB_SECRET }}')"
curl -v -k \
--fail-with-body \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: start_aws" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @command.json \
${{ secrets.SLAB_URL }}

View File

@@ -13,11 +13,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Save repo
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: repo-archive
path: '.'

View File

@@ -3,16 +3,52 @@ name: PR AWS build trigger
on:
pull_request:
pull_request_review:
types: [submitted]
jobs:
test:
trigger-tests:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- uses: mshick/add-pr-comment@a65df5f64fc741e91c59b8359a4bc56e57aaf5b1
- name: Get current labels
uses: snnaplab/get-labels-action@f426df40304808ace3b5282d4f036515f7609576
- name: Remove approved label
if: ${{ github.event_name == 'pull_request' && contains(fromJSON(env.LABELS), 'approved') }}
uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
labels: approved
- name: Launch fast tests
if: ${{ github.event_name == 'pull_request' }}
uses: mshick/add-pr-comment@a65df5f64fc741e91c59b8359a4bc56e57aaf5b1
with:
allow-repeats: true
message: |
@slab-ci cpu_fast_test
- name: Add approved label
uses: actions-ecosystem/action-add-labels@18f1af5e3544586314bbe15c0273249c770b2daf
if: ${{ github.event_name == 'pull_request_review' && github.event.review.state == 'approved' && !contains(fromJSON(env.LABELS), 'approved') }}
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
labels: approved
# PR label 'approved' presence is checked to avoid running the full test suite several times
# in case of multiple approvals without new commits in between.
- name: Launch full tests suite
if: ${{ github.event_name == 'pull_request_review' && github.event.review.state == 'approved' && !contains(fromJSON(env.LABELS), 'approved') }}
uses: mshick/add-pr-comment@a65df5f64fc741e91c59b8359a4bc56e57aaf5b1
with:
allow-repeats: true
message: |
Pull Request has been approved :tada:
Launching full test suite...
@slab-ci cpu_test
@slab-ci cpu_integer_test
@slab-ci cpu_unsigned_integer_test
@slab-ci cpu_signed_integer_test
@slab-ci cpu_wasm_test
@slab-ci csprng_randomness_testing

View File

@@ -0,0 +1,136 @@
# Run WASM client benchmarks on an AWS instance and return parsed results to Slab CI bot.
name: WASM client benchmarks
on:
workflow_dispatch:
inputs:
instance_id:
description: "Instance ID"
type: string
instance_image_id:
description: "Instance AMI ID"
type: string
instance_type:
description: "Instance product type"
type: string
runner_name:
description: "Action runner name"
type: string
request_id:
description: "Slab request ID"
type: string
# This input is not used in this workflow but still mandatory since a calling workflow could
# use it. If a triggering command include a user_inputs field, then the triggered workflow
# must include this very input, otherwise the workflow won't be called.
# See start_full_benchmarks.yml as example.
user_inputs:
description: "Type of benchmarks to run"
type: string
default: "weekly_benchmarks"
env:
CARGO_TERM_COLOR: always
RESULTS_FILENAME: parsed_benchmark_results_${{ github.sha }}.json
ACTION_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
jobs:
run-wasm-client-benchmarks:
name: Execute WASM client benchmarks in EC2
runs-on: ${{ github.event.inputs.runner_name }}
if: ${{ !cancelled() }}
steps:
- name: Instance configuration used
run: |
echo "IDs: ${{ inputs.instance_id }}"
echo "AMI: ${{ inputs.instance_image_id }}"
echo "Type: ${{ inputs.instance_type }}"
echo "Request ID: ${{ inputs.request_id }}"
- name: Get benchmark date
run: |
echo "BENCH_DATE=$(date --iso-8601=seconds)" >> "${GITHUB_ENV}"
- name: Checkout tfhe-rs repo with tags
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
fetch-depth: 0
- name: Set up home
# "Install rust" step require root user to have a HOME directory which is not set.
run: |
echo "HOME=/home/ubuntu" >> "${GITHUB_ENV}"
- name: Install rust
uses: actions-rs/toolchain@16499b5e05bf2e26879000db0c1d13f7e13fa3af
with:
toolchain: nightly
override: true
- name: Run benchmarks
run: |
make install_node
make ci_bench_web_js_api_parallel
- name: Parse results
run: |
make parse_wasm_benchmarks
COMMIT_DATE="$(git --no-pager show -s --format=%cd --date=iso8601-strict ${{ github.sha }})"
COMMIT_HASH="$(git describe --tags --dirty)"
python3 ./ci/benchmark_parser.py tfhe/wasm_pk_gen.csv ${{ env.RESULTS_FILENAME }} \
--database tfhe_rs \
--hardware ${{ inputs.instance_type }} \
--project-version "${COMMIT_HASH}" \
--branch ${{ github.ref_name }} \
--commit-date "${COMMIT_DATE}" \
--bench-date "${{ env.BENCH_DATE }}" \
--key-gen
- name: Measure public key and ciphertext sizes in HL Api
run: |
make measure_hlapi_compact_pk_ct_sizes
- name: Parse key and ciphertext sizes results
run: |
python3 ./ci/benchmark_parser.py tfhe/hlapi_cpk_and_cctl_sizes.csv ${{ env.RESULTS_FILENAME }} \
--key-gen \
--append-results
- name: Upload parsed results artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: ${{ github.sha }}_wasm
path: ${{ env.RESULTS_FILENAME }}
- name: Checkout Slab repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
with:
repository: zama-ai/slab
path: slab
token: ${{ secrets.CONCRETE_ACTIONS_TOKEN }}
- name: Send data to Slab
shell: bash
run: |
echo "Computing HMac on results file"
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${{ env.RESULTS_FILENAME }} '${{ secrets.JOB_SECRET }}')"
echo "Sending results to Slab..."
curl -v -k \
-H "Content-Type: application/json" \
-H "X-Slab-Repository: ${{ github.repository }}" \
-H "X-Slab-Command: store_data_v2" \
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
-d @${{ env.RESULTS_FILENAME }} \
${{ secrets.SLAB_URL }}
- name: Slack Notification
if: ${{ failure() }}
continue-on-error: true
uses: rtCamp/action-slack-notify@b24d75fe0e728a4bf9fc42ee217caa686d141ee8
env:
SLACK_COLOR: ${{ job.status }}
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_MESSAGE: "WASM benchmarks failed. (${{ env.ACTION_RUN_URL }})"
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

11
.gitignore vendored
View File

@@ -3,12 +3,19 @@ target/
.vscode/
# Path we use for internal-keycache during tests
./keys/
/keys/
# In case of symlinked keys
./keys
/keys
**/Cargo.lock
**/*.bin
# Some of our bench outputs
/tfhe/benchmarks_parameters
**/*.csv
# dieharder run log
dieharder_run.log
# Coverage reports
/coverage/

14
.linelint.yml Normal file
View File

@@ -0,0 +1,14 @@
ignore:
- .git
- target
- tfhe/benchmarks_parameters
- tfhe/web_wasm_parallel_tests/node_modules
- tfhe/web_wasm_parallel_tests/dist
- keys
- coverage
rules:
# checks if file ends in a newline character
end-of-file:
enable: true
single-new-line: true

131
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,131 @@
# Contributor Covenant Code of Conduct
## Our pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our standards
Examples of behavior that contributes to a positive environment for our
community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall
community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of
any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address,
without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting us anonymously through [this form](https://forms.gle/569j3cZqGRFgrR3u9).
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of
actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or permanent
ban.
### 3. Temporary ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the
community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][mozilla coc].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][faq]. Translations are available at
[https://www.contributor-covenant.org/translations][translations].
[faq]: https://www.contributor-covenant.org/faq
[homepage]: https://www.contributor-covenant.org
[mozilla coc]: https://github.com/mozilla/diversity
[translations]: https://www.contributor-covenant.org/translations
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html

View File

@@ -1,6 +1,6 @@
[workspace]
resolver = "2"
members = ["tfhe", "tasks"]
members = ["tfhe", "tasks", "apps/trivium", "concrete-csprng"]
[profile.bench]
lto = "fat"
@@ -8,6 +8,10 @@ lto = "fat"
[profile.release]
lto = "fat"
[profile.release_lto_off]
inherits = "release"
lto = "off"
# Compiles much faster for tests and allows reasonable performance for iterating
[profile.devo]
inherits = "dev"

528
Makefile
View File

@@ -3,14 +3,30 @@ OS:=$(shell uname)
RS_CHECK_TOOLCHAIN:=$(shell cat toolchain.txt | tr -d '\n')
CARGO_RS_CHECK_TOOLCHAIN:=+$(RS_CHECK_TOOLCHAIN)
TARGET_ARCH_FEATURE:=$(shell ./scripts/get_arch_feature.sh)
RS_BUILD_TOOLCHAIN:=$(shell \
( (echo $(TARGET_ARCH_FEATURE) | grep -q x86) && echo stable) || echo $(RS_CHECK_TOOLCHAIN))
RS_BUILD_TOOLCHAIN:=stable
CARGO_RS_BUILD_TOOLCHAIN:=+$(RS_BUILD_TOOLCHAIN)
CARGO_PROFILE?=release
MIN_RUST_VERSION:=1.65
MIN_RUST_VERSION:=$(shell grep '^rust-version[[:space:]]*=' tfhe/Cargo.toml | cut -d '=' -f 2 | xargs)
AVX512_SUPPORT?=OFF
WASM_RUSTFLAGS:=
BIG_TESTS_INSTANCE?=FALSE
GEN_KEY_CACHE_MULTI_BIT_ONLY?=FALSE
GEN_KEY_CACHE_COVERAGE_ONLY?=FALSE
PARSE_INTEGER_BENCH_CSV_FILE?=tfhe_rs_integer_benches.csv
FAST_TESTS?=FALSE
FAST_BENCH?=FALSE
BENCH_OP_FLAVOR?=DEFAULT
NODE_VERSION=20
# sed: -n, do not print input stream, -e means a script/expression
# 1,/version/ indicates from the first line, to the line matching version at the start of the line
# p indicates to print, so we keep only the start of the Cargo.toml until we hit the first version
# entry which should be the version of tfhe
TFHE_CURRENT_VERSION:=\
$(shell sed -n -e '1,/^version/p' tfhe/Cargo.toml | \
grep '^version[[:space:]]*=' | cut -d '=' -f 2 | xargs)
# Cargo has a hard time distinguishing between our package from the workspace and a package that
# could be a dependency, so we build an unambiguous spec here
TFHE_SPEC:=tfhe@$(TFHE_CURRENT_VERSION)
# This is done to avoid forgetting it, we still precise the RUSTFLAGS in the commands to be able to
# copy paste the command in the terminal and change them if required without forgetting the flags
export RUSTFLAGS?=-C target-cpu=native
@@ -21,6 +37,38 @@ else
AVX512_FEATURE=
endif
ifeq ($(GEN_KEY_CACHE_MULTI_BIT_ONLY),TRUE)
MULTI_BIT_ONLY=--multi-bit-only
else
MULTI_BIT_ONLY=
endif
ifeq ($(GEN_KEY_CACHE_COVERAGE_ONLY),TRUE)
COVERAGE_ONLY=--coverage-only
else
COVERAGE_ONLY=
endif
# Variables used only for regex_engine example
REGEX_STRING?=''
REGEX_PATTERN?=''
# Exclude these files from coverage reports
define COVERAGE_EXCLUDED_FILES
--exclude-files apps/trivium/src/trivium/* \
--exclude-files apps/trivium/src/kreyvium/* \
--exclude-files apps/trivium/src/static_deque/* \
--exclude-files apps/trivium/src/trans_ciphering/* \
--exclude-files tasks/src/* \
--exclude-files tfhe/benches/boolean/* \
--exclude-files tfhe/benches/core_crypto/* \
--exclude-files tfhe/benches/shortint/* \
--exclude-files tfhe/benches/integer/* \
--exclude-files tfhe/benches/* \
--exclude-files tfhe/examples/regex_engine/* \
--exclude-files tfhe/examples/utilities/*
endef
.PHONY: rs_check_toolchain # Echo the rust toolchain used for checks
rs_check_toolchain:
@echo $(RS_CHECK_TOOLCHAIN)
@@ -52,196 +100,402 @@ install_cargo_nextest: install_rs_build_toolchain
cargo $(CARGO_RS_BUILD_TOOLCHAIN) install cargo-nextest --locked || \
( echo "Unable to install cargo nextest, unknown error." && exit 1 )
.PHONY: install_wasm_pack # Install wasm-pack to build JS packages
install_wasm_pack: install_rs_build_toolchain
@wasm-pack --version > /dev/null 2>&1 || \
cargo $(CARGO_RS_BUILD_TOOLCHAIN) install wasm-pack || \
( echo "Unable to install cargo wasm-pack, unknown error." && exit 1 )
.PHONY: install_node # Install last version of NodeJS via nvm
install_node:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | $(SHELL)
source ~/.bashrc
$(SHELL) -i -c 'nvm install $(NODE_VERSION)' || \
( echo "Unable to install node, unknown error." && exit 1 )
.PHONY: install_dieharder # Install dieharder for apt distributions or macOS
install_dieharder:
@dieharder -h > /dev/null 2>&1 || \
if [[ "$(OS)" == "Linux" ]]; then \
sudo apt update && sudo apt install -y dieharder; \
elif [[ "$(OS)" == "Darwin" ]]; then\
brew install dieharder; \
fi || ( echo "Unable to install dieharder, unknown error." && exit 1 )
.PHONY: install_tarpaulin # Install tarpaulin to perform code coverage
install_tarpaulin: install_rs_build_toolchain
@cargo tarpaulin --version > /dev/null 2>&1 || \
cargo $(CARGO_RS_BUILD_TOOLCHAIN) install cargo-tarpaulin --locked || \
( echo "Unable to install cargo tarpaulin, unknown error." && exit 1 )
.PHONY: check_linelint_installed # Check if linelint newline linter is installed
check_linelint_installed:
@printf "\n" | linelint - > /dev/null 2>&1 || \
( echo "Unable to locate linelint. Try installing it: https://github.com/fernandrone/linelint/releases" && exit 1 )
.PHONY: fmt # Format rust code
fmt: install_rs_check_toolchain
cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" fmt
.PHONT: check_fmt # Check rust code format
.PHONY: check_fmt # Check rust code format
check_fmt: install_rs_check_toolchain
cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" fmt --check
.PHONY: fix_newline # Fix newline at end of file issues to be UNIX compliant
fix_newline: check_linelint_installed
linelint -a .
.PHONY: check_newline # Check for newline at end of file to be UNIX compliant
check_newline: check_linelint_installed
linelint .
.PHONY: clippy_core # Run clippy lints on core_crypto with and without experimental features
clippy_core: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE) \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE),experimental \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_boolean # Run clippy lints enabling the boolean features
clippy_boolean: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE),boolean \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_shortint # Run clippy lints enabling the shortint features
clippy_shortint: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE),shortint \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_integer # Run clippy lints enabling the integer features
clippy_integer: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE),integer \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy # Run clippy lints enabling the boolean, shortint, integer
clippy: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy --all-targets \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_c_api # Run clippy lints enabling the boolean, shortint and the C API
clippy_c_api: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=$(TARGET_ARCH_FEATURE),boolean-c-api,shortint-c-api \
-p tfhe -- --no-deps -D warnings
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_js_wasm_api # Run clippy lints enabling the boolean, shortint and the js wasm API
.PHONY: clippy_js_wasm_api # Run clippy lints enabling the boolean, shortint, integer and the js wasm API
clippy_js_wasm_api: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
--features=boolean-client-js-wasm-api,shortint-client-js-wasm-api \
-p tfhe -- --no-deps -D warnings
--features=boolean-client-js-wasm-api,shortint-client-js-wasm-api,integer-client-js-wasm-api \
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_tasks # Run clippy lints on helper tasks crate.
clippy_tasks:
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
-p tasks -- --no-deps -D warnings
.PHONY: clippy_trivium # Run clippy lints on Trivium app
clippy_trivium: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy \
-p tfhe-trivium -- --no-deps -D warnings
.PHONY: clippy_all_targets # Run clippy lints on all targets (benches, examples, etc.)
clippy_all_targets:
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy --all-targets \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache \
-p tfhe -- --no-deps -D warnings
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache,safe-deserialization \
-p $(TFHE_SPEC) -- --no-deps -D warnings
.PHONY: clippy_concrete_csprng # Run clippy lints on concrete-csprng
clippy_concrete_csprng:
RUSTFLAGS="$(RUSTFLAGS)" cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" clippy --all-targets \
--features=$(TARGET_ARCH_FEATURE) \
-p concrete-csprng -- --no-deps -D warnings
.PHONY: clippy_all # Run all clippy targets
clippy_all: clippy clippy_boolean clippy_shortint clippy_integer clippy_all_targets clippy_c_api \
clippy_js_wasm_api clippy_tasks clippy_core
clippy_js_wasm_api clippy_tasks clippy_core clippy_concrete_csprng clippy_trivium
.PHONY: clippy_fast # Run main clippy targets
clippy_fast: clippy clippy_all_targets clippy_c_api clippy_js_wasm_api clippy_tasks clippy_core
.PHONY: gen_key_cache # Run the script to generate keys and cache them for shortint tests
gen_key_cache: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example generates_test_keys \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache -p tfhe
clippy_fast: clippy clippy_all_targets clippy_c_api clippy_js_wasm_api clippy_tasks clippy_core \
clippy_concrete_csprng
.PHONY: build_core # Build core_crypto without experimental features
build_core: install_rs_build_toolchain install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE) -p tfhe
--features=$(TARGET_ARCH_FEATURE) -p $(TFHE_SPEC)
@if [[ "$(AVX512_SUPPORT)" == "ON" ]]; then \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),$(AVX512_FEATURE) -p tfhe; \
--features=$(TARGET_ARCH_FEATURE),$(AVX512_FEATURE) -p $(TFHE_SPEC); \
fi
.PHONY: build_core_experimental # Build core_crypto with experimental features
build_core_experimental: install_rs_build_toolchain install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),experimental -p tfhe
--features=$(TARGET_ARCH_FEATURE),experimental -p $(TFHE_SPEC)
@if [[ "$(AVX512_SUPPORT)" == "ON" ]]; then \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),experimental,$(AVX512_FEATURE) -p tfhe; \
--features=$(TARGET_ARCH_FEATURE),experimental,$(AVX512_FEATURE) -p $(TFHE_SPEC); \
fi
.PHONY: build_boolean # Build with boolean enabled
build_boolean: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean -p tfhe --all-targets
--features=$(TARGET_ARCH_FEATURE),boolean -p $(TFHE_SPEC) --all-targets
.PHONY: build_shortint # Build with shortint enabled
build_shortint: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),shortint -p tfhe --all-targets
--features=$(TARGET_ARCH_FEATURE),shortint -p $(TFHE_SPEC) --all-targets
.PHONY: build_integer # Build with integer enabled
build_integer: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),integer -p tfhe --all-targets
--features=$(TARGET_ARCH_FEATURE),integer -p $(TFHE_SPEC) --all-targets
.PHONY: build_tfhe_full # Build with boolean, shortint and integer enabled
build_tfhe_full: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer -p tfhe --all-targets
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer -p $(TFHE_SPEC) --all-targets
.PHONY: build_c_api # Build the C API for boolean and shortint
.PHONY: build_c_api # Build the C API for boolean, shortint and integer
build_c_api: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean-c-api,shortint-c-api,high-level-c-api \
-p tfhe
--features=$(TARGET_ARCH_FEATURE),boolean-c-api,shortint-c-api,high-level-c-api,safe-deserialization \
-p $(TFHE_SPEC)
.PHONY: build_c_api_experimental_deterministic_fft # Build the C API for boolean, shortint and integer with experimental deterministic FFT
build_c_api_experimental_deterministic_fft: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean-c-api,shortint-c-api,high-level-c-api,safe-deserialization,experimental-force_fft_algo_dif4 \
-p $(TFHE_SPEC)
.PHONY: build_web_js_api # Build the js API targeting the web browser
build_web_js_api: install_rs_build_toolchain
build_web_js_api: install_rs_build_toolchain install_wasm_pack
cd tfhe && \
RUSTFLAGS="$(WASM_RUSTFLAGS)" rustup run "$(RS_BUILD_TOOLCHAIN)" \
wasm-pack build --release --target=web \
-- --features=boolean-client-js-wasm-api,shortint-client-js-wasm-api
-- --features=boolean-client-js-wasm-api,shortint-client-js-wasm-api,integer-client-js-wasm-api
.PHONY: build_web_js_api_parallel # Build the js API targeting the web browser with parallelism support
build_web_js_api_parallel: install_rs_check_toolchain install_wasm_pack
cd tfhe && \
rustup component add rust-src --toolchain $(RS_CHECK_TOOLCHAIN) && \
RUSTFLAGS="$(WASM_RUSTFLAGS) -C target-feature=+atomics,+bulk-memory,+mutable-globals" rustup run $(RS_CHECK_TOOLCHAIN) \
wasm-pack build --release --target=web \
-- --features=boolean-client-js-wasm-api,shortint-client-js-wasm-api,integer-client-js-wasm-api,parallel-wasm-api \
-Z build-std=panic_abort,std
.PHONY: build_node_js_api # Build the js API targeting nodejs
build_node_js_api: install_rs_build_toolchain
build_node_js_api: install_rs_build_toolchain install_wasm_pack
cd tfhe && \
RUSTFLAGS="$(WASM_RUSTFLAGS)" rustup run "$(RS_BUILD_TOOLCHAIN)" \
wasm-pack build --release --target=nodejs \
-- --features=boolean-client-js-wasm-api,shortint-client-js-wasm-api
-- --features=boolean-client-js-wasm-api,shortint-client-js-wasm-api,integer-client-js-wasm-api
.PHONY: build_concrete_csprng # Build concrete_csprng
build_concrete_csprng: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) build --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE) -p concrete-csprng --all-targets
.PHONY: test_core_crypto # Run the tests of the core_crypto module including experimental ones
test_core_crypto: install_rs_build_toolchain install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),experimental -p tfhe -- core_crypto::
--features=$(TARGET_ARCH_FEATURE),experimental -p $(TFHE_SPEC) -- core_crypto::
@if [[ "$(AVX512_SUPPORT)" == "ON" ]]; then \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),experimental,$(AVX512_FEATURE) -p tfhe -- core_crypto::; \
--features=$(TARGET_ARCH_FEATURE),experimental,$(AVX512_FEATURE) -p $(TFHE_SPEC) -- core_crypto::; \
fi
.PHONY: test_core_crypto_cov # Run the tests of the core_crypto module with code coverage
test_core_crypto_cov: install_rs_build_toolchain install_rs_check_toolchain install_tarpaulin
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) tarpaulin --profile $(CARGO_PROFILE) \
--out xml --ignore-panics --output-dir coverage/core_crypto --line --engine llvm --timeout 500 \
--implicit-test-threads $(COVERAGE_EXCLUDED_FILES) \
--features=$(TARGET_ARCH_FEATURE),experimental,internal-keycache,__coverage \
-p $(TFHE_SPEC) -- core_crypto::
@if [[ "$(AVX512_SUPPORT)" == "ON" ]]; then \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) tarpaulin --profile $(CARGO_PROFILE) \
--out xml --ignore-panics --output-dir coverage/core_crypto_avx512 --line --engine llvm --timeout 500 \
--implicit-test-threads $(COVERAGE_EXCLUDED_FILES) \
--features=$(TARGET_ARCH_FEATURE),experimental,internal-keycache,__coverage,$(AVX512_FEATURE) \
-p $(TFHE_SPEC) -- core_crypto::; \
fi
.PHONY: test_boolean # Run the tests of the boolean module
test_boolean: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean -p tfhe -- boolean::
--features=$(TARGET_ARCH_FEATURE),boolean -p $(TFHE_SPEC) -- boolean::
.PHONY: test_c_api # Run the tests for the C API
test_c_api: build_c_api
.PHONY: test_boolean_cov # Run the tests of the boolean module with code coverage
test_boolean_cov: install_rs_check_toolchain install_tarpaulin
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) tarpaulin --profile $(CARGO_PROFILE) \
--out xml --ignore-panics --output-dir coverage/boolean --line --engine llvm --timeout 500 \
$(COVERAGE_EXCLUDED_FILES) \
--features=$(TARGET_ARCH_FEATURE),boolean,internal-keycache,__coverage \
-p $(TFHE_SPEC) -- boolean::
.PHONY: test_c_api_rs # Run the rust tests for the C API
test_c_api_rs: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean-c-api,shortint-c-api,high-level-c-api,safe-deserialization \
-p $(TFHE_SPEC) \
c_api
.PHONY: test_c_api_c # Run the C tests for the C API
test_c_api_c: build_c_api
./scripts/c_api_tests.sh
.PHONY: test_c_api # Run all the tests for the C API
test_c_api: test_c_api_rs test_c_api_c
.PHONY: test_shortint_ci # Run the tests for shortint ci
test_shortint_ci: install_rs_build_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
./scripts/shortint-tests.sh $(CARGO_RS_BUILD_TOOLCHAIN)
FAST_TESTS="$(FAST_TESTS)" \
./scripts/shortint-tests.sh --rust-toolchain $(CARGO_RS_BUILD_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)"
.PHONY: test_shortint_multi_bit_ci # Run the tests for shortint ci running only multibit tests
test_shortint_multi_bit_ci: install_rs_build_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/shortint-tests.sh --rust-toolchain $(CARGO_RS_BUILD_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --multi-bit
.PHONY: test_shortint # Run all the tests for shortint
test_shortint: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache -p tfhe -- shortint::
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache -p $(TFHE_SPEC) -- shortint::
.PHONY: test_shortint_cov # Run the tests of the shortint module with code coverage
test_shortint_cov: install_rs_check_toolchain install_tarpaulin
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) tarpaulin --profile $(CARGO_PROFILE) \
--out xml --ignore-panics --output-dir coverage/shortint --line --engine llvm --timeout 500 \
$(COVERAGE_EXCLUDED_FILES) \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache,__coverage \
-p $(TFHE_SPEC) -- shortint::
.PHONY: test_integer_ci # Run the tests for integer ci
test_integer_ci: install_rs_build_toolchain install_cargo_nextest
test_integer_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
./scripts/integer-tests.sh $(CARGO_RS_BUILD_TOOLCHAIN)
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --avx512-support "$(AVX512_SUPPORT)"
.PHONY: test_unsigned_integer_ci # Run the tests for unsigned integer ci
test_unsigned_integer_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --avx512-support "$(AVX512_SUPPORT)" \
--unsigned-only
.PHONY: test_signed_integer_ci # Run the tests for signed integer ci
test_signed_integer_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --avx512-support "$(AVX512_SUPPORT)" \
--signed-only
.PHONY: test_integer_multi_bit_ci # Run the tests for integer ci running only multibit tests
test_integer_multi_bit_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --multi-bit --avx512-support "$(AVX512_SUPPORT)"
.PHONY: test_unsigned_integer_multi_bit_ci # Run the tests for nsigned integer ci running only multibit tests
test_unsigned_integer_multi_bit_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --multi-bit --avx512-support "$(AVX512_SUPPORT)" \
--unsigned-only
.PHONY: test_signed_integer_multi_bit_ci # Run the tests for nsigned integer ci running only multibit tests
test_signed_integer_multi_bit_ci: install_rs_check_toolchain install_cargo_nextest
BIG_TESTS_INSTANCE="$(BIG_TESTS_INSTANCE)" \
FAST_TESTS="$(FAST_TESTS)" \
./scripts/integer-tests.sh --rust-toolchain $(CARGO_RS_CHECK_TOOLCHAIN) \
--cargo-profile "$(CARGO_PROFILE)" --multi-bit --avx512-support "$(AVX512_SUPPORT)" \
--signed-only
.PHONY: test_safe_deserialization # Run the tests for safe deserialization
test_safe_deserialization: install_rs_build_toolchain install_cargo_nextest
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache,safe-deserialization -p $(TFHE_SPEC) -- safe_deserialization::
.PHONY: test_integer # Run all the tests for integer
test_integer: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache -p tfhe -- integer::
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache -p $(TFHE_SPEC) -- integer::
.PHONY: test_high_level_api # Run all the tests for high_level_api
test_high_level_api: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache -p tfhe -- high_level_api::
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache -p $(TFHE_SPEC) \
-- high_level_api::
.PHONY: test_user_doc # Run tests from the .md documentation
test_user_doc: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) --doc \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache -p tfhe \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer,internal-keycache -p $(TFHE_SPEC) \
-- test_user_docs::
.PHONY: test_regex_engine # Run tests for regex_engine example
test_regex_engine: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--example regex_engine \
--features=$(TARGET_ARCH_FEATURE),integer
.PHONY: test_sha256_bool # Run tests for sha256_bool example
test_sha256_bool: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--example sha256_bool \
--features=$(TARGET_ARCH_FEATURE),boolean
.PHONY: test_examples # Run tests for examples
test_examples: test_sha256_bool test_regex_engine
.PHONY: test_trivium # Run tests for trivium
test_trivium: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
-p tfhe-trivium -- --test-threads=1 trivium::
.PHONY: test_kreyvium # Run tests for kreyvium
test_kreyvium: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
-p tfhe-trivium -- --test-threads=1 kreyvium::
.PHONY: test_concrete_csprng # Run concrete-csprng tests
test_concrete_csprng:
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE) -p concrete-csprng
.PHONY: doc # Build rust doc
doc: install_rs_check_toolchain
RUSTDOCFLAGS="--html-in-header katex-header.html" \
cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" doc \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer --no-deps
.PHONY: docs # Build rust doc alias for doc
docs: doc
.PHONY: lint_doc # Build rust doc with linting enabled
lint_doc: install_rs_check_toolchain
RUSTDOCFLAGS="--html-in-header katex-header.html -Dwarnings" \
cargo "$(CARGO_RS_CHECK_TOOLCHAIN)" doc \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,integer --no-deps
.PHONY: lint_docs # Build rust doc with linting enabled alias for lint_doc
lint_docs: lint_doc
.PHONY: format_doc_latex # Format the documentation latex equations to avoid broken rendering.
format_doc_latex:
cargo xtask format_latex_doc
@@ -252,18 +506,20 @@ format_doc_latex:
@printf "\n===============================\n"
.PHONY: check_compile_tests # Build tests in debug without running them
check_compile_tests: build_c_api
check_compile_tests:
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --no-run \
--features=$(TARGET_ARCH_FEATURE),experimental,boolean,shortint,integer,internal-keycache \
-p tfhe
@if [[ "$(OS)" == "Linux" || "$(OS)" == "Darwin" ]]; then \
./scripts/c_api_tests.sh --build-only; \
fi
--features=$(TARGET_ARCH_FEATURE),experimental,boolean,shortint,integer,internal-keycache,safe-deserialization \
-p $(TFHE_SPEC)
@if [[ "$(OS)" == "Linux" || "$(OS)" == "Darwin" ]]; then \
"$(MAKE)" build_c_api && \
./scripts/c_api_tests.sh --build-only; \
fi
.PHONY: build_nodejs_test_docker # Build a docker image with tools to run nodejs tests for wasm API
build_nodejs_test_docker:
DOCKER_BUILDKIT=1 docker build --build-arg RUST_TOOLCHAIN="$(RS_BUILD_TOOLCHAIN)" \
-f docker/Dockerfile.wasm_tests -t tfhe-wasm-tests .
-f docker/Dockerfile.wasm_tests --build-arg NODE_VERSION=$(NODE_VERSION) -t tfhe-wasm-tests .
.PHONY: test_nodejs_wasm_api_in_docker # Run tests for the nodejs on wasm API in a docker container
test_nodejs_wasm_api_in_docker: build_nodejs_test_docker
@@ -280,54 +536,186 @@ test_nodejs_wasm_api_in_docker: build_nodejs_test_docker
test_nodejs_wasm_api: build_node_js_api
cd tfhe && node --test js_on_wasm_tests
.PHONY: test_web_js_api_parallel # Run tests for the web wasm api
test_web_js_api_parallel: build_web_js_api_parallel
$(MAKE) -C tfhe/web_wasm_parallel_tests test
.PHONY: ci_test_web_js_api_parallel # Run tests for the web wasm api
ci_test_web_js_api_parallel: build_web_js_api_parallel
source ~/.nvm/nvm.sh && \
nvm install $(NODE_VERSION) && \
nvm use $(NODE_VERSION) && \
$(MAKE) -C tfhe/web_wasm_parallel_tests test-ci
.PHONY: no_tfhe_typo # Check we did not invert the h and f in tfhe
no_tfhe_typo:
@./scripts/no_tfhe_typo.sh
.PHONY: bench_integer # Run benchmarks for integer
.PHONY: no_dbg_log # Check we did not leave dbg macro calls in the rust code
no_dbg_log:
@./scripts/no_dbg_calls.sh
.PHONY: dieharder_csprng # Run the dieharder test suite on our CSPRNG implementation
dieharder_csprng: install_dieharder build_concrete_csprng
./scripts/dieharder_test.sh
#
# Benchmarks
#
.PHONY: bench_integer # Run benchmarks for unsigned integer
bench_integer: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) __TFHE_RS_FAST_BENCH=$(FAST_BENCH) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench integer-bench \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,$(AVX512_FEATURE) -p tfhe
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC) --
.PHONY: bench_integer_multi_bit # Run benchmarks for unsigned integer using multi-bit parameters
bench_integer_multi_bit: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_TYPE=MULTI_BIT \
__TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) __TFHE_RS_FAST_BENCH=$(FAST_BENCH) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench integer-bench \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC) --
.PHONY: bench_signed_integer # Run benchmarks for signed integer
bench_signed_integer: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) __TFHE_RS_FAST_BENCH=$(FAST_BENCH) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench integer-signed-bench \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC) --
.PHONY: bench_signed_integer_multi_bit # Run benchmarks for signed integer using multi-bit parameters
bench_signed_integer_multi_bit: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_TYPE=MULTI_BIT \
__TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) __TFHE_RS_FAST_BENCH=$(FAST_BENCH) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench integer-signed-bench \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC) --
.PHONY: bench_shortint # Run benchmarks for shortint
bench_shortint: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench shortint-bench \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache,$(AVX512_FEATURE) -p tfhe
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC)
.PHONY: bench_shortint_multi_bit # Run benchmarks for shortint using multi-bit parameters
bench_shortint_multi_bit: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" __TFHE_RS_BENCH_TYPE=MULTI_BIT \
__TFHE_RS_BENCH_OP_FLAVOR=$(BENCH_OP_FLAVOR) \
cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench shortint-bench \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC) --
.PHONY: bench_boolean # Run benchmarks for boolean
bench_boolean: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench boolean-bench \
--features=$(TARGET_ARCH_FEATURE),boolean,internal-keycache,$(AVX512_FEATURE) -p tfhe
--features=$(TARGET_ARCH_FEATURE),boolean,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC)
.PHONY: bench_pbs # Run benchmarks for PBS
bench_pbs: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) bench \
--bench pbs-bench \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,internal-keycache,$(AVX512_FEATURE) -p tfhe
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,internal-keycache,$(AVX512_FEATURE) -p $(TFHE_SPEC)
.PHONY: bench_web_js_api_parallel # Run benchmarks for the web wasm api
bench_web_js_api_parallel: build_web_js_api_parallel
$(MAKE) -C tfhe/web_wasm_parallel_tests bench
.PHONY: ci_bench_web_js_api_parallel # Run benchmarks for the web wasm api
ci_bench_web_js_api_parallel: build_web_js_api_parallel
source ~/.nvm/nvm.sh && \
nvm use node && \
$(MAKE) -C tfhe/web_wasm_parallel_tests bench-ci
#
# Utility tools
#
.PHONY: gen_key_cache # Run the script to generate keys and cache them for shortint tests
gen_key_cache: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example generates_test_keys \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,internal-keycache -- \
$(MULTI_BIT_ONLY) $(COVERAGE_ONLY)
.PHONY: gen_key_cache_core_crypto # Run function to generate keys and cache them for core_crypto tests
gen_key_cache_core_crypto: install_rs_build_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_BUILD_TOOLCHAIN) test --tests --profile $(CARGO_PROFILE) \
--features=$(TARGET_ARCH_FEATURE),experimental,internal-keycache -p $(TFHE_SPEC) -- --nocapture \
core_crypto::keycache::generate_keys
.PHONY: measure_hlapi_compact_pk_ct_sizes # Measure sizes of public keys and ciphertext for high-level API
measure_hlapi_compact_pk_ct_sizes: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example hlapi_compact_pk_ct_sizes \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache
.PHONY: measure_shortint_key_sizes # Measure sizes of bootstrapping and key switching keys for shortint
measure_shortint_key_sizes: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example shortint_key_sizes \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache
.PHONY: measure_boolean_key_sizes # Measure sizes of bootstrapping and key switching keys for boolean
measure_boolean_key_sizes: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run \
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example boolean_key_sizes \
--features=$(TARGET_ARCH_FEATURE),boolean,internal-keycache
.PHONY: parse_integer_benches # Run python parser to output a csv containing integer benches data
parse_integer_benches:
python3 ./ci/parse_integer_benches_to_csv.py \
--criterion-dir target/criterion \
--output-file "$(PARSE_INTEGER_BENCH_CSV_FILE)"
.PHONY: parse_wasm_benchmarks # Parse benchmarks performed with WASM web client into a CSV file
parse_wasm_benchmarks: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example wasm_benchmarks_parser \
--features=$(TARGET_ARCH_FEATURE),shortint,internal-keycache \
-- web_wasm_parallel_tests/test/benchmark_results
.PHONY: write_params_to_file # Gather all crypto parameters into a file with a Sage readable format.
write_params_to_file: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example write_params_to_file \
--features=$(TARGET_ARCH_FEATURE),boolean,shortint,internal-keycache
#
# Real use case examples
#
.PHONY: regex_engine # Run regex_engine example
regex_engine: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example regex_engine \
--features=$(TARGET_ARCH_FEATURE),integer \
-- $(REGEX_STRING) $(REGEX_PATTERN)
.PHONY: dark_market # Run dark market example
dark_market: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example dark_market \
--features=$(TARGET_ARCH_FEATURE),integer,internal-keycache \
-- fhe-modified fhe-parallel plain fhe
.PHONY: sha256_bool # Run sha256_bool example
sha256_bool: install_rs_check_toolchain
RUSTFLAGS="$(RUSTFLAGS)" cargo $(CARGO_RS_CHECK_TOOLCHAIN) run --profile $(CARGO_PROFILE) \
--example sha256_bool \
--features=$(TARGET_ARCH_FEATURE),boolean
.PHONY: pcc # pcc stands for pre commit checks
pcc: no_tfhe_typo check_fmt doc clippy_all check_compile_tests
pcc: no_tfhe_typo no_dbg_log check_fmt lint_doc clippy_all check_compile_tests
.PHONY: fpcc # pcc stands for pre commit checks, the f stands for fast
fpcc: no_tfhe_typo check_fmt doc clippy_fast check_compile_tests
fpcc: no_tfhe_typo no_dbg_log check_fmt lint_doc clippy_fast check_compile_tests
.PHONY: conformance # Automatically fix problems that can be fixed
conformance: fmt
conformance: fix_newline fmt
.PHONY: help # Generate list of targets with descriptions
help:

163
README.md
View File

@@ -31,7 +31,9 @@ implementation. The goal is to have a stable, simple, high-performance, and
production-ready library for all the advanced features of TFHE.
## Getting Started
The steps to run a first example are described below.
### Cargo.toml configuration
To use the latest version of `TFHE-rs` in your project, you first need to add it as a dependency in your `Cargo.toml`:
+ For x86_64-based machines running Unix-like OSes:
@@ -45,7 +47,7 @@ tfhe = { version = "*", features = ["boolean", "shortint", "integer", "x86_64-un
```toml
tfhe = { version = "*", features = ["boolean", "shortint", "integer", "aarch64-unix"] }
```
Note: users with ARM devices must use `TFHE-rs` by compiling using the `nightly` toolchain.
Note: users with ARM devices must compile `TFHE-rs` using a stable toolchain with version >= 1.72.
+ For x86_64-based machines with the [`rdseed instruction`](https://en.wikipedia.org/wiki/RDRAND)
@@ -57,95 +59,67 @@ tfhe = { version = "*", features = ["boolean", "shortint", "integer", "x86_64"]
Note: aarch64-based machines are not yet supported for Windows as it's currently missing an entropy source to be able to seed the [CSPRNGs](https://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator) used in TFHE-rs
## A simple example
Here is a full example:
``` rust
use tfhe::prelude::*;
use tfhe::{generate_keys, set_server_key, ConfigBuilder, FheUint32, FheUint8};
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Basic configuration to use homomorphic integers
let config = ConfigBuilder::default().build();
// Key generation
let (client_key, server_keys) = generate_keys(config);
let clear_a = 1344u32;
let clear_b = 5u32;
let clear_c = 7u8;
// Encrypting the input data using the (private) client_key
// FheUint32: Encrypted equivalent to u32
let mut encrypted_a = FheUint32::try_encrypt(clear_a, &client_key)?;
let encrypted_b = FheUint32::try_encrypt(clear_b, &client_key)?;
// FheUint8: Encrypted equivalent to u8
let encrypted_c = FheUint8::try_encrypt(clear_c, &client_key)?;
// On the server side:
set_server_key(server_keys);
// Clear equivalent computations: 1344 * 5 = 6720
let encrypted_res_mul = &encrypted_a * &encrypted_b;
// Clear equivalent computations: 1344 >> 5 = 42
encrypted_a = &encrypted_res_mul >> &encrypted_b;
// Clear equivalent computations: let casted_a = a as u8;
let casted_a: FheUint8 = encrypted_a.cast_into();
// Clear equivalent computations: min(42, 7) = 7
let encrypted_res_min = &casted_a.min(&encrypted_c);
// Operation between clear and encrypted data:
// Clear equivalent computations: 7 & 1 = 1
let encrypted_res = encrypted_res_min & 1_u8;
// Decrypting on the client side:
let clear_res: u8 = encrypted_res.decrypt(&client_key);
assert_eq!(clear_res, 1_u8);
Ok(())
}
```
To run this code, use the following command:
<p align="center"> <code> cargo run --release </code> </p>
Note that when running code that uses `tfhe-rs`, it is highly recommended
to run in release mode with cargo`s `--release` flag to have the best performances possible,
eg: `cargo run --release`.
to run in release mode with cargo's `--release` flag to have the best performances possible,
Here is a full example evaluating a Boolean circuit:
```rust
use tfhe::boolean::prelude::*;
fn main() {
// We generate a set of client/server keys, using the default parameters:
let (mut client_key, mut server_key) = gen_keys();
// We use the client secret key to encrypt two messages:
let ct_1 = client_key.encrypt(true);
let ct_2 = client_key.encrypt(false);
// We use the server public key to execute a boolean circuit:
// if ((NOT ct_2) NAND (ct_1 AND ct_2)) then (NOT ct_2) else (ct_1 AND ct_2)
let ct_3 = server_key.not(&ct_2);
let ct_4 = server_key.and(&ct_1, &ct_2);
let ct_5 = server_key.nand(&ct_3, &ct_4);
let ct_6 = server_key.mux(&ct_5, &ct_3, &ct_4);
// We use the client key to decrypt the output of the circuit:
let output = client_key.decrypt(&ct_6);
assert_eq!(output, true);
}
```
Another example of how the library can be used with shortints:
```rust
use tfhe::shortint::prelude::*;
fn main() {
// Generate a set of client/server keys
// with 2 bits of message and 2 bits of carry
let (client_key, server_key) = gen_keys(PARAM_MESSAGE_2_CARRY_2);
let msg1 = 3;
let msg2 = 2;
// Encrypt two messages using the (private) client key:
let ct_1 = client_key.encrypt(msg1);
let ct_2 = client_key.encrypt(msg2);
// Homomorphically compute an addition
let ct_add = server_key.unchecked_add(&ct_1, &ct_2);
// Define the Hamming weight function
// f: x -> sum of the bits of x
let f = |x:u64| x.count_ones() as u64;
// Generate the accumulator for the function
let acc = server_key.generate_accumulator(f);
// Compute the function over the ciphertext using the PBS
let ct_res = server_key.apply_lookup_table(&ct_add, &acc);
// Decrypt the ciphertext using the (private) client key
let output = client_key.decrypt(&ct_res);
assert_eq!(output, f(msg1 + msg2));
}
```
An example using integer:
```rust
use tfhe::integer::gen_keys_radix;
use tfhe::shortint::parameters::PARAM_MESSAGE_2_CARRY_2;
fn main() {
// We create keys to create 16 bits integers
// using 8 blocks of 2 bits
let (cks, sks) = gen_keys_radix(&PARAM_MESSAGE_2_CARRY_2, 8);
let clear_a = 2382u16;
let clear_b = 29374u16;
let mut a = cks.encrypt(clear_a as u64);
let mut b = cks.encrypt(clear_b as u64);
let encrypted_max = sks.smart_max_parallelized(&mut a, &mut b);
let decrypted_max: u64 = cks.decrypt(&encrypted_max);
assert_eq!(decrypted_max as u16, clear_a.max(clear_b))
}
```
## Contributing
@@ -167,6 +141,19 @@ libraries.
<img src="https://user-images.githubusercontent.com/5758427/231115030-21195b55-2629-4c01-9809-be5059243999.png">
</a>
## Citing TFHE-rs
To cite TFHE-rs in academic papers, please use the following entry:
```text
@Misc{TFHE-rs,
title={{TFHE-rs: A Pure Rust Implementation of the TFHE Scheme for Boolean and Integer Arithmetics Over Encrypted Data}},
author={Zama},
year={2022},
note={\url{https://github.com/zama-ai/tfhe-rs}},
}
```
## License
This software is distributed under the BSD-3-Clause-Clear license. If you have any questions,

24
apps/trivium/Cargo.toml Normal file
View File

@@ -0,0 +1,24 @@
[package]
name = "tfhe-trivium"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rayon = { version = "1.7.0"}
[target.'cfg(target_arch = "x86_64")'.dependencies.tfhe]
path = "../../tfhe"
features = [ "boolean", "shortint", "integer", "x86_64" ]
[target.'cfg(target_arch = "aarch64")'.dependencies.tfhe]
path = "../../tfhe"
features = [ "boolean", "shortint", "integer", "aarch64-unix" ]
[dev-dependencies]
criterion = { version = "0.5.1", features = [ "html_reports" ]}
[[bench]]
name = "trivium"
harness = false

204
apps/trivium/README.md Normal file
View File

@@ -0,0 +1,204 @@
# FHE boolean Trivium implementation using TFHE-rs
The cleartext boolean Trivium is available to be built using the function `TriviumStream::<bool>::new`.
This takes as input 2 arrays of 80 bool: the Trivium key and the IV. After initialization, it returns a TriviumStream on
which the user can call `next`, getting the next bit of the cipher stream, or `next_64`, which will compute 64 values at once,
using multithreading to accelerate the computation.
Quite similarly, the function `TriviumStream::<FheBool>::new` will return a very similar object running in FHE space. Its arguments are
2 arrays of 80 FheBool representing the encrypted Trivium key, and the encrypted IV. It also requires a reference to the the server key of the
current scheme. This means that any user of this feature must also have the `tfhe-rs` crate as a dependency.
Example of a Rust main below:
```rust
use tfhe::{ConfigBuilder, generate_keys, FheBool};
use tfhe::prelude::*;
use tfhe_trivium::TriviumStream;
fn get_hexadecimal_string_from_lsb_first_stream(a: Vec<bool>) -> String {
assert!(a.len() % 8 == 0);
let mut hexadecimal: String = "".to_string();
for test in a.chunks(8) {
// Encoding is bytes in LSB order
match test[4..8] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => ()
};
match test[0..4] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => ()
};
}
return hexadecimal;
}
fn main() {
let config = ConfigBuilder::all_disabled().enable_default_bool().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [false; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i+2], 16).unwrap();
for j in 0..8 {
key[8*(i>>1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [false; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i+2], 16).unwrap();
for j in 0..8 {
iv[8*(i>>1) + j] = val % 2 == 1;
val >>= 1;
}
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let cipher_iv = iv.map(|x| FheBool::encrypt(x, &client_key));
let mut trivium = TriviumStream::<FheBool>::new(cipher_key, cipher_iv, &server_key);
let mut vec = Vec::<bool>::with_capacity(64*8);
while vec.len() < 64*8 {
let cipher_outputs = trivium.next_64();
for c in cipher_outputs {
vec.push(c.decrypt(&client_key))
}
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64*2]);
}
```
# FHE byte Trivium implementation
The same objects have also been implemented to stream bytes instead of booleans. They can be constructed and used in the same way via the functions `TriviumStreamByte::<u8>::new` and
`TriviumStreamByte::<FheUint8>::new` with the same arguments as before. The `FheUint8` version is significantly slower than the `FheBool` version, because not running
with the same cryptographic parameters. Its interest lie in its trans-ciphering capabilities: `TriviumStreamByte<FheUint8>` implements the trait `TransCiphering`,
meaning it implements the functions `trans_encrypt_64`. This function takes as input a `FheUint64` and outputs a `FheUint64`, the output being
encrypted via tfhe and trivium. For convenience we also provide `trans_decrypt_64`, but this is of course the exact same function.
Other sizes than 64 bit are expected to be available in the future.
# FHE shortint Trivium implementation
The same implementation is also available for generic Ciphertexts representing bits (meant to be used with parameters `PARAM_MESSAGE_1_CARRY_1_KS_PBS`). It uses a lower level API
of tfhe-rs, so the syntax is a little bit different. It also implements the `TransCiphering` trait. For optimization purposes, it does not internally run on the same
cryptographic parameters as the high level API of tfhe-rs. As such, it requires the usage of a casting key, to switch from one parameter space to another, which makes
its setup a little more intricate.
Example code:
```rust
use tfhe::shortint::prelude::*;
use tfhe::shortint::CastingKey;
use tfhe::{ConfigBuilder, generate_keys, FheUint64};
use tfhe::prelude::*;
use tfhe_trivium::TriviumStreamShortint;
fn test_shortint() {
let config = ConfigBuilder::all_disabled().enable_default_integers().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = CastingKey::new((&client_key, &server_key), (&hl_client_key, &hl_server_key));
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i+2], 16).unwrap();
for j in 0..8 {
key[8*(i>>1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i+2], 16).unwrap();
for j in 0..8 {
iv[8*(i>>1) + j] = val % 2;
val >>= 1;
}
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| client_key.encrypt(x));
let cipher_iv = iv.map(|x| client_key.encrypt(x));
let mut ciphered_message = vec![FheUint64::try_encrypt(0u64, &hl_client_key).unwrap(); 9];
let mut trivium = TriviumStreamShortint::new(cipher_key, cipher_iv, &server_key, &ksk);
let mut vec = Vec::<u64>::with_capacity(8);
while vec.len() < 8 {
let trans_ciphered_message = trivium.trans_encrypt_64(ciphered_message.pop().unwrap(), &hl_server_key);
vec.push(trans_ciphered_message.decrypt(&hl_client_key));
}
let hexadecimal = get_hexagonal_string_from_u64(vec);
assert_eq!(output_0_63, hexadecimal[0..64*2]);
}
```
# FHE Kreyvium implementation using tfhe-rs crate
This will work in exactly the same way as the Trivium implementation, except that the key and iv need to be 128 bits now. Available for the same internal types as Trivium, with similar syntax.
`KreyviumStreamByte<FheUint8>` and `KreyviumStreamShortint` also implement the `TransCiphering` trait.
# Testing
If you wish to run tests on this app, please run `cargo test -r trivium -- --test-threads=1` as multithreading provokes interferences between several running
Triviums at the same time.

View File

@@ -0,0 +1,75 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheBool};
use tfhe_trivium::KreyviumStream;
use criterion::Criterion;
pub fn kreyvium_bool_gen(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [false; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [false; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let mut kreyvium = KreyviumStream::<FheBool>::new(cipher_key, iv, &server_key);
c.bench_function("kreyvium bool generate 64 bits", |b| {
b.iter(|| kreyvium.next_64())
});
}
pub fn kreyvium_bool_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [false; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [false; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
c.bench_function("kreyvium bool warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let _kreyvium = KreyviumStream::<FheBool>::new(cipher_key, iv, &server_key);
})
});
}

View File

@@ -0,0 +1,93 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheUint64, FheUint8};
use tfhe_trivium::{KreyviumStreamByte, TransCiphering};
use criterion::Criterion;
pub fn kreyvium_byte_gen(c: &mut Criterion) {
let config = ConfigBuilder::default()
.enable_function_evaluation()
.build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let mut kreyvium = KreyviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
c.bench_function("kreyvium byte generate 64 bits", |b| {
b.iter(|| kreyvium.next_64())
});
}
pub fn kreyvium_byte_trans(c: &mut Criterion) {
let config = ConfigBuilder::default()
.enable_function_evaluation()
.build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let ciphered_message = FheUint64::try_encrypt(0u64, &client_key).unwrap();
let mut kreyvium = KreyviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
c.bench_function("kreyvium byte transencrypt 64 bits", |b| {
b.iter(|| kreyvium.trans_encrypt_64(ciphered_message.clone()))
});
}
pub fn kreyvium_byte_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default()
.enable_function_evaluation()
.build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
c.bench_function("kreyvium byte warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let _kreyvium = KreyviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
})
});
}

View File

@@ -0,0 +1,149 @@
use tfhe::prelude::*;
use tfhe::shortint::prelude::*;
use tfhe::shortint::KeySwitchingKey;
use tfhe::{generate_keys, ConfigBuilder, FheUint64};
use tfhe_trivium::{KreyviumStreamShortint, TransCiphering};
use criterion::Criterion;
pub fn kreyvium_shortint_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
c.bench_function("kreyvium 1_1 warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| client_key.encrypt(x));
let _kreyvium = KreyviumStreamShortint::new(
cipher_key,
iv,
server_key.clone(),
ksk.clone(),
hl_server_key.clone(),
);
})
});
}
pub fn kreyvium_shortint_gen(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let cipher_key = key.map(|x| client_key.encrypt(x));
let mut kreyvium = KreyviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
c.bench_function("kreyvium 1_1 generate 64 bits", |b| {
b.iter(|| kreyvium.next_64())
});
}
pub fn kreyvium_shortint_trans(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let cipher_key = key.map(|x| client_key.encrypt(x));
let ciphered_message = FheUint64::try_encrypt(0u64, &hl_client_key).unwrap();
let mut kreyvium = KreyviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
c.bench_function("kreyvium 1_1 transencrypt 64 bits", |b| {
b.iter(|| kreyvium.trans_encrypt_64(ciphered_message.clone()))
});
}

View File

@@ -0,0 +1,53 @@
use criterion::{criterion_group, criterion_main};
mod trivium_bool;
criterion_group!(
trivium_bool,
trivium_bool::trivium_bool_gen,
trivium_bool::trivium_bool_warmup
);
mod kreyvium_bool;
criterion_group!(
kreyvium_bool,
kreyvium_bool::kreyvium_bool_gen,
kreyvium_bool::kreyvium_bool_warmup
);
mod trivium_shortint;
criterion_group!(
trivium_shortint,
trivium_shortint::trivium_shortint_gen,
trivium_shortint::trivium_shortint_warmup,
trivium_shortint::trivium_shortint_trans
);
mod kreyvium_shortint;
criterion_group!(
kreyvium_shortint,
kreyvium_shortint::kreyvium_shortint_gen,
kreyvium_shortint::kreyvium_shortint_warmup,
kreyvium_shortint::kreyvium_shortint_trans
);
mod trivium_byte;
criterion_group!(
trivium_byte,
trivium_byte::trivium_byte_gen,
trivium_byte::trivium_byte_trans,
trivium_byte::trivium_byte_warmup
);
mod kreyvium_byte;
criterion_group!(
kreyvium_byte,
kreyvium_byte::kreyvium_byte_gen,
kreyvium_byte::kreyvium_byte_trans,
kreyvium_byte::kreyvium_byte_warmup
);
criterion_main!(
trivium_bool,
trivium_shortint,
trivium_byte,
kreyvium_bool,
kreyvium_shortint,
kreyvium_byte,
);

View File

@@ -0,0 +1,75 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheBool};
use tfhe_trivium::TriviumStream;
use criterion::Criterion;
pub fn trivium_bool_gen(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [false; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [false; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let mut trivium = TriviumStream::<FheBool>::new(cipher_key, iv, &server_key);
c.bench_function("trivium bool generate 64 bits", |b| {
b.iter(|| trivium.next_64())
});
}
pub fn trivium_bool_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [false; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [false; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
c.bench_function("trivium bool warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let _trivium = TriviumStream::<FheBool>::new(cipher_key, iv, &server_key);
})
});
}

View File

@@ -0,0 +1,87 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheUint64, FheUint8};
use tfhe_trivium::{TransCiphering, TriviumStreamByte};
use criterion::Criterion;
pub fn trivium_byte_gen(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let mut trivium = TriviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
c.bench_function("trivium byte generate 64 bits", |b| {
b.iter(|| trivium.next_64())
});
}
pub fn trivium_byte_trans(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let ciphered_message = FheUint64::try_encrypt(0u64, &client_key).unwrap();
let mut trivium = TriviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
c.bench_function("trivium byte transencrypt 64 bits", |b| {
b.iter(|| trivium.trans_encrypt_64(ciphered_message.clone()))
});
}
pub fn trivium_byte_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
c.bench_function("trivium byte warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let _trivium = TriviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
})
});
}

View File

@@ -0,0 +1,149 @@
use tfhe::prelude::*;
use tfhe::shortint::prelude::*;
use tfhe::shortint::KeySwitchingKey;
use tfhe::{generate_keys, ConfigBuilder, FheUint64};
use tfhe_trivium::{TransCiphering, TriviumStreamShortint};
use criterion::Criterion;
pub fn trivium_shortint_warmup(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
c.bench_function("trivium 1_1 warmup", |b| {
b.iter(|| {
let cipher_key = key.map(|x| client_key.encrypt(x));
let _trivium = TriviumStreamShortint::new(
cipher_key,
iv,
server_key.clone(),
ksk.clone(),
hl_server_key.clone(),
);
})
});
}
pub fn trivium_shortint_gen(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let cipher_key = key.map(|x| client_key.encrypt(x));
let mut trivium = TriviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
c.bench_function("trivium 1_1 generate 64 bits", |b| {
b.iter(|| trivium.next_64())
});
}
pub fn trivium_shortint_trans(c: &mut Criterion) {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let cipher_key = key.map(|x| client_key.encrypt(x));
let ciphered_message = FheUint64::try_encrypt(0u64, &hl_client_key).unwrap();
let mut trivium = TriviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
c.bench_function("trivium 1_1 transencrypt 64 bits", |b| {
b.iter(|| trivium.trans_encrypt_64(ciphered_message.clone()))
});
}

View File

@@ -0,0 +1,257 @@
//! This module implements the Kreyvium stream cipher, using booleans or FheBool
//! for the representation of the inner bits.
use crate::static_deque::StaticDeque;
use tfhe::prelude::*;
use tfhe::{set_server_key, unset_server_key, FheBool, ServerKey};
use rayon::prelude::*;
/// Internal trait specifying which operations are necessary for KreyviumStream generic type
pub trait KreyviumBoolInput<OpOutput>:
Sized
+ Clone
+ std::ops::BitXor<Output = OpOutput>
+ std::ops::BitAnd<Output = OpOutput>
+ std::ops::Not<Output = OpOutput>
{
}
impl KreyviumBoolInput<bool> for bool {}
impl KreyviumBoolInput<bool> for &bool {}
impl KreyviumBoolInput<FheBool> for FheBool {}
impl KreyviumBoolInput<FheBool> for &FheBool {}
/// KreyviumStream: a struct implementing the Kreyvium stream cipher, using T for the internal
/// representation of bits (bool or FheBool). To be able to compute FHE operations, it also owns
/// an Option for a ServerKey.
pub struct KreyviumStream<T> {
a: StaticDeque<93, T>,
b: StaticDeque<84, T>,
c: StaticDeque<111, T>,
k: StaticDeque<128, T>,
iv: StaticDeque<128, T>,
fhe_key: Option<ServerKey>,
}
impl KreyviumStream<bool> {
/// Constructor for `KreyviumStream<bool>`: arguments are the secret key and the input vector.
/// Outputs a KreyviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(mut key: [bool; 128], mut iv: [bool; 128]) -> KreyviumStream<bool> {
// Initialization of Kreyvium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register = [false; 93];
let mut b_register = [false; 84];
let mut c_register = [false; 111];
for i in 0..93 {
a_register[i] = key[128 - 93 + i];
}
for i in 0..84 {
b_register[i] = iv[128 - 84 + i];
}
for i in 0..44 {
c_register[111 - 44 + i] = iv[i];
}
for i in 0..66 {
c_register[i + 1] = true;
}
key.reverse();
iv.reverse();
KreyviumStream::<bool>::new_from_registers(
a_register, b_register, c_register, key, iv, None,
)
}
}
impl KreyviumStream<FheBool> {
/// Constructor for `KreyviumStream<FheBool>`: arguments are the encrypted secret key and input
/// vector, and the FHE server key.
/// Outputs a KreyviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(
mut key: [FheBool; 128],
mut iv: [bool; 128],
sk: &ServerKey,
) -> KreyviumStream<FheBool> {
set_server_key(sk.clone());
// Initialization of Kreyvium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register = [false; 93].map(FheBool::encrypt_trivial);
let mut b_register = [false; 84].map(FheBool::encrypt_trivial);
let mut c_register = [false; 111].map(FheBool::encrypt_trivial);
for i in 0..93 {
a_register[i] = key[128 - 93 + i].clone();
}
for i in 0..84 {
b_register[i] = FheBool::encrypt_trivial(iv[128 - 84 + i]);
}
for i in 0..44 {
c_register[111 - 44 + i] = FheBool::encrypt_trivial(iv[i]);
}
for i in 0..66 {
c_register[i + 1] = FheBool::encrypt_trivial(true);
}
key.reverse();
iv.reverse();
let iv = iv.map(FheBool::encrypt_trivial);
unset_server_key();
KreyviumStream::<FheBool>::new_from_registers(
a_register,
b_register,
c_register,
key,
iv,
Some(sk.clone()),
)
}
}
impl<T> KreyviumStream<T>
where
T: KreyviumBoolInput<T> + std::marker::Send + std::marker::Sync,
for<'a> &'a T: KreyviumBoolInput<T>,
{
/// Internal generic constructor: arguments are already prepared registers, and an optional FHE
/// server key
fn new_from_registers(
a_register: [T; 93],
b_register: [T; 84],
c_register: [T; 111],
k_register: [T; 128],
iv_register: [T; 128],
key: Option<ServerKey>,
) -> Self {
let mut ret = Self {
a: StaticDeque::<93, T>::new(a_register),
b: StaticDeque::<84, T>::new(b_register),
c: StaticDeque::<111, T>::new(c_register),
k: StaticDeque::<128, T>::new(k_register),
iv: StaticDeque::<128, T>::new(iv_register),
fhe_key: key,
};
ret.init();
ret
}
/// The specification of Kreyvium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes one turn of the stream, updating registers and outputting the new bit.
pub fn next_bool(&mut self) -> T {
match &self.fhe_key {
Some(sk) => set_server_key(sk.clone()),
None => (),
};
let [o, a, b, c] = self.get_output_and_values(0);
self.a.push(a);
self.b.push(b);
self.c.push(c);
self.k.shift();
self.iv.shift();
o
}
/// Computes a potential future step of Kreyvium, n terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, n: usize) -> [T; 4] {
assert!(n < 65);
let (((temp_a, temp_b), (temp_c, a_and)), (b_and, c_and)) = rayon::join(
|| {
rayon::join(
|| {
rayon::join(
|| &self.a[65 - n] ^ &self.a[92 - n],
|| &self.b[68 - n] ^ &self.b[83 - n],
)
},
|| {
rayon::join(
|| &(&self.c[65 - n] ^ &self.c[110 - n]) ^ &self.k[127 - n],
|| &(&self.a[91 - n] & &self.a[90 - n]) ^ &self.iv[127 - n],
)
},
)
},
|| {
rayon::join(
|| &self.b[82 - n] & &self.b[81 - n],
|| &self.c[109 - n] & &self.c[108 - n],
)
},
);
let ((o, a), (b, c)) = rayon::join(
|| {
rayon::join(
|| &(&temp_a ^ &temp_b) ^ &temp_c,
|| &temp_c ^ &(&c_and ^ &self.a[68 - n]),
)
},
|| {
rayon::join(
|| &temp_a ^ &(&a_and ^ &self.b[77 - n]),
|| &temp_b ^ &(&b_and ^ &self.c[86 - n]),
)
},
);
[o, a, b, c]
}
/// This calls `get_output_and_values` in parallel 64 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[T; 4]> {
(0..64)
.into_par_iter()
.map(|x| self.get_output_and_values(x))
.rev()
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<T> {
match &self.fhe_key {
Some(sk) => {
rayon::broadcast(|_| set_server_key(sk.clone()));
}
None => (),
}
let mut values = self.get_64_output_and_values();
match &self.fhe_key {
Some(_) => {
rayon::broadcast(|_| unset_server_key());
}
None => (),
}
let mut ret = Vec::<T>::with_capacity(64);
while let Some([o, a, b, c]) = values.pop() {
ret.push(o);
self.a.push(a);
self.b.push(b);
self.c.push(c);
}
self.k.n_shifts(64);
self.iv.n_shifts(64);
ret
}
}

View File

@@ -0,0 +1,293 @@
//! This module implements the Kreyvium stream cipher, using u8 or FheUint8
//! for the representation of the inner bits.
use crate::static_deque::{StaticByteDeque, StaticByteDequeInput};
use tfhe::prelude::*;
use tfhe::{set_server_key, unset_server_key, FheUint8, ServerKey};
use rayon::prelude::*;
/// Internal trait specifying which operations are necessary for KreyviumStreamByte generic type
pub trait KreyviumByteInput<OpOutput>:
Sized
+ Send
+ Sync
+ Clone
+ StaticByteDequeInput<OpOutput>
+ std::ops::BitXor<Output = OpOutput>
+ std::ops::BitAnd<Output = OpOutput>
+ std::ops::Shr<u8, Output = OpOutput>
+ std::ops::Shl<u8, Output = OpOutput>
+ std::ops::Add<Output = OpOutput>
{
}
impl KreyviumByteInput<u8> for u8 {}
impl KreyviumByteInput<u8> for &u8 {}
impl KreyviumByteInput<FheUint8> for FheUint8 {}
impl KreyviumByteInput<FheUint8> for &FheUint8 {}
/// KreyviumStreamByte: a struct implementing the Kreyvium stream cipher, using T for the internal
/// representation of bits (u8 or FheUint8). To be able to compute FHE operations, it also owns
/// an Option for a ServerKey.
/// Since the original Kreyvium registers' sizes are not a multiple of 8, these registers (which
/// store byte-like objects) have a size that is the eighth of the closest multiple of 8 above the
/// originals' sizes.
pub struct KreyviumStreamByte<T> {
a_byte: StaticByteDeque<12, T>,
b_byte: StaticByteDeque<11, T>,
c_byte: StaticByteDeque<14, T>,
k_byte: StaticByteDeque<16, T>,
iv_byte: StaticByteDeque<16, T>,
fhe_key: Option<ServerKey>,
}
impl KreyviumStreamByte<u8> {
/// Constructor for `KreyviumStreamByte<u8>`: arguments are the secret key and the input vector.
/// Outputs a KreyviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(key_bytes: [u8; 16], iv_bytes: [u8; 16]) -> KreyviumStreamByte<u8> {
// Initialization of Kreyvium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_byte_reg = [0u8; 12];
let mut b_byte_reg = [0u8; 11];
let mut c_byte_reg = [0u8; 14];
// Copy key bits into a register
a_byte_reg.copy_from_slice(&key_bytes[4..]);
// Copy iv bits into a register
b_byte_reg.copy_from_slice(&iv_bytes[5..]);
// Copy a lot of ones in the c register
c_byte_reg[0] = 252;
c_byte_reg[1..8].fill(255);
// Copy iv bits in the c register
c_byte_reg[8] = (iv_bytes[0] << 4) | 31;
for b in 9..14 {
c_byte_reg[b] = (iv_bytes[b - 9] >> 4) | (iv_bytes[b - 8] << 4);
}
// Key and iv are stored in reverse in their shift registers
let mut key = key_bytes.map(|b| b.reverse_bits());
let mut iv = iv_bytes.map(|b| b.reverse_bits());
key.reverse();
iv.reverse();
let mut ret = KreyviumStreamByte::<u8>::new_from_registers(
a_byte_reg, b_byte_reg, c_byte_reg, key, iv, None,
);
ret.init();
ret
}
}
impl KreyviumStreamByte<FheUint8> {
/// Constructor for `KreyviumStream<FheUint8>`: arguments are the encrypted secret key and input
/// vector, and the FHE server key.
/// Outputs a KreyviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(
key_bytes: [FheUint8; 16],
iv_bytes: [u8; 16],
server_key: &ServerKey,
) -> KreyviumStreamByte<FheUint8> {
set_server_key(server_key.clone());
// Initialization of Kreyvium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_byte_reg = [0u8; 12].map(FheUint8::encrypt_trivial);
let mut b_byte_reg = [0u8; 11].map(FheUint8::encrypt_trivial);
let mut c_byte_reg = [0u8; 14].map(FheUint8::encrypt_trivial);
// Copy key bits into a register
a_byte_reg.clone_from_slice(&key_bytes[4..]);
// Copy iv bits into a register
for b in 0..11 {
b_byte_reg[b] = FheUint8::encrypt_trivial(iv_bytes[b + 5]);
}
// Copy a lot of ones in the c register
c_byte_reg[0] = FheUint8::encrypt_trivial(252u8);
c_byte_reg[1..8].fill_with(|| FheUint8::encrypt_trivial(255u8));
// Copy iv bits in the c register
c_byte_reg[8] = FheUint8::encrypt_trivial((&iv_bytes[0] << 4u8) | 31u8);
for b in 9..14 {
c_byte_reg[b] =
FheUint8::encrypt_trivial((&iv_bytes[b - 9] >> 4u8) | (&iv_bytes[b - 8] << 4u8));
}
// Key and iv are stored in reverse in their shift registers
let mut key = key_bytes.map(|b| b.map(|x| (x as u8).reverse_bits() as u64));
let mut iv = iv_bytes.map(|x| FheUint8::encrypt_trivial(x.reverse_bits()));
key.reverse();
iv.reverse();
unset_server_key();
let mut ret = KreyviumStreamByte::<FheUint8>::new_from_registers(
a_byte_reg,
b_byte_reg,
c_byte_reg,
key,
iv,
Some(server_key.clone()),
);
ret.init();
ret
}
}
impl<T> KreyviumStreamByte<T>
where
T: KreyviumByteInput<T> + Send,
for<'a> &'a T: KreyviumByteInput<T>,
{
/// Internal generic constructor: arguments are already prepared registers, and an optional FHE
/// server key
fn new_from_registers(
a_register: [T; 12],
b_register: [T; 11],
c_register: [T; 14],
k_register: [T; 16],
iv_register: [T; 16],
sk: Option<ServerKey>,
) -> Self {
Self {
a_byte: StaticByteDeque::<12, T>::new(a_register),
b_byte: StaticByteDeque::<11, T>::new(b_register),
c_byte: StaticByteDeque::<14, T>::new(c_register),
k_byte: StaticByteDeque::<16, T>::new(k_register),
iv_byte: StaticByteDeque::<16, T>::new(iv_register),
fhe_key: sk,
}
}
/// The specification of Kreyvium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes 8 potential future step of Kreyvium, b*8 terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, b: usize) -> [T; 4] {
let n = b * 8 + 7;
assert!(n < 65);
let (((k, iv), (a1, a2, a3, a4, a5)), ((b1, b2, b3, b4, b5), (c1, c2, c3, c4, c5))) =
rayon::join(
|| {
rayon::join(
|| (self.k_byte.byte(127 - n), self.iv_byte.byte(127 - n)),
|| Self::get_bytes(&self.a_byte, [91 - n, 90 - n, 68 - n, 65 - n, 92 - n]),
)
},
|| {
rayon::join(
|| Self::get_bytes(&self.b_byte, [82 - n, 81 - n, 77 - n, 68 - n, 83 - n]),
|| {
Self::get_bytes(
&self.c_byte,
[109 - n, 108 - n, 86 - n, 65 - n, 110 - n],
)
},
)
},
);
let (((temp_a, temp_b), (temp_c, a_and)), (b_and, c_and)) = rayon::join(
|| {
rayon::join(
|| rayon::join(|| a4 ^ a5, || b4 ^ b5),
|| rayon::join(|| c4 ^ c5 ^ k, || a1 & a2 ^ iv),
)
},
|| rayon::join(|| b1 & b2, || c1 & c2),
);
let (temp_a_2, temp_b_2, temp_c_2) = (temp_a.clone(), temp_b.clone(), temp_c.clone());
let ((o, a), (b, c)) = rayon::join(
|| {
rayon::join(
|| (temp_a_2 ^ temp_b_2) ^ temp_c_2,
|| temp_c ^ ((c_and) ^ a3),
)
},
|| rayon::join(|| temp_a ^ (a_and ^ b3), || temp_b ^ (b_and ^ c3)),
);
[o, a, b, c]
}
/// This calls `get_output_and_values` in parallel 8 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[T; 4]> {
(0..8)
.into_par_iter()
.map(|i| self.get_output_and_values(i))
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits (in 8 bytes) all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<T> {
match &self.fhe_key {
Some(sk) => {
rayon::broadcast(|_| set_server_key(sk.clone()));
}
None => (),
}
let values = self.get_64_output_and_values();
match &self.fhe_key {
Some(_) => {
rayon::broadcast(|_| unset_server_key());
}
None => (),
}
let mut bytes = Vec::<T>::with_capacity(8);
for [o, a, b, c] in values {
self.a_byte.push(a);
self.b_byte.push(b);
self.c_byte.push(c);
bytes.push(o);
}
self.k_byte.n_shifts(8);
self.iv_byte.n_shifts(8);
bytes
}
/// Reconstructs a bunch of 5 bytes in a parallel fashion.
fn get_bytes<const N: usize>(
reg: &StaticByteDeque<N, T>,
offsets: [usize; 5],
) -> (T, T, T, T, T) {
let mut ret = offsets
.par_iter()
.rev()
.map(|&i| reg.byte(i))
.collect::<Vec<_>>();
(
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
)
}
}
impl KreyviumStreamByte<FheUint8> {
pub fn get_server_key(&self) -> &ServerKey {
self.fhe_key.as_ref().unwrap()
}
}

View File

@@ -0,0 +1,205 @@
use crate::static_deque::StaticDeque;
use tfhe::shortint::prelude::*;
use rayon::prelude::*;
/// KreyviumStreamShortint: a struct implementing the Kreyvium stream cipher, using a generic
/// Ciphertext for the internal representation of bits (intended to represent a single bit). To be
/// able to compute FHE operations, it also owns a ServerKey.
pub struct KreyviumStreamShortint {
a: StaticDeque<93, Ciphertext>,
b: StaticDeque<84, Ciphertext>,
c: StaticDeque<111, Ciphertext>,
k: StaticDeque<128, Ciphertext>,
iv: StaticDeque<128, Ciphertext>,
internal_server_key: ServerKey,
transciphering_casting_key: KeySwitchingKey,
hl_server_key: tfhe::ServerKey,
}
impl KreyviumStreamShortint {
/// Constructor for KreyviumStreamShortint: arguments are the secret key and the input vector,
/// and a ServerKey reference. Outputs a KreyviumStream object already initialized (1152
/// steps have been run before returning)
pub fn new(
mut key: [Ciphertext; 128],
mut iv: [u64; 128],
sk: ServerKey,
ksk: KeySwitchingKey,
hl_sk: tfhe::ServerKey,
) -> Self {
// Initialization of Kreyvium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register: [Ciphertext; 93] = [0; 93].map(|x| sk.create_trivial(x));
let mut b_register: [Ciphertext; 84] = [0; 84].map(|x| sk.create_trivial(x));
let mut c_register: [Ciphertext; 111] = [0; 111].map(|x| sk.create_trivial(x));
for i in 0..93 {
a_register[i] = key[128 - 93 + i].clone();
}
for i in 0..84 {
b_register[i] = sk.create_trivial(iv[128 - 84 + i]);
}
for i in 0..44 {
c_register[111 - 44 + i] = sk.create_trivial(iv[i]);
}
for i in 0..66 {
c_register[i + 1] = sk.create_trivial(1);
}
key.reverse();
iv.reverse();
let iv = iv.map(|x| sk.create_trivial(x));
let mut ret = Self {
a: StaticDeque::<93, Ciphertext>::new(a_register),
b: StaticDeque::<84, Ciphertext>::new(b_register),
c: StaticDeque::<111, Ciphertext>::new(c_register),
k: StaticDeque::<128, Ciphertext>::new(key),
iv: StaticDeque::<128, Ciphertext>::new(iv),
internal_server_key: sk,
transciphering_casting_key: ksk,
hl_server_key: hl_sk,
};
ret.init();
ret
}
/// The specification of Kreyvium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes one turn of the stream, updating registers and outputting the new bit.
pub fn next_ct(&mut self) -> Ciphertext {
let [o, a, b, c] = self.get_output_and_values(0);
self.a.push(a);
self.b.push(b);
self.c.push(c);
o
}
/// Computes a potential future step of Kreyvium, n terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, n: usize) -> [Ciphertext; 4] {
let (k, iv) = (&self.k[127 - n], &self.iv[127 - n]);
let (a1, a2, a3, a4, a5) = (
&self.a[65 - n],
&self.a[92 - n],
&self.a[91 - n],
&self.a[90 - n],
&self.a[68 - n],
);
let (b1, b2, b3, b4, b5) = (
&self.b[68 - n],
&self.b[83 - n],
&self.b[82 - n],
&self.b[81 - n],
&self.b[77 - n],
);
let (c1, c2, c3, c4, c5) = (
&self.c[65 - n],
&self.c[110 - n],
&self.c[109 - n],
&self.c[108 - n],
&self.c[86 - n],
);
let temp_a = self.internal_server_key.unchecked_add(a1, a2);
let temp_b = self.internal_server_key.unchecked_add(b1, b2);
let mut temp_c = self.internal_server_key.unchecked_add(c1, c2);
self.internal_server_key
.unchecked_add_assign(&mut temp_c, k);
let ((new_a, new_b), (new_c, o)) = rayon::join(
|| {
rayon::join(
|| {
let mut new_a = self.internal_server_key.unchecked_bitand(c3, c4);
self.internal_server_key
.unchecked_add_assign(&mut new_a, a5);
self.internal_server_key.add_assign(&mut new_a, &temp_c);
new_a
},
|| {
let mut new_b = self.internal_server_key.unchecked_bitand(a3, a4);
self.internal_server_key
.unchecked_add_assign(&mut new_b, b5);
self.internal_server_key
.unchecked_add_assign(&mut new_b, &temp_a);
self.internal_server_key.add_assign(&mut new_b, iv);
new_b
},
)
},
|| {
rayon::join(
|| {
let mut new_c = self.internal_server_key.unchecked_bitand(b3, b4);
self.internal_server_key
.unchecked_add_assign(&mut new_c, c5);
self.internal_server_key
.unchecked_add_assign(&mut new_c, &temp_b);
self.internal_server_key.message_extract_assign(&mut new_c);
new_c
},
|| {
self.internal_server_key.bitxor(
&self.internal_server_key.unchecked_add(&temp_a, &temp_b),
&temp_c,
)
},
)
},
);
[o, new_a, new_b, new_c]
}
/// This calls `get_output_and_values` in parallel 64 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[Ciphertext; 4]> {
(0..64)
.into_par_iter()
.map(|x| self.get_output_and_values(x))
.rev()
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<Ciphertext> {
let mut values = self.get_64_output_and_values();
let mut ret = Vec::<Ciphertext>::with_capacity(64);
while let Some([o, a, b, c]) = values.pop() {
ret.push(o);
self.a.push(a);
self.b.push(b);
self.c.push(c);
}
self.k.n_shifts(64);
self.iv.n_shifts(64);
ret
}
pub fn get_internal_server_key(&self) -> &ServerKey {
&self.internal_server_key
}
pub fn get_casting_key(&self) -> &KeySwitchingKey {
&self.transciphering_casting_key
}
pub fn get_hl_server_key(&self) -> &tfhe::ServerKey {
&self.hl_server_key
}
}

View File

@@ -0,0 +1,12 @@
#[allow(clippy::module_inception)]
mod kreyvium;
pub use kreyvium::KreyviumStream;
mod kreyvium_byte;
pub use kreyvium_byte::KreyviumStreamByte;
mod kreyvium_shortint;
pub use kreyvium_shortint::KreyviumStreamShortint;
#[cfg(test)]
mod test;

View File

@@ -0,0 +1,374 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheBool, FheUint64, FheUint8};
use crate::{KreyviumStream, KreyviumStreamByte, KreyviumStreamShortint, TransCiphering};
// Values for these tests come from the github repo renaud1239/Kreyvium,
// commit fd6828f68711276c25f55e605935028f5e843f43
fn get_hexadecimal_string_from_lsb_first_stream(a: Vec<bool>) -> String {
assert!(a.len() % 8 == 0);
let mut hexadecimal: String = "".to_string();
for test in a.chunks(8) {
// Encoding is bytes in LSB order
match test[4..8] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => (),
};
match test[0..4] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => (),
};
}
hexadecimal
}
fn get_hexagonal_string_from_bytes(a: Vec<u8>) -> String {
assert!(a.len() % 8 == 0);
let mut hexadecimal: String = "".to_string();
for test in a {
hexadecimal.push_str(&format!("{:02X?}", test));
}
hexadecimal
}
fn get_hexagonal_string_from_u64(a: Vec<u64>) -> String {
let mut hexadecimal: String = "".to_string();
for test in a {
hexadecimal.push_str(&format!("{:016X?}", test));
}
hexadecimal
}
#[test]
fn kreyvium_test_1() {
let key = [false; 128];
let iv = [false; 128];
let output = "26DCF1F4BC0F1922";
let mut kreyvium = KreyviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(64);
while vec.len() < 64 {
vec.push(kreyvium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_2() {
let mut key = [false; 128];
let iv = [false; 128];
key[0] = true;
let output = "4FD421D4DA3D2C8A";
let mut kreyvium = KreyviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(64);
while vec.len() < 64 {
vec.push(kreyvium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_3() {
let key = [false; 128];
let mut iv = [false; 128];
iv[0] = true;
let output = "C9217BA0D762ACA1";
let mut kreyvium = KreyviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(64);
while vec.len() < 64 {
vec.push(kreyvium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_4() {
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [false; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [false; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let output = "D1F0303482061111";
let mut kreyvium = KreyviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(64);
while vec.len() < 64 {
vec.push(kreyvium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(hexadecimal, output);
}
#[test]
fn kreyvium_test_fhe_long() {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [false; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [false; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let output = "D1F0303482061111";
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let mut kreyvium = KreyviumStream::<FheBool>::new(cipher_key, iv, &server_key);
let mut vec = Vec::<bool>::with_capacity(64);
while vec.len() < 64 {
let cipher_outputs = kreyvium.next_64();
for c in cipher_outputs {
vec.push(c.decrypt(&client_key))
}
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output, hexadecimal);
}
use tfhe::shortint::prelude::*;
#[test]
fn kreyvium_test_shortint_long() {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0; 128];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0; 128];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let output = "D1F0303482061111".to_string();
let cipher_key = key.map(|x| client_key.encrypt(x));
let ciphered_message = FheUint64::try_encrypt(0u64, &hl_client_key).unwrap();
let mut kreyvium = KreyviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
let trans_ciphered_message = kreyvium.trans_encrypt_64(ciphered_message);
let ciphered_message = trans_ciphered_message.decrypt(&hl_client_key);
let hexadecimal = get_hexagonal_string_from_u64(vec![ciphered_message]);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_clear_byte() {
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key_bytes = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key_bytes[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv_bytes = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv_bytes[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let output = "D1F0303482061111".to_string();
let mut kreyvium = KreyviumStreamByte::<u8>::new(key_bytes, iv_bytes);
let mut vec = Vec::<u8>::with_capacity(8);
while vec.len() < 8 {
let outputs = kreyvium.next_64();
for c in outputs {
vec.push(c)
}
}
let hexadecimal = get_hexagonal_string_from_bytes(vec);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_byte_long() {
let config = ConfigBuilder::default()
.enable_function_evaluation()
.build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key_bytes = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key_bytes[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv_bytes = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv_bytes[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let cipher_key = key_bytes.map(|x| FheUint8::encrypt(x, &client_key));
let output = "D1F0303482061111".to_string();
let mut kreyvium = KreyviumStreamByte::<FheUint8>::new(cipher_key, iv_bytes, &server_key);
let mut vec = Vec::<u8>::with_capacity(8);
while vec.len() < 8 {
let cipher_outputs = kreyvium.next_64();
for c in cipher_outputs {
vec.push(c.decrypt(&client_key))
}
}
let hexadecimal = get_hexagonal_string_from_bytes(vec);
assert_eq!(output, hexadecimal);
}
#[test]
fn kreyvium_test_fhe_byte_transciphering_long() {
let config = ConfigBuilder::default()
.enable_function_evaluation()
.build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB000000000000".to_string();
let mut key = [0u8; 16];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC000000000000".to_string();
let mut iv = [0u8; 16];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let output = "D1F0303482061111".to_string();
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let ciphered_message = FheUint64::try_encrypt(0u64, &client_key).unwrap();
let mut kreyvium = KreyviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
let trans_ciphered_message = kreyvium.trans_encrypt_64(ciphered_message);
let ciphered_message = trans_ciphered_message.decrypt(&client_key);
let hexadecimal = get_hexagonal_string_from_u64(vec![ciphered_message]);
assert_eq!(output, hexadecimal);
}

10
apps/trivium/src/lib.rs Normal file
View File

@@ -0,0 +1,10 @@
mod static_deque;
mod kreyvium;
pub use kreyvium::{KreyviumStream, KreyviumStreamByte, KreyviumStreamShortint};
mod trivium;
pub use trivium::{TriviumStream, TriviumStreamByte, TriviumStreamShortint};
mod trans_ciphering;
pub use trans_ciphering::TransCiphering;

View File

@@ -0,0 +1,5 @@
#[allow(clippy::module_inception)]
mod static_deque;
pub use static_deque::StaticDeque;
mod static_byte_deque;
pub use static_byte_deque::{StaticByteDeque, StaticByteDequeInput};

View File

@@ -0,0 +1,141 @@
//! This module implements the StaticByteDeque struct: a deque of bytes. The idea
//! is that this is a wrapper around StaticDeque, but StaticByteDeque has an additional
//! functionality: it can construct the "intermediate" bytes, made of parts of other bytes.
//! This is pretending to store bits, and allows accessing bits in chunks of 8 consecutive.
use crate::static_deque::StaticDeque;
use tfhe::FheUint8;
/// Internal trait specifying which operations are needed by StaticByteDeque
pub trait StaticByteDequeInput<OpOutput>:
Clone
+ std::ops::Shr<u8, Output = OpOutput>
+ std::ops::Shl<u8, Output = OpOutput>
+ std::ops::BitOr<Output = OpOutput>
{
}
impl StaticByteDequeInput<u8> for u8 {}
impl StaticByteDequeInput<u8> for &u8 {}
impl StaticByteDequeInput<FheUint8> for FheUint8 {}
impl StaticByteDequeInput<FheUint8> for &FheUint8 {}
/// Here T must represent a type covering a byte, like u8 or FheUint8.
#[derive(Clone)]
pub struct StaticByteDeque<const N: usize, T> {
deque: StaticDeque<N, T>,
}
impl<const N: usize, T> StaticByteDeque<N, T>
where
T: StaticByteDequeInput<T>,
for<'a> &'a T: StaticByteDequeInput<T>,
{
/// Constructor always uses a fully initialized array, the first element of
/// which is oldest, the last is newest
pub fn new(_arr: [T; N]) -> Self {
Self {
deque: StaticDeque::<N, T>::new(_arr),
}
}
/// Elements are pushed via a byte element (covering 8 underlying bits)
pub fn push(&mut self, val: T) {
self.deque.push(val)
}
/// computes n shift in a row
pub fn n_shifts(&mut self, n: usize) {
self.deque.n_shifts(n);
}
/// Getter for the internal memory
#[allow(dead_code)]
fn get_arr(&self) -> &[T; N] {
self.deque.get_arr()
}
/// This returns a byte full of zeros, except maybe a one
/// at the specified location, if it is present in the deque
#[allow(dead_code)]
fn bit(&self, i: usize) -> T
where
for<'a> &'a T: std::ops::BitAnd<u8, Output = T>,
{
let byte: &T = &self.deque[i / 8];
let bit_selector: u8 = 1u8 << (i % 8);
byte & bit_selector
}
/// This function reconstructs an intermediate byte if necessary
pub fn byte(&self, i: usize) -> T {
let byte: &T = &self.deque[i / 8];
let bit_idx: u8 = (i % 8) as u8;
if bit_idx == 0 {
return byte.clone();
}
let byte_next: &T = &self.deque[i / 8 + 1];
(byte << bit_idx) | (byte_next >> (8 - bit_idx))
}
}
#[cfg(test)]
mod tests {
use crate::static_deque::StaticByteDeque;
#[test]
fn byte_deque_test() {
let mut deque = StaticByteDeque::<3, u8>::new([2, 64, 128]);
deque.push(4);
// Youngest: 4
assert!(deque.bit(0) == 0);
assert!(deque.bit(1) == 0);
assert!(deque.bit(2) > 0);
assert!(deque.bit(3) == 0);
assert!(deque.bit(4) == 0);
assert!(deque.bit(5) == 0);
assert!(deque.bit(6) == 0);
assert!(deque.bit(7) == 0);
// second youngest: 128
assert!(deque.bit(8) == 0);
assert!(deque.bit(8 + 1) == 0);
assert!(deque.bit(8 + 2) == 0);
assert!(deque.bit(8 + 3) == 0);
assert!(deque.bit(8 + 4) == 0);
assert!(deque.bit(8 + 5) == 0);
assert!(deque.bit(8 + 6) == 0);
assert!(deque.bit(8 + 7) > 0);
// oldest: 64
assert!(deque.bit(16) == 0);
assert!(deque.bit(16 + 1) == 0);
assert!(deque.bit(16 + 2) == 0);
assert!(deque.bit(16 + 3) == 0);
assert!(deque.bit(16 + 4) == 0);
assert!(deque.bit(16 + 5) == 0);
assert!(deque.bit(16 + 6) > 0);
assert!(deque.bit(16 + 7) == 0);
assert_eq!(deque.byte(0), 4u8);
assert_eq!(deque.byte(1), 9u8);
assert_eq!(deque.byte(2), 18u8);
assert_eq!(deque.byte(3), 36u8);
assert_eq!(deque.byte(4), 72u8);
assert_eq!(deque.byte(5), 144u8);
assert_eq!(deque.byte(6), 32u8);
assert_eq!(deque.byte(7), 64u8);
assert_eq!(deque.byte(8), 128u8);
assert_eq!(deque.byte(9), 0u8);
assert_eq!(deque.byte(10), 1u8);
assert_eq!(deque.byte(11), 2u8);
assert_eq!(deque.byte(12), 4u8);
assert_eq!(deque.byte(13), 8u8);
assert_eq!(deque.byte(14), 16u8);
assert_eq!(deque.byte(15), 32u8);
assert_eq!(deque.byte(16), 64u8);
}
}

View File

@@ -0,0 +1,135 @@
//! This module implements the StaticDeque struct: a deque utility whose size
//! is known at compile time. Construction, push, and indexing are publicly
//! available.
use core::ops::{Index, IndexMut};
/// StaticDeque: a struct implementing a deque whose size is known at compile time.
/// It has 2 members: the static array containing the data (never empty), and a cursor
/// equal to the index of the oldest element (and the next one to be overwritten).
#[derive(Clone)]
pub struct StaticDeque<const N: usize, T> {
arr: [T; N],
cursor: usize,
}
impl<const N: usize, T> StaticDeque<N, T> {
/// Constructor always uses a fully initialized array, the first element of
/// which is oldest, the last is newest
pub fn new(_arr: [T; N]) -> Self {
Self {
arr: _arr,
cursor: 0,
}
}
/// Push a new element to the deque, overwriting the oldest at the same time.
pub fn push(&mut self, val: T) {
self.arr[self.cursor] = val;
self.shift();
}
/// Shift: equivalent to pushing the oldest element
pub fn shift(&mut self) {
self.n_shifts(1);
}
/// computes n shift in a row
pub fn n_shifts(&mut self, n: usize) {
self.cursor += n;
self.cursor %= N;
}
/// Getter for the internal memory
#[allow(dead_code)]
pub fn get_arr(&self) -> &[T; N] {
&self.arr
}
}
/// Index trait for the StaticDeque: 0 is the youngest element, N-1 is the oldest,
/// and above N will panic.
impl<const N: usize, T> Index<usize> for StaticDeque<N, T> {
type Output = T;
/// 0 is youngest
fn index(&self, i: usize) -> &T {
if i >= N {
panic!("Index {:?} too high for size {:?}", i, N);
}
&self.arr[(N + self.cursor - i - 1) % N]
}
}
/// IndexMut trait for the StaticDeque: 0 is the youngest element, N-1 is the oldest,
/// and above N will panic.
impl<const N: usize, T> IndexMut<usize> for StaticDeque<N, T> {
/// 0 is youngest
fn index_mut(&mut self, i: usize) -> &mut T {
if i >= N {
panic!("Index {:?} too high for size {:?}", i, N);
}
&mut self.arr[(N + self.cursor - i - 1) % N]
}
}
#[cfg(test)]
mod tests {
use crate::static_deque::StaticDeque;
#[test]
fn test_static_deque() {
let a = [1, 2, 3, 4, 5, 6];
let mut static_deque = StaticDeque::new(a);
for i in 7..11 {
static_deque.push(i);
}
assert_eq!(*static_deque.get_arr(), [7, 8, 9, 10, 5, 6]);
for i in 11..15 {
static_deque.push(i);
}
assert_eq!(*static_deque.get_arr(), [13, 14, 9, 10, 11, 12]);
assert_eq!(static_deque[0], 14);
assert_eq!(static_deque[1], 13);
assert_eq!(static_deque[2], 12);
assert_eq!(static_deque[3], 11);
assert_eq!(static_deque[4], 10);
assert_eq!(static_deque[5], 9);
}
#[test]
fn test_static_deque_indexmut() {
let a = [1, 2, 3, 4, 5, 6];
let mut static_deque = StaticDeque::new(a);
for i in 7..11 {
static_deque.push(i);
}
assert_eq!(*static_deque.get_arr(), [7, 8, 9, 10, 5, 6]);
for i in 11..15 {
static_deque.push(i);
}
assert_eq!(*static_deque.get_arr(), [13, 14, 9, 10, 11, 12]);
static_deque[1] = 100;
assert_eq!(static_deque[0], 14);
assert_eq!(static_deque[1], 100);
assert_eq!(static_deque[2], 12);
assert_eq!(static_deque[3], 11);
assert_eq!(static_deque[4], 10);
assert_eq!(static_deque[5], 9);
}
#[test]
#[should_panic]
fn test_static_deque_index_fail() {
let a = [1, 2, 3, 4, 5, 6];
let static_deque = StaticDeque::new(a);
let _ = static_deque[6];
}
}

View File

@@ -0,0 +1,119 @@
//! This module will contain extensions of some TriviumStream of KreyviumStream objects,
//! when trans ciphering is available to them.
use crate::{KreyviumStreamByte, KreyviumStreamShortint, TriviumStreamByte, TriviumStreamShortint};
use tfhe::shortint::Ciphertext;
use tfhe::prelude::*;
use tfhe::{set_server_key, unset_server_key, FheUint64, FheUint8, ServerKey};
use rayon::prelude::*;
/// Triat specifying the interface for trans ciphering a FheUint64 object. Since it is meant
/// to be used with stream ciphers, encryption and decryption are by default the same.
pub trait TransCiphering {
fn trans_encrypt_64(&mut self, cipher: FheUint64) -> FheUint64;
fn trans_decrypt_64(&mut self, cipher: FheUint64) -> FheUint64 {
self.trans_encrypt_64(cipher)
}
}
fn transcipher_from_fheu8_stream(
stream: Vec<FheUint8>,
cipher: FheUint64,
fhe_server_key: &ServerKey,
) -> FheUint64 {
assert_eq!(stream.len(), 8);
set_server_key(fhe_server_key.clone());
rayon::broadcast(|_| set_server_key(fhe_server_key.clone()));
let ret: FheUint64 = stream
.into_par_iter()
.enumerate()
.map(|(i, x)| &cipher ^ &(FheUint64::cast_from(x) << (8 * (7 - i) as u8)))
.reduce_with(|a, b| a | b)
.unwrap();
unset_server_key();
rayon::broadcast(|_| unset_server_key());
ret
}
fn transcipher_from_1_1_stream(
stream: Vec<Ciphertext>,
cipher: FheUint64,
hl_server_key: &ServerKey,
internal_server_key: &tfhe::shortint::ServerKey,
casting_key: &tfhe::shortint::KeySwitchingKey,
) -> FheUint64 {
assert_eq!(stream.len(), 64);
let pairs = (0..32)
.into_par_iter()
.map(|i| {
let byte_idx = 7 - i / 4;
let pair_idx = i % 4;
let b0 = &stream[8 * byte_idx + 2 * pair_idx];
let b1 = &stream[8 * byte_idx + 2 * pair_idx + 1];
casting_key.cast(
&internal_server_key
.unchecked_add(b0, &internal_server_key.unchecked_scalar_mul(b1, 2)),
)
})
.collect::<Vec<_>>();
set_server_key(hl_server_key.clone());
let ret = &cipher ^ &FheUint64::try_from(pairs).unwrap();
unset_server_key();
ret
}
impl TransCiphering for TriviumStreamByte<FheUint8> {
/// `TriviumStreamByte<FheUint8>`: since a full step outputs 8 bytes, these bytes
/// are each shifted by a number in [0, 8), and XORed with the input cipher
fn trans_encrypt_64(&mut self, cipher: FheUint64) -> FheUint64 {
transcipher_from_fheu8_stream(self.next_64(), cipher, self.get_server_key())
}
}
impl TransCiphering for KreyviumStreamByte<FheUint8> {
/// `KreyviumStreamByte<FheUint8>`: since a full step outputs 8 bytes, these bytes
/// are each shifted by a number in [0, 8), and XORed with the input cipher
fn trans_encrypt_64(&mut self, cipher: FheUint64) -> FheUint64 {
transcipher_from_fheu8_stream(self.next_64(), cipher, self.get_server_key())
}
}
impl TransCiphering for TriviumStreamShortint {
/// TriviumStreamShortint: since a full step outputs 64 shortints, these bits
/// are paired 2 by 2 in the HL parameter space and packed in a full word,
/// and XORed with the input cipher
fn trans_encrypt_64(&mut self, cipher: FheUint64) -> FheUint64 {
transcipher_from_1_1_stream(
self.next_64(),
cipher,
self.get_hl_server_key(),
self.get_internal_server_key(),
self.get_casting_key(),
)
}
}
impl TransCiphering for KreyviumStreamShortint {
/// KreyviumStreamShortint: since a full step outputs 64 shortints, these bits
/// are paired 2 by 2 in the HL parameter space and packed in a full word,
/// and XORed with the input cipher
fn trans_encrypt_64(&mut self, cipher: FheUint64) -> FheUint64 {
transcipher_from_1_1_stream(
self.next_64(),
cipher,
self.get_hl_server_key(),
self.get_internal_server_key(),
self.get_casting_key(),
)
}
}

View File

@@ -0,0 +1,11 @@
mod trivium_bool;
pub use trivium_bool::TriviumStream;
mod trivium_byte;
pub use trivium_byte::TriviumStreamByte;
mod trivium_shortint;
pub use trivium_shortint::TriviumStreamShortint;
#[cfg(test)]
mod test;

View File

@@ -0,0 +1,406 @@
use tfhe::prelude::*;
use tfhe::{generate_keys, ConfigBuilder, FheBool, FheUint64, FheUint8};
use crate::{TransCiphering, TriviumStream, TriviumStreamByte, TriviumStreamShortint};
// Values for these tests come from the github repo cantora/avr-crypto-lib, commit 2a5b018,
// file testvectors/trivium-80.80.test-vectors
fn get_hexadecimal_string_from_lsb_first_stream(a: Vec<bool>) -> String {
assert!(a.len() % 8 == 0);
let mut hexadecimal: String = "".to_string();
for test in a.chunks(8) {
// Encoding is bytes in LSB order
match test[4..8] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => (),
};
match test[0..4] {
[false, false, false, false] => hexadecimal.push('0'),
[true, false, false, false] => hexadecimal.push('1'),
[false, true, false, false] => hexadecimal.push('2'),
[true, true, false, false] => hexadecimal.push('3'),
[false, false, true, false] => hexadecimal.push('4'),
[true, false, true, false] => hexadecimal.push('5'),
[false, true, true, false] => hexadecimal.push('6'),
[true, true, true, false] => hexadecimal.push('7'),
[false, false, false, true] => hexadecimal.push('8'),
[true, false, false, true] => hexadecimal.push('9'),
[false, true, false, true] => hexadecimal.push('A'),
[true, true, false, true] => hexadecimal.push('B'),
[false, false, true, true] => hexadecimal.push('C'),
[true, false, true, true] => hexadecimal.push('D'),
[false, true, true, true] => hexadecimal.push('E'),
[true, true, true, true] => hexadecimal.push('F'),
_ => (),
};
}
hexadecimal
}
fn get_hexagonal_string_from_bytes(a: Vec<u8>) -> String {
assert!(a.len() % 8 == 0);
let mut hexadecimal: String = "".to_string();
for test in a {
hexadecimal.push_str(&format!("{:02X?}", test));
}
hexadecimal
}
fn get_hexagonal_string_from_u64(a: Vec<u64>) -> String {
let mut hexadecimal: String = "".to_string();
for test in a {
hexadecimal.push_str(&format!("{:016X?}", test));
}
hexadecimal
}
#[test]
fn trivium_test_1() {
let key = [false; 80];
let iv = [false; 80];
let output_0_63 = "FBE0BF265859051B517A2E4E239FC97F563203161907CF2DE7A8790FA1B2E9CDF75292030268B7382B4C1A759AA2599A285549986E74805903801A4CB5A5D4F2".to_string();
let output_192_255 = "0F1BE95091B8EA857B062AD52BADF47784AC6D9B2E3F85A9D79995043302F0FDF8B76E5BC8B7B4F0AA46CD20DDA04FDD197BC5E1635496828F2DBFB23F6BD5D0".to_string();
let output_256_319 = "80F9075437BAC73F696D0ABE3972F5FCE2192E5FCC13C0CB77D0ABA09126838D31A2D38A2087C46304C8A63B54109F679B0B1BC71E72A58D6DD3E0A3FF890D4A".to_string();
let output_448_511 = "68450EB0910A98EF1853E0FC1BED8AB6BB08DF5F167D34008C2A85284D4B886DD56883EE92BF18E69121670B4C81A5689C9B0538373D22EB923A28A2DB44C0EB".to_string();
let mut trivium = TriviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(512 * 8);
while vec.len() < 512 * 8 {
vec.push(trivium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
assert_eq!(output_192_255, hexadecimal[192 * 2..256 * 2]);
assert_eq!(output_256_319, hexadecimal[256 * 2..320 * 2]);
assert_eq!(output_448_511, hexadecimal[448 * 2..512 * 2]);
}
#[test]
fn trivium_test_2() {
let mut key = [false; 80];
let iv = [false; 80];
key[7] = true;
let output_0_63 = "38EB86FF730D7A9CAF8DF13A4420540DBB7B651464C87501552041C249F29A64D2FBF515610921EBE06C8F92CECF7F8098FF20CCCC6A62B97BE8EF7454FC80F9".to_string();
let output_192_255 = "EAF2625D411F61E41F6BAEEDDD5FE202600BD472F6C9CD1E9134A745D900EF6C023E4486538F09930CFD37157C0EB57C3EF6C954C42E707D52B743AD83CFF297".to_string();
let output_256_319 = "9A203CF7B2F3F09C43D188AA13A5A2021EE998C42F777E9B67C3FA221A0AA1B041AA9E86BC2F5C52AFF11F7D9EE480CB1187B20EB46D582743A52D7CD080A24A".to_string();
let output_448_511 = "EBF14772061C210843C18CEA2D2A275AE02FCB18E5D7942455FF77524E8A4CA51E369A847D1AEEFB9002FCD02342983CEAFA9D487CC2032B10192CD416310FA4".to_string();
let mut trivium = TriviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(512 * 8);
while vec.len() < 512 * 8 {
vec.push(trivium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
assert_eq!(output_192_255, hexadecimal[192 * 2..256 * 2]);
assert_eq!(output_256_319, hexadecimal[256 * 2..320 * 2]);
assert_eq!(output_448_511, hexadecimal[448 * 2..512 * 2]);
}
#[test]
fn trivium_test_3() {
let key = [false; 80];
let mut iv = [false; 80];
iv[7] = true;
let output_0_63 = "F8901736640549E3BA7D42EA2D07B9F49233C18D773008BD755585B1A8CBAB86C1E9A9B91F1AD33483FD6EE3696D659C9374260456A36AAE11F033A519CBD5D7".to_string();
let output_192_255 = "87423582AF64475C3A9C092E32A53C5FE07D35B4C9CA288A89A43DEF3913EA9237CA43342F3F8E83AD3A5C38D463516F94E3724455656A36279E3E924D442F06".to_string();
let output_256_319 = "D94389A90E6F3BF2BB4C8B057339AAD8AA2FEA238C29FCAC0D1FF1CB2535A07058BA995DD44CFC54CCEC54A5405B944C532D74E50EA370CDF1BA1CBAE93FC0B5".to_string();
let output_448_511 = "4844151714E56A3A2BBFBA426A1D60F9A4F265210A91EC29259AE2035234091C49FFB1893FA102D425C57C39EB4916F6D148DC83EBF7DE51EEB9ABFE045FB282".to_string();
let mut trivium = TriviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(512 * 8);
while vec.len() < 512 * 8 {
vec.push(trivium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
assert_eq!(output_192_255, hexadecimal[192 * 2..256 * 2]);
assert_eq!(output_256_319, hexadecimal[256 * 2..320 * 2]);
assert_eq!(output_448_511, hexadecimal[448 * 2..512 * 2]);
}
#[test]
fn trivium_test_4() {
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [false; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [false; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let output_65472_65535 = "C04C24A6938C8AF8A491D5E481271E0E601338F01067A86A795CA493AA4FF265619B8D448B706B7C88EE8395FC79E5B51AB40245BBF7773AE67DF86FCFB71F30".to_string();
let output_65536_65599 = "011A0D7EC32FA102C66C164CFCB189AED9F6982E8C7370A6A37414781192CEB155C534C1C8C9E53FDEADF2D3D0577DAD3A8EB2F6E5265F1E831C86844670BC69".to_string();
let output_131008_131071 = "48107374A9CE3AAF78221AE77789247CF6896A249ED75DCE0CF2D30EB9D889A0C61C9F480E5C07381DED9FAB2AD54333E82C89BA92E6E47FD828F1A66A8656E0".to_string();
let mut trivium = TriviumStream::<bool>::new(key, iv);
let mut vec = Vec::<bool>::with_capacity(131072 * 8);
while vec.len() < 131072 * 8 {
vec.push(trivium.next_bool());
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
assert_eq!(output_65472_65535, hexadecimal[65472 * 2..65536 * 2]);
assert_eq!(output_65536_65599, hexadecimal[65536 * 2..65600 * 2]);
assert_eq!(output_131008_131071, hexadecimal[131008 * 2..131072 * 2]);
}
#[test]
fn trivium_test_clear_byte() {
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let output_65472_65535 = "C04C24A6938C8AF8A491D5E481271E0E601338F01067A86A795CA493AA4FF265619B8D448B706B7C88EE8395FC79E5B51AB40245BBF7773AE67DF86FCFB71F30".to_string();
let output_65536_65599 = "011A0D7EC32FA102C66C164CFCB189AED9F6982E8C7370A6A37414781192CEB155C534C1C8C9E53FDEADF2D3D0577DAD3A8EB2F6E5265F1E831C86844670BC69".to_string();
let output_131008_131071 = "48107374A9CE3AAF78221AE77789247CF6896A249ED75DCE0CF2D30EB9D889A0C61C9F480E5C07381DED9FAB2AD54333E82C89BA92E6E47FD828F1A66A8656E0".to_string();
let mut trivium = TriviumStreamByte::<u8>::new(key, iv);
let mut vec = Vec::<u8>::with_capacity(131072);
while vec.len() < 131072 {
let outputs = trivium.next_64();
for c in outputs {
vec.push(c)
}
}
let hexadecimal = get_hexagonal_string_from_bytes(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
assert_eq!(output_65472_65535, hexadecimal[65472 * 2..65536 * 2]);
assert_eq!(output_65536_65599, hexadecimal[65536 * 2..65600 * 2]);
assert_eq!(output_131008_131071, hexadecimal[131008 * 2..131072 * 2]);
}
#[test]
fn trivium_test_fhe_long() {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [false; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [false; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val: u8 = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2 == 1;
val >>= 1;
}
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| FheBool::encrypt(x, &client_key));
let mut trivium = TriviumStream::<FheBool>::new(cipher_key, iv, &server_key);
let mut vec = Vec::<bool>::with_capacity(64 * 8);
while vec.len() < 64 * 8 {
let cipher_outputs = trivium.next_64();
for c in cipher_outputs {
vec.push(c.decrypt(&client_key))
}
}
let hexadecimal = get_hexadecimal_string_from_lsb_first_stream(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
}
#[test]
fn trivium_test_fhe_byte_long() {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let mut trivium = TriviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
let mut vec = Vec::<u8>::with_capacity(64);
while vec.len() < 64 {
let cipher_outputs = trivium.next_64();
for c in cipher_outputs {
vec.push(c.decrypt(&client_key))
}
}
let hexadecimal = get_hexagonal_string_from_bytes(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
}
#[test]
fn trivium_test_fhe_byte_transciphering_long() {
let config = ConfigBuilder::default().build();
let (client_key, server_key) = generate_keys(config);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0u8; 10];
for i in (0..key_string.len()).step_by(2) {
key[i >> 1] = u8::from_str_radix(&key_string[i..i + 2], 16).unwrap();
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0u8; 10];
for i in (0..iv_string.len()).step_by(2) {
iv[i >> 1] = u8::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| FheUint8::encrypt(x, &client_key));
let mut ciphered_message = vec![FheUint64::try_encrypt(0u64, &client_key).unwrap(); 9];
let mut trivium = TriviumStreamByte::<FheUint8>::new(cipher_key, iv, &server_key);
let mut vec = Vec::<u64>::with_capacity(8);
while vec.len() < 8 {
let trans_ciphered_message = trivium.trans_encrypt_64(ciphered_message.pop().unwrap());
vec.push(trans_ciphered_message.decrypt(&client_key));
}
let hexadecimal = get_hexagonal_string_from_u64(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
}
use tfhe::shortint::prelude::*;
#[test]
fn trivium_test_shortint_long() {
let config = ConfigBuilder::default().build();
let (hl_client_key, hl_server_key) = generate_keys(config);
let underlying_ck: tfhe::shortint::ClientKey = (*hl_client_key.as_ref()).clone().into();
let underlying_sk: tfhe::shortint::ServerKey = (*hl_server_key.as_ref()).clone().into();
let (client_key, server_key): (ClientKey, ServerKey) = gen_keys(PARAM_MESSAGE_1_CARRY_1_KS_PBS);
let ksk = KeySwitchingKey::new(
(&client_key, &server_key),
(&underlying_ck, &underlying_sk),
PARAM_KEYSWITCH_1_1_KS_PBS_TO_2_2_KS_PBS,
);
let key_string = "0053A6F94C9FF24598EB".to_string();
let mut key = [0; 80];
for i in (0..key_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&key_string[i..i + 2], 16).unwrap();
for j in 0..8 {
key[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let iv_string = "0D74DB42A91077DE45AC".to_string();
let mut iv = [0; 80];
for i in (0..iv_string.len()).step_by(2) {
let mut val = u64::from_str_radix(&iv_string[i..i + 2], 16).unwrap();
for j in 0..8 {
iv[8 * (i >> 1) + j] = val % 2;
val >>= 1;
}
}
let output_0_63 = "F4CD954A717F26A7D6930830C4E7CF0819F80E03F25F342C64ADC66ABA7F8A8E6EAA49F23632AE3CD41A7BD290A0132F81C6D4043B6E397D7388F3A03B5FE358".to_string();
let cipher_key = key.map(|x| client_key.encrypt(x));
let mut ciphered_message = vec![FheUint64::try_encrypt(0u64, &hl_client_key).unwrap(); 9];
let mut trivium = TriviumStreamShortint::new(cipher_key, iv, server_key, ksk, hl_server_key);
let mut vec = Vec::<u64>::with_capacity(8);
while vec.len() < 8 {
let trans_ciphered_message = trivium.trans_encrypt_64(ciphered_message.pop().unwrap());
vec.push(trans_ciphered_message.decrypt(&hl_client_key));
}
let hexadecimal = get_hexagonal_string_from_u64(vec);
assert_eq!(output_0_63, hexadecimal[0..64 * 2]);
}

View File

@@ -0,0 +1,225 @@
//! This module implements the Trivium stream cipher, using booleans or FheBool
//! for the representation of the inner bits.
use crate::static_deque::StaticDeque;
use tfhe::prelude::*;
use tfhe::{set_server_key, unset_server_key, FheBool, ServerKey};
use rayon::prelude::*;
/// Internal trait specifying which operations are necessary for TriviumStream generic type
pub trait TriviumBoolInput<OpOutput>:
Sized
+ Clone
+ std::ops::BitXor<Output = OpOutput>
+ std::ops::BitAnd<Output = OpOutput>
+ std::ops::Not<Output = OpOutput>
{
}
impl TriviumBoolInput<bool> for bool {}
impl TriviumBoolInput<bool> for &bool {}
impl TriviumBoolInput<FheBool> for FheBool {}
impl TriviumBoolInput<FheBool> for &FheBool {}
/// TriviumStream: a struct implementing the Trivium stream cipher, using T for the internal
/// representation of bits (bool or FheBool). To be able to compute FHE operations, it also owns
/// an Option for a ServerKey.
pub struct TriviumStream<T> {
a: StaticDeque<93, T>,
b: StaticDeque<84, T>,
c: StaticDeque<111, T>,
fhe_key: Option<ServerKey>,
}
impl TriviumStream<bool> {
/// Constructor for `TriviumStream<bool>`: arguments are the secret key and the input vector.
/// Outputs a TriviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(key: [bool; 80], iv: [bool; 80]) -> TriviumStream<bool> {
// Initialization of Trivium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register = [false; 93];
let mut b_register = [false; 84];
let mut c_register = [false; 111];
for i in 0..80 {
a_register[93 - 80 + i] = key[i];
b_register[84 - 80 + i] = iv[i];
}
c_register[0] = true;
c_register[1] = true;
c_register[2] = true;
TriviumStream::<bool>::new_from_registers(a_register, b_register, c_register, None)
}
}
impl TriviumStream<FheBool> {
/// Constructor for `TriviumStream<FheBool>`: arguments are the encrypted secret key and input
/// vector, and the FHE server key.
/// Outputs a TriviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(key: [FheBool; 80], iv: [bool; 80], sk: &ServerKey) -> TriviumStream<FheBool> {
set_server_key(sk.clone());
// Initialization of Trivium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register = [false; 93].map(FheBool::encrypt_trivial);
let mut b_register = [false; 84].map(FheBool::encrypt_trivial);
let mut c_register = [false; 111].map(FheBool::encrypt_trivial);
for i in 0..80 {
a_register[93 - 80 + i] = key[i].clone();
b_register[84 - 80 + i] = FheBool::encrypt_trivial(iv[i]);
}
c_register[0] = FheBool::try_encrypt_trivial(true).unwrap();
c_register[1] = FheBool::try_encrypt_trivial(true).unwrap();
c_register[2] = FheBool::try_encrypt_trivial(true).unwrap();
unset_server_key();
TriviumStream::<FheBool>::new_from_registers(
a_register,
b_register,
c_register,
Some(sk.clone()),
)
}
}
impl<T> TriviumStream<T>
where
T: TriviumBoolInput<T> + std::marker::Send + std::marker::Sync,
for<'a> &'a T: TriviumBoolInput<T>,
{
/// Internal generic constructor: arguments are already prepared registers, and an optional FHE
/// server key
fn new_from_registers(
a_register: [T; 93],
b_register: [T; 84],
c_register: [T; 111],
key: Option<ServerKey>,
) -> Self {
let mut ret = Self {
a: StaticDeque::<93, T>::new(a_register),
b: StaticDeque::<84, T>::new(b_register),
c: StaticDeque::<111, T>::new(c_register),
fhe_key: key,
};
ret.init();
ret
}
/// The specification of Trivium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes one turn of the stream, updating registers and outputting the new bit.
pub fn next_bool(&mut self) -> T {
match &self.fhe_key {
Some(sk) => set_server_key(sk.clone()),
None => (),
};
let [o, a, b, c] = self.get_output_and_values(0);
self.a.push(a);
self.b.push(b);
self.c.push(c);
o
}
/// Computes a potential future step of Trivium, n terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, n: usize) -> [T; 4] {
assert!(n < 65);
let (((temp_a, temp_b), (temp_c, a_and)), (b_and, c_and)) = rayon::join(
|| {
rayon::join(
|| {
rayon::join(
|| &self.a[65 - n] ^ &self.a[92 - n],
|| &self.b[68 - n] ^ &self.b[83 - n],
)
},
|| {
rayon::join(
|| &self.c[65 - n] ^ &self.c[110 - n],
|| &self.a[91 - n] & &self.a[90 - n],
)
},
)
},
|| {
rayon::join(
|| &self.b[82 - n] & &self.b[81 - n],
|| &self.c[109 - n] & &self.c[108 - n],
)
},
);
let ((o, a), (b, c)) = rayon::join(
|| {
rayon::join(
|| &(&temp_a ^ &temp_b) ^ &temp_c,
|| &temp_c ^ &(&c_and ^ &self.a[68 - n]),
)
},
|| {
rayon::join(
|| &temp_a ^ &(&a_and ^ &self.b[77 - n]),
|| &temp_b ^ &(&b_and ^ &self.c[86 - n]),
)
},
);
[o, a, b, c]
}
/// This calls `get_output_and_values` in parallel 64 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[T; 4]> {
(0..64)
.into_par_iter()
.map(|x| self.get_output_and_values(x))
.rev()
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<T> {
match &self.fhe_key {
Some(sk) => {
rayon::broadcast(|_| set_server_key(sk.clone()));
}
None => (),
}
let mut values = self.get_64_output_and_values();
match &self.fhe_key {
Some(_) => {
rayon::broadcast(|_| unset_server_key());
}
None => (),
}
let mut ret = Vec::<T>::with_capacity(64);
while let Some([o, a, b, c]) = values.pop() {
ret.push(o);
self.a.push(a);
self.b.push(b);
self.c.push(c);
}
ret
}
}

View File

@@ -0,0 +1,241 @@
//! This module implements the Trivium stream cipher, using u8 or FheUint8
//! for the representation of the inner bits.
use crate::static_deque::{StaticByteDeque, StaticByteDequeInput};
use tfhe::prelude::*;
use tfhe::{set_server_key, unset_server_key, FheUint8, ServerKey};
use rayon::prelude::*;
/// Internal trait specifying which operations are necessary for TriviumStreamByte generic type
pub trait TriviumByteInput<OpOutput>:
Sized
+ Clone
+ Send
+ Sync
+ StaticByteDequeInput<OpOutput>
+ std::ops::BitXor<Output = OpOutput>
+ std::ops::BitAnd<Output = OpOutput>
+ std::ops::Shr<u8, Output = OpOutput>
+ std::ops::Shl<u8, Output = OpOutput>
+ std::ops::Add<Output = OpOutput>
{
}
impl TriviumByteInput<u8> for u8 {}
impl TriviumByteInput<u8> for &u8 {}
impl TriviumByteInput<FheUint8> for FheUint8 {}
impl TriviumByteInput<FheUint8> for &FheUint8 {}
/// TriviumStreamByte: a struct implementing the Trivium stream cipher, using T for the internal
/// representation of bits (u8 or FheUint8). To be able to compute FHE operations, it also owns
/// an Option for a ServerKey.
/// Since the original Trivium registers' sizes are not a multiple of 8, these registers (which
/// store byte-like objects) have a size that is the eighth of the closest multiple of 8 above the
/// originals' sizes.
pub struct TriviumStreamByte<T> {
a_byte: StaticByteDeque<12, T>,
b_byte: StaticByteDeque<11, T>,
c_byte: StaticByteDeque<14, T>,
fhe_key: Option<ServerKey>,
}
impl TriviumStreamByte<u8> {
/// Constructor for `TriviumStreamByte<u8>`: arguments are the secret key and the input vector.
/// Outputs a TriviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(key: [u8; 10], iv: [u8; 10]) -> TriviumStreamByte<u8> {
// Initialization of Trivium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_byte_reg = [0u8; 12];
let mut b_byte_reg = [0u8; 11];
let mut c_byte_reg = [0u8; 14];
for i in 0..10 {
a_byte_reg[12 - 10 + i] = key[i];
b_byte_reg[11 - 10 + i] = iv[i];
}
// Magic number 14, aka 00001110: this represents the 3 ones at the beginning of the c
// registers, with additional zeros to make the register's size a multiple of 8.
c_byte_reg[0] = 14;
let mut ret =
TriviumStreamByte::<u8>::new_from_registers(a_byte_reg, b_byte_reg, c_byte_reg, None);
ret.init();
ret
}
}
impl TriviumStreamByte<FheUint8> {
/// Constructor for `TriviumStream<FheUint8>`: arguments are the encrypted secret key and input
/// vector, and the FHE server key.
/// Outputs a TriviumStream object already initialized (1152 steps have been run before
/// returning)
pub fn new(
key: [FheUint8; 10],
iv: [u8; 10],
server_key: &ServerKey,
) -> TriviumStreamByte<FheUint8> {
set_server_key(server_key.clone());
// Initialization of Trivium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_byte_reg = [0u8; 12].map(FheUint8::encrypt_trivial);
let mut b_byte_reg = [0u8; 11].map(FheUint8::encrypt_trivial);
let mut c_byte_reg = [0u8; 14].map(FheUint8::encrypt_trivial);
for i in 0..10 {
a_byte_reg[12 - 10 + i] = key[i].clone();
b_byte_reg[11 - 10 + i] = FheUint8::encrypt_trivial(iv[i]);
}
// Magic number 14, aka 00001110: this represents the 3 ones at the beginning of the c
// registers, with additional zeros to make the register's size a multiple of 8.
c_byte_reg[0] = FheUint8::encrypt_trivial(14u8);
unset_server_key();
let mut ret = TriviumStreamByte::<FheUint8>::new_from_registers(
a_byte_reg,
b_byte_reg,
c_byte_reg,
Some(server_key.clone()),
);
ret.init();
ret
}
}
impl<T> TriviumStreamByte<T>
where
T: TriviumByteInput<T> + Send,
for<'a> &'a T: TriviumByteInput<T>,
{
/// Internal generic constructor: arguments are already prepared registers, and an optional FHE
/// server key
fn new_from_registers(
a_register: [T; 12],
b_register: [T; 11],
c_register: [T; 14],
sk: Option<ServerKey>,
) -> Self {
Self {
a_byte: StaticByteDeque::<12, T>::new(a_register),
b_byte: StaticByteDeque::<11, T>::new(b_register),
c_byte: StaticByteDeque::<14, T>::new(c_register),
fhe_key: sk,
}
}
/// The specification of Trivium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes 8 potential future step of Trivium, b*8 terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, b: usize) -> [T; 4] {
let n = b * 8 + 7;
assert!(n < 65);
let ((a1, a2, a3, a4, a5), ((b1, b2, b3, b4, b5), (c1, c2, c3, c4, c5))) = rayon::join(
|| Self::get_bytes(&self.a_byte, [91 - n, 90 - n, 68 - n, 65 - n, 92 - n]),
|| {
rayon::join(
|| Self::get_bytes(&self.b_byte, [82 - n, 81 - n, 77 - n, 68 - n, 83 - n]),
|| Self::get_bytes(&self.c_byte, [109 - n, 108 - n, 86 - n, 65 - n, 110 - n]),
)
},
);
let (((temp_a, temp_b), (temp_c, a_and)), (b_and, c_and)) = rayon::join(
|| {
rayon::join(
|| rayon::join(|| a4 ^ a5, || b4 ^ b5),
|| rayon::join(|| c4 ^ c5, || a1 & a2),
)
},
|| rayon::join(|| b1 & b2, || c1 & c2),
);
let (temp_a_2, temp_b_2, temp_c_2) = (temp_a.clone(), temp_b.clone(), temp_c.clone());
let ((o, a), (b, c)) = rayon::join(
|| {
rayon::join(
|| (temp_a_2 ^ temp_b_2) ^ temp_c_2,
|| temp_c ^ ((c_and) ^ a3),
)
},
|| rayon::join(|| temp_a ^ (a_and ^ b3), || temp_b ^ (b_and ^ c3)),
);
[o, a, b, c]
}
/// This calls `get_output_and_values` in parallel 8 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[T; 4]> {
(0..8)
.into_par_iter()
.map(|i| self.get_output_and_values(i))
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits (in 8 bytes) all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<T> {
match &self.fhe_key {
Some(sk) => {
rayon::broadcast(|_| set_server_key(sk.clone()));
}
None => (),
}
let values = self.get_64_output_and_values();
match &self.fhe_key {
Some(_) => {
rayon::broadcast(|_| unset_server_key());
}
None => (),
}
let mut bytes = Vec::<T>::with_capacity(8);
for [o, a, b, c] in values {
self.a_byte.push(a);
self.b_byte.push(b);
self.c_byte.push(c);
bytes.push(o);
}
bytes
}
/// Reconstructs a bunch of 5 bytes in a parallel fashion.
fn get_bytes<const N: usize>(
reg: &StaticByteDeque<N, T>,
offsets: [usize; 5],
) -> (T, T, T, T, T) {
let mut ret = offsets
.par_iter()
.rev()
.map(|&i| reg.byte(i))
.collect::<Vec<_>>();
(
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
ret.pop().unwrap(),
)
}
}
impl TriviumStreamByte<FheUint8> {
pub fn get_server_key(&self) -> &ServerKey {
self.fhe_key.as_ref().unwrap()
}
}

View File

@@ -0,0 +1,189 @@
use crate::static_deque::StaticDeque;
use tfhe::shortint::prelude::*;
use rayon::prelude::*;
/// TriviumStreamShortint: a struct implementing the Trivium stream cipher, using a generic
/// Ciphertext for the internal representation of bits (intended to represent a single bit). To be
/// able to compute FHE operations, it also owns a ServerKey.
pub struct TriviumStreamShortint {
a: StaticDeque<93, Ciphertext>,
b: StaticDeque<84, Ciphertext>,
c: StaticDeque<111, Ciphertext>,
internal_server_key: ServerKey,
transciphering_casting_key: KeySwitchingKey,
hl_server_key: tfhe::ServerKey,
}
impl TriviumStreamShortint {
/// Constructor for TriviumStreamShortint: arguments are the secret key and the input vector,
/// and a ServerKey reference. Outputs a TriviumStream object already initialized (1152
/// steps have been run before returning)
pub fn new(
key: [Ciphertext; 80],
iv: [u64; 80],
sk: ServerKey,
ksk: KeySwitchingKey,
hl_sk: tfhe::ServerKey,
) -> Self {
// Initialization of Trivium registers: a has the secret key, b the input vector,
// and c a few ones.
let mut a_register: [Ciphertext; 93] = [0; 93].map(|x| sk.create_trivial(x));
let mut b_register: [Ciphertext; 84] = [0; 84].map(|x| sk.create_trivial(x));
let mut c_register: [Ciphertext; 111] = [0; 111].map(|x| sk.create_trivial(x));
for i in 0..80 {
a_register[93 - 80 + i] = key[i].clone();
b_register[84 - 80 + i] = sk.create_trivial(iv[i]);
}
c_register[0] = sk.create_trivial(1);
c_register[1] = sk.create_trivial(1);
c_register[2] = sk.create_trivial(1);
let mut ret = Self {
a: StaticDeque::<93, Ciphertext>::new(a_register),
b: StaticDeque::<84, Ciphertext>::new(b_register),
c: StaticDeque::<111, Ciphertext>::new(c_register),
internal_server_key: sk,
transciphering_casting_key: ksk,
hl_server_key: hl_sk,
};
ret.init();
ret
}
/// The specification of Trivium includes running 1152 (= 18*64) unused steps to mix up the
/// registers, before starting the proper stream
fn init(&mut self) {
for _ in 0..18 {
self.next_64();
}
}
/// Computes one turn of the stream, updating registers and outputting the new bit.
pub fn next_ct(&mut self) -> Ciphertext {
let [o, a, b, c] = self.get_output_and_values(0);
self.a.push(a);
self.b.push(b);
self.c.push(c);
o
}
/// Computes a potential future step of Trivium, n terms in the future. This does not update
/// registers, but rather returns with the output, the three values that will be used to
/// update the registers, when the time is right. This function is meant to be used in
/// parallel.
fn get_output_and_values(&self, n: usize) -> [Ciphertext; 4] {
let (a1, a2, a3, a4, a5) = (
&self.a[65 - n],
&self.a[92 - n],
&self.a[91 - n],
&self.a[90 - n],
&self.a[68 - n],
);
let (b1, b2, b3, b4, b5) = (
&self.b[68 - n],
&self.b[83 - n],
&self.b[82 - n],
&self.b[81 - n],
&self.b[77 - n],
);
let (c1, c2, c3, c4, c5) = (
&self.c[65 - n],
&self.c[110 - n],
&self.c[109 - n],
&self.c[108 - n],
&self.c[86 - n],
);
let temp_a = self.internal_server_key.unchecked_add(a1, a2);
let temp_b = self.internal_server_key.unchecked_add(b1, b2);
let temp_c = self.internal_server_key.unchecked_add(c1, c2);
let ((new_a, new_b), (new_c, o)) = rayon::join(
|| {
rayon::join(
|| {
let mut new_a = self.internal_server_key.unchecked_bitand(c3, c4);
self.internal_server_key
.unchecked_add_assign(&mut new_a, a5);
self.internal_server_key
.unchecked_add_assign(&mut new_a, &temp_c);
self.internal_server_key.message_extract_assign(&mut new_a);
new_a
},
|| {
let mut new_b = self.internal_server_key.unchecked_bitand(a3, a4);
self.internal_server_key
.unchecked_add_assign(&mut new_b, b5);
self.internal_server_key
.unchecked_add_assign(&mut new_b, &temp_a);
self.internal_server_key.message_extract_assign(&mut new_b);
new_b
},
)
},
|| {
rayon::join(
|| {
let mut new_c = self.internal_server_key.unchecked_bitand(b3, b4);
self.internal_server_key
.unchecked_add_assign(&mut new_c, c5);
self.internal_server_key
.unchecked_add_assign(&mut new_c, &temp_b);
self.internal_server_key.message_extract_assign(&mut new_c);
new_c
},
|| {
self.internal_server_key.bitxor(
&self.internal_server_key.unchecked_add(&temp_a, &temp_b),
&temp_c,
)
},
)
},
);
[o, new_a, new_b, new_c]
}
/// This calls `get_output_and_values` in parallel 64 times, and stores all results in a Vec.
fn get_64_output_and_values(&self) -> Vec<[Ciphertext; 4]> {
(0..64)
.into_par_iter()
.map(|x| self.get_output_and_values(x))
.rev()
.collect()
}
/// Computes 64 turns of the stream, outputting the 64 bits all at once in a
/// Vec (first value is oldest, last is newest)
pub fn next_64(&mut self) -> Vec<Ciphertext> {
let mut values = self.get_64_output_and_values();
let mut ret = Vec::<Ciphertext>::with_capacity(64);
while let Some([o, a, b, c]) = values.pop() {
ret.push(o);
self.a.push(a);
self.b.push(b);
self.c.push(c);
}
ret
}
pub fn get_internal_server_key(&self) -> &ServerKey {
&self.internal_server_key
}
pub fn get_casting_key(&self) -> &KeySwitchingKey {
&self.transciphering_casting_key
}
pub fn get_hl_server_key(&self) -> &tfhe::ServerKey {
&self.hl_server_key
}
}

View File

@@ -39,8 +39,10 @@ parser.add_argument('--walk-subdirs', dest='walk_subdirs', action='store_true',
help='Check for results in subdirectories')
parser.add_argument('--key-sizes', dest='key_sizes', action='store_true',
help='Parse only the results regarding keys size measurements')
parser.add_argument('--key-gen', dest='key_gen', action='store_true',
help='Parse only the results regarding keys generation time measurements')
parser.add_argument('--throughput', dest='throughput', action='store_true',
help='Compute and append number of operations per millisecond and'
help='Compute and append number of operations per second and'
'operations per dollar')
parser.add_argument('--backend', dest='backend', default='cpu',
help='Backend on which benchmarks have run')
@@ -55,7 +57,7 @@ def recursive_parse(directory, walk_subdirs=False, name_suffix="", compute_throu
:param directory: path to directory that contains raw results as :class:`pathlib.Path`
:param walk_subdirs: traverse results subdirectories if parameters changes for benchmark case.
:param name_suffix: a :class:`str` suffix to apply to each test name found
:param compute_throughput: compute number of operations per millisecond and operations per
:param compute_throughput: compute number of operations per second and operations per
dollar
:param hardware_hourly_cost: hourly cost of the hardware used in dollar
@@ -93,6 +95,7 @@ def recursive_parse(directory, walk_subdirs=False, name_suffix="", compute_throu
for stat_name, value in parse_estimate_file(subdir).items():
test_name_parts = list(filter(None, [test_name, stat_name, name_suffix]))
result_values.append(
_create_point(
value,
@@ -105,12 +108,25 @@ def recursive_parse(directory, walk_subdirs=False, name_suffix="", compute_throu
)
)
# This is a special case where PBS are blasted as vector LWE ciphertext with
# variable length to saturate the machine. To get the actual throughput we need to
# multiply by the length of the vector.
if "PBS_throughput" in test_name and "chunk" in test_name:
try:
multiplier = int(test_name.split("chunk")[0].split("_")[-1])
except ValueError:
parsing_failures.append((full_name,
"failed to extract throughput multiplier"))
continue
else:
multiplier = 1
if stat_name == "mean" and compute_throughput:
test_suffix = "ops-per-ms"
test_suffix = "ops-per-sec"
test_name_parts.append(test_suffix)
result_values.append(
_create_point(
compute_ops_per_millisecond(value),
multiplier * compute_ops_per_second(value),
"_".join(test_name_parts),
bench_class,
"throughput",
@@ -126,7 +142,7 @@ def recursive_parse(directory, walk_subdirs=False, name_suffix="", compute_throu
test_name_parts.append(test_suffix)
result_values.append(
_create_point(
compute_ops_per_dollar(value, hardware_hourly_cost),
multiplier * compute_ops_per_dollar(value, hardware_hourly_cost),
"_".join(test_name_parts),
bench_class,
"throughput",
@@ -177,9 +193,9 @@ def parse_estimate_file(directory):
}
def parse_key_sizes(result_file):
def _parse_key_results(result_file, bench_type):
"""
Parse file containing key sizes results. The file must be formatted as CSV.
Parse file containing results about operation on keys. The file must be formatted as CSV.
:param result_file: results file as :class:`pathlib.Path`
@@ -202,13 +218,35 @@ def parse_key_sizes(result_file):
"test": test_name,
"name": display_name,
"class": "keygen",
"type": "keysize",
"type": bench_type,
"operator": operator,
"params": params})
return result_values, parsing_failures
def parse_key_sizes(result_file):
"""
Parse file containing key sizes results. The file must be formatted as CSV.
:param result_file: results file as :class:`pathlib.Path`
:return: tuple of :class:`list` as (data points, parsing failures)
"""
return _parse_key_results(result_file, "keysize")
def parse_key_gen_time(result_file):
"""
Parse file containing key generation time results. The file must be formatted as CSV.
:param result_file: results file as :class:`pathlib.Path`
:return: tuple of :class:`list` as (data points, parsing failures)
"""
return _parse_key_results(result_file, "latency")
def get_parameters(bench_id):
"""
Get benchmarks parameters recorded for a given benchmark case.
@@ -242,15 +280,15 @@ def compute_ops_per_dollar(data_point, product_hourly_cost):
return ONE_HOUR_IN_NANOSECONDS / (product_hourly_cost * data_point)
def compute_ops_per_millisecond(data_point):
def compute_ops_per_second(data_point):
"""
Compute numbers of operations per millisecond for a given ``data_point``.
Compute numbers of operations per second for a given ``data_point``.
:param data_point: timing value measured during benchmark in nanoseconds
:return: number of operations per millisecond
:return: number of operations per second
"""
return 1E6 / data_point
return 1E9 / data_point
def _parse_file_to_json(directory, filename):
@@ -301,7 +339,7 @@ def check_mandatory_args(input_args):
for arg_name in vars(input_args):
if arg_name in ["results_dir", "output_file", "name_suffix",
"append_results", "walk_subdirs", "key_sizes",
"throughput"]:
"key_gen", "throughput"]:
continue
if not getattr(input_args, arg_name):
missing_args.append(arg_name)
@@ -318,7 +356,15 @@ if __name__ == "__main__":
#failures = []
raw_results = pathlib.Path(args.results)
if not args.key_sizes:
if args.key_sizes or args.key_gen:
if args.key_sizes:
print("Parsing key sizes results... ")
results, failures = parse_key_sizes(raw_results)
if args.key_gen:
print("Parsing key generation time results... ")
results, failures = parse_key_gen_time(raw_results)
else:
print("Parsing benchmark results... ")
hardware_cost = None
if args.throughput:
@@ -334,9 +380,7 @@ if __name__ == "__main__":
results, failures = recursive_parse(raw_results, args.walk_subdirs, args.name_suffix,
args.throughput, hardware_cost)
else:
print("Parsing key sizes results... ")
results, failures = parse_key_sizes(raw_results)
print("Parsing results done")
output_file = pathlib.Path(args.output_file)

View File

@@ -1,3 +1,4 @@
{
"m6i.metal": 7.168
"m6i.metal": 7.168,
"hpc7a.96xlarge": 7.7252
}

76
ci/lattice_estimator.sage Executable file
View File

@@ -0,0 +1,76 @@
"""
lattice_estimator
-----------------
Test cryptographic parameters set against several attacks to estimate their security level.
"""
import pathlib
import sys
sys.path.insert(1, 'lattice-estimator')
from estimator import *
model = RC.BDGL16
def check_security(filename):
"""
Run lattice estimator to determine if a parameters set is secure or not.
:param filename: name of the file containing parameters set
:return: :class:`list` of parameters to update
"""
filepath = pathlib.Path("ci", filename)
load(filepath)
print(f"Parsing parameters in {filepath}")
to_update = []
for param in all_params:
if param.tag.startswith("TFHE_LIB_PARAMETERS"):
# This third-party parameters set is known to be less secure, just skip the analysis.
continue
print(f"\t{param.tag}...\t", end= "")
try:
# The lattice estimator is not able to manage such large dimension.
# If we have the security for smaller `n` then we have security for larger ones.
if param.n > 16384:
param = param.updated(n = 16384)
usvp_level = LWE.primal_usvp(param, red_cost_model = model)
dual_level = LWE.dual_hybrid(param, red_cost_model = model)
estimator_level = log(min(usvp_level["rop"], dual_level["rop"]),2 )
security_level = f"security level = {estimator_level} bits"
if estimator_level < 127:
print("FAIL\t({security_level})")
reason = f"attained {security_level} target is 128 bits"
to_update.append((param, reason))
continue
except Exception as err:
print("FAIL")
to_update.append((param, f"{repr(err)}"))
else:
print(f"OK\t({security_level})")
return to_update
if __name__ == "__main__":
params_to_update = []
for params_filename in ("boolean_parameters_lattice_estimator.sage",
"shortint_classic_parameters_lattice_estimator.sage",
"shortint_multi_bit_parameters_lattice_estimator.sage"):
params_to_update.extend(check_security(params_filename))
if params_to_update:
print("Some parameters need update")
print("----------------------------")
for param, reason in params_to_update:
print(f"[{param.tag}] reason: {reason} (param)")
sys.exit(int(1)) # Explicit conversion is needed to make this call work
else:
print("All parameters passed the security check")

View File

@@ -0,0 +1,102 @@
import argparse
from pathlib import Path
import json
def main(args):
criterion_dir = Path(args.criterion_dir)
output_file = Path(args.output_file)
data = []
for json_file in sorted(criterion_dir.glob("**/*.json")):
if json_file.parent.name == "base" or json_file.name != "benchmark.json":
continue
try:
bench_data = json.loads(json_file.read_text())
estimate_file = json_file.with_name("estimates.json")
estimate_data = json.loads(estimate_file.read_text())
bench_function_id = bench_data["function_id"]
split = bench_function_id.split("::")
if split.len() == 5: # Signed integers
(_, _, function_name, parameter_set, bits) = split
else: # Unsigned integers
(_, function_name, parameter_set, bits) = split
if "_scalar_" in bits:
(bits, scalar) = bits.split("_bits_scalar_")
bits = int(bits)
scalar = int(scalar)
else:
(bits, _) = bits.split("_")
bits = int(bits)
scalar = None
estimate_mean_ms = estimate_data["mean"]["point_estimate"] / 1000000
estimate_lower_bound_ms = (
estimate_data["mean"]["confidence_interval"]["lower_bound"] / 1000000
)
estimate_upper_bound_ms = (
estimate_data["mean"]["confidence_interval"]["upper_bound"] / 1000000
)
data.append(
(
function_name,
parameter_set,
bits,
scalar,
estimate_mean_ms,
estimate_lower_bound_ms,
estimate_upper_bound_ms,
)
)
except:
pass
if len(data) == 0:
print("No integer bench found, skipping writing output file")
return
with open(output_file, "w", encoding="utf-8") as output:
output.write(
"function_name,parameter_set,bits,scalar,mean_ms,"
"confidence_interval_lower_bound_ms,confidence_interval_upper_bound_ms\n"
)
# Sort by func_name, bit width and then parameters
data.sort(key=lambda x: (x[0], x[2], x[1]))
for dat in data:
(
function_name,
parameter_set,
bits,
scalar,
estimate_mean_ms,
estimate_lower_bound_ms,
estimate_upper_bound_ms,
) = dat
output.write(
f"{function_name},{parameter_set},{bits},{scalar},{estimate_mean_ms},"
f"{estimate_lower_bound_ms},{estimate_upper_bound_ms}\n"
)
if __name__ == "__main__":
parser = argparse.ArgumentParser("Parse criterion results to csv file")
parser.add_argument(
"--criterion-dir",
type=str,
default="target/criterion",
help="Where to look for criterion result json files",
)
parser.add_argument(
"--output-file",
type=str,
default="parsed_benches.csv",
help="Path of the output file, will be csv formatted",
)
main(parser.parse_args())

View File

@@ -1,28 +1,83 @@
[profile.cpu-big]
region = "eu-west-3"
image_id = "ami-04deffe45b5b236fd"
image_id = "ami-051942e4055555752"
instance_type = "m6i.32xlarge"
[profile.bench]
[profile.cpu-big_fallback]
region = "us-east-1"
image_id = "ami-04e3bb9aebb6786df"
instance_type = "m6i.32xlarge"
[profile.cpu-small]
region = "eu-west-3"
image_id = "ami-04deffe45b5b236fd"
instance_type = "m6i.metal"
image_id = "ami-051942e4055555752"
instance_type = "m6i.4xlarge"
[profile.bench]
region = "eu-west-1"
image_id = "ami-0e88d98b86aff13de"
instance_type = "hpc7a.96xlarge"
[command.cpu_test]
workflow = "aws_tfhe_tests.yml"
profile = "cpu-big"
check_run_name = "CPU AWS Tests"
[command.cpu_integer_test]
[command.cpu_unsigned_integer_test]
workflow = "aws_tfhe_integer_tests.yml"
profile = "cpu-big"
check_run_name = "CPU Integer AWS Tests"
check_run_name = "CPU Unsigned Integer AWS Tests"
[command.cpu_signed_integer_test]
workflow = "aws_tfhe_signed_integer_tests.yml"
profile = "cpu-big"
check_run_name = "CPU Signed Integer AWS Tests"
[command.cpu_wasm_test]
workflow = "aws_tfhe_wasm_tests.yml"
profile = "cpu-small"
check_run_name = "CPU AWS WASM Tests"
[command.cpu_fast_test]
workflow = "aws_tfhe_fast_tests.yml"
profile = "cpu-big"
check_run_name = "CPU AWS Fast Tests"
[command.integer_full_bench]
workflow = "integer_full_benchmark.yml"
profile = "bench"
check_run_name = "Integer CPU AWS Benchmarks Full Suite"
[command.signed_integer_full_bench]
workflow = "signed_integer_full_benchmark.yml"
profile = "bench"
check_run_name = "Signed Integer CPU AWS Benchmarks Full Suite"
[command.integer_bench]
workflow = "integer_benchmark.yml"
profile = "bench"
check_run_name = "Integer CPU AWS Benchmarks"
[command.integer_multi_bit_bench]
workflow = "integer_multi_bit_benchmark.yml"
profile = "bench"
check_run_name = "Integer multi bit CPU AWS Benchmarks"
[command.signed_integer_bench]
workflow = "signed_integer_benchmark.yml"
profile = "bench"
check_run_name = "Signed integer CPU AWS Benchmarks"
[command.signed_integer_multi_bit_bench]
workflow = "signed_integer_multi_bit_benchmark.yml"
profile = "bench"
check_run_name = "Signed integer multi bit CPU AWS Benchmarks"
[command.shortint_full_bench]
workflow = "shortint_full_benchmark.yml"
profile = "bench"
check_run_name = "Shortint CPU AWS Benchmarks Full Suite"
[command.shortint_bench]
workflow = "shortint_benchmark.yml"
profile = "bench"
@@ -37,3 +92,18 @@ check_run_name = "Boolean CPU AWS Benchmarks"
workflow = "pbs_benchmark.yml"
profile = "bench"
check_run_name = "PBS CPU AWS Benchmarks"
[command.wasm_client_bench]
workflow = "wasm_client_benchmark.yml"
profile = "cpu-small"
check_run_name = "WASM Client AWS Benchmarks"
[command.csprng_randomness_testing]
workflow = "csprng_randomness_testing.yml"
profile = "cpu-small"
check_run_name = "CSPRNG randomness testing"
[command.code_coverage]
workflow = "code_coverage.yml"
profile = "cpu-small"
check_run_name = "Code coverage"

4
codecov.yml Normal file
View File

@@ -0,0 +1,4 @@
coverage:
status:
# Disable patch checks in GitHub until all tfhe-rs layers have coverage implemented.
patch: false

View File

@@ -0,0 +1,53 @@
[package]
name = "concrete-csprng"
version = "0.4.0"
edition = "2021"
license = "BSD-3-Clause-Clear"
description = "Cryptographically Secure PRNG used in the TFHE-rs library."
homepage = "https://zama.ai/"
documentation = "https://docs.zama.ai/tfhe-rs"
repository = "https://github.com/zama-ai/tfhe-rs"
readme = "README.md"
keywords = ["fully", "homomorphic", "encryption", "fhe", "cryptography"]
rust-version = "1.72"
[dependencies]
aes = "0.8.2"
rayon = { version = "1.5.0", optional = true }
[target.'cfg(target_os = "macos")'.dependencies]
libc = "0.2.133"
[dev-dependencies]
rand = "0.8.3"
criterion = "0.5.1"
clap = "=4.4.4"
[features]
parallel = ["rayon"]
seeder_x86_64_rdseed = []
seeder_unix = []
generator_x86_64_aesni = []
generator_fallback = []
generator_aarch64_aes = []
x86_64 = [
"parallel",
"seeder_x86_64_rdseed",
"generator_x86_64_aesni",
"generator_fallback",
]
x86_64-unix = ["x86_64", "seeder_unix"]
aarch64 = ["parallel", "generator_aarch64_aes", "generator_fallback"]
aarch64-unix = ["aarch64", "seeder_unix"]
[[bench]]
name = "benchmark"
path = "benches/benchmark.rs"
harness = false
required-features = ["seeder_x86_64_rdseed", "generator_x86_64_aesni"]
[[example]]
name = "generate"
path = "examples/generate.rs"
required-features = ["seeder_unix", "generator_fallback"]

28
concrete-csprng/LICENSE Normal file
View File

@@ -0,0 +1,28 @@
BSD 3-Clause Clear License
Copyright © 2023 ZAMA.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimer in the documentation and/or other
materials provided with the distribution.
3. Neither the name of ZAMA nor the names of its contributors may be used to endorse
or promote products derived from this software without specific prior written permission.
NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY THIS LICENSE.
THIS SOFTWARE IS PROVIDED BY THE ZAMA AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
ZAMA OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

23
concrete-csprng/README.md Normal file
View File

@@ -0,0 +1,23 @@
# Concrete CSPRNG
This crate contains a fast *Cryptographically Secure Pseudoramdon Number Generator*, used in the
['concrete-core'](https://crates.io/crates/concrete-core) library, you can find it [here](../concrete-core/) in this repo.
The implementation is based on the AES blockcipher used in CTR mode, as described in the ISO/IEC
18033-4 standard.
Two implementations are available, an accelerated one on x86_64 CPUs with the `aes` feature and the `sse2` feature, and a pure software one that can be used on other platforms.
The crate also makes two seeders available, one needing the x86_64 feature `rdseed` and another one based on the Unix random device `/dev/random` the latter requires the user to provide a secret.
## Running the benchmarks
To execute the benchmarks on an x86_64 platform:
```shell
RUSTFLAGS="-Ctarget-cpu=native" cargo bench --features=seeder_x86_64_rdseed,generator_x86_64_aesni
```
## License
This software is distributed under the BSD-3-Clause-Clear license. If you have any questions,
please contact us at `hello@zama.ai`.

View File

@@ -0,0 +1,54 @@
use concrete_csprng::generators::{
AesniRandomGenerator, BytesPerChild, ChildrenCount, RandomGenerator,
};
use concrete_csprng::seeders::{RdseedSeeder, Seeder};
use criterion::{black_box, criterion_group, criterion_main, Criterion};
// The number of bytes to generate during one benchmark iteration.
const N_GEN: usize = 1_000_000;
fn parent_generate(c: &mut Criterion) {
let mut seeder = RdseedSeeder;
let mut generator = AesniRandomGenerator::new(seeder.seed());
c.bench_function("parent_generate", |b| {
b.iter(|| {
(0..N_GEN).for_each(|_| {
generator.next();
})
})
});
}
fn child_generate(c: &mut Criterion) {
let mut seeder = RdseedSeeder;
let mut generator = AesniRandomGenerator::new(seeder.seed());
let mut generator = generator
.try_fork(ChildrenCount(1), BytesPerChild(N_GEN * 10_000))
.unwrap()
.next()
.unwrap();
c.bench_function("child_generate", |b| {
b.iter(|| {
(0..N_GEN).for_each(|_| {
generator.next();
})
})
});
}
fn fork(c: &mut Criterion) {
let mut seeder = RdseedSeeder;
let mut generator = AesniRandomGenerator::new(seeder.seed());
c.bench_function("fork", |b| {
b.iter(|| {
black_box(
generator
.try_fork(ChildrenCount(2048), BytesPerChild(2048))
.unwrap(),
)
})
});
}
criterion_group!(benches, parent_generate, child_generate, fork);
criterion_main!(benches);

112
concrete-csprng/build.rs Normal file
View File

@@ -0,0 +1,112 @@
// To have clear error messages during compilation about why some piece of code may not be available
// we decided to check the features compatibility with the target configuration in this script.
use std::collections::HashMap;
use std::env;
// See https://doc.rust-lang.org/reference/conditional-compilation.html#target_arch for various
// compilation configuration
// Can be easily extended if needed
pub struct FeatureRequirement {
pub feature_name: &'static str,
// target_arch requirement
pub feature_req_target_arch: Option<&'static str>,
// target_family requirement
pub feature_req_target_family: Option<&'static str>,
}
// We implement a version of default that is const which is not possible through the Default trait
impl FeatureRequirement {
// As we cannot use cfg!(feature = "feature_name") with something else than a literal, we need
// a reference to the HashMap we populate with the enabled features
fn is_activated(&self, build_activated_features: &HashMap<&'static str, bool>) -> bool {
*build_activated_features.get(self.feature_name).unwrap()
}
// panics if the requirements are not met
fn check_requirements(&self) {
let target_arch = get_target_arch_cfg();
if let Some(feature_req_target_arch) = self.feature_req_target_arch {
if feature_req_target_arch != target_arch {
panic!(
"Feature `{}` requires target_arch `{}`, current cfg: `{}`",
self.feature_name, feature_req_target_arch, target_arch
)
}
}
let target_family = get_target_family_cfg();
if let Some(feature_req_target_family) = self.feature_req_target_family {
if feature_req_target_family != target_family {
panic!(
"Feature `{}` requires target_family `{}`, current cfg: `{}`",
self.feature_name, feature_req_target_family, target_family
)
}
}
}
}
// const vecs are not yet a thing so use a fixed size array (update the array size when adding
// requirements)
static FEATURE_REQUIREMENTS: [FeatureRequirement; 4] = [
FeatureRequirement {
feature_name: "seeder_x86_64_rdseed",
feature_req_target_arch: Some("x86_64"),
feature_req_target_family: None,
},
FeatureRequirement {
feature_name: "generator_x86_64_aesni",
feature_req_target_arch: Some("x86_64"),
feature_req_target_family: None,
},
FeatureRequirement {
feature_name: "seeder_unix",
feature_req_target_arch: None,
feature_req_target_family: Some("unix"),
},
FeatureRequirement {
feature_name: "generator_aarch64_aes",
feature_req_target_arch: Some("aarch64"),
feature_req_target_family: None,
},
];
// For a "feature_name" feature_cfg!("feature_name") expands to
// ("feature_name", cfg!(feature = "feature_name"))
macro_rules! feature_cfg {
($feat_name:literal) => {
($feat_name, cfg!(feature = $feat_name))
};
}
// Static HashMap would require an additional crate (phf or lazy static e.g.), so we just write a
// function that returns the HashMap we are interested in
fn get_feature_enabled_status() -> HashMap<&'static str, bool> {
HashMap::from([
feature_cfg!("seeder_x86_64_rdseed"),
feature_cfg!("generator_x86_64_aesni"),
feature_cfg!("seeder_unix"),
feature_cfg!("generator_aarch64_aes"),
])
}
// See https://stackoverflow.com/a/43435335/18088947 for the inspiration of this code
fn get_target_arch_cfg() -> String {
env::var("CARGO_CFG_TARGET_ARCH").expect("CARGO_CFG_TARGET_ARCH is not set")
}
fn get_target_family_cfg() -> String {
env::var("CARGO_CFG_TARGET_FAMILY").expect("CARGO_CFG_TARGET_FAMILY is not set")
}
fn main() {
let feature_enabled_status = get_feature_enabled_status();
// This will panic if some requirements for a feature are not met
FEATURE_REQUIREMENTS
.iter()
.filter(|&req| FeatureRequirement::is_activated(req, &feature_enabled_status))
.for_each(FeatureRequirement::check_requirements);
}

View File

@@ -0,0 +1,113 @@
//! This program uses the concrete csprng to generate an infinite stream of random bytes on
//! the program stdout. It can also generate a fixed number of bytes by passing a value along the
//! optional argument `--bytes_total`. For testing purpose.
use clap::{value_parser, Arg, Command};
#[cfg(feature = "generator_x86_64_aesni")]
use concrete_csprng::generators::AesniRandomGenerator as ActivatedRandomGenerator;
#[cfg(feature = "generator_aarch64_aes")]
use concrete_csprng::generators::NeonAesRandomGenerator as ActivatedRandomGenerator;
#[cfg(all(
not(feature = "generator_x86_64_aesni"),
not(feature = "generator_aarch64_aes"),
feature = "generator_fallback"
))]
use concrete_csprng::generators::SoftwareRandomGenerator as ActivatedRandomGenerator;
use concrete_csprng::generators::RandomGenerator;
#[cfg(target_os = "macos")]
use concrete_csprng::seeders::AppleSecureEnclaveSeeder as ActivatedSeeder;
#[cfg(all(not(target_os = "macos"), feature = "seeder_x86_64_rdseed"))]
use concrete_csprng::seeders::RdseedSeeder as ActivatedSeeder;
#[cfg(all(
not(target_os = "macos"),
not(feature = "seeder_x86_64_rdseed"),
feature = "seeder_unix"
))]
use concrete_csprng::seeders::UnixSeeder as ActivatedSeeder;
use concrete_csprng::seeders::Seeder;
use std::io::prelude::*;
use std::io::{stdout, StdoutLock};
fn write_bytes(
buffer: &mut [u8],
generator: &mut ActivatedRandomGenerator,
stdout: &mut StdoutLock<'_>,
) -> std::io::Result<()> {
buffer.iter_mut().zip(generator).for_each(|(b, g)| *b = g);
stdout.write_all(buffer)
}
fn infinite_bytes_generation(
buffer: &mut [u8],
generator: &mut ActivatedRandomGenerator,
stdout: &mut StdoutLock<'_>,
) {
while write_bytes(buffer, generator, stdout).is_ok() {}
}
fn bytes_generation(
bytes_total: usize,
buffer: &mut [u8],
generator: &mut ActivatedRandomGenerator,
stdout: &mut StdoutLock<'_>,
) {
let quotient = bytes_total / buffer.len();
let remaining = bytes_total % buffer.len();
for _ in 0..quotient {
write_bytes(buffer, generator, stdout).unwrap();
}
write_bytes(&mut buffer[0..remaining], generator, stdout).unwrap()
}
pub fn main() {
let matches = Command::new(
"Generate a stream of random numbers, specify no flags for infinite generation",
)
.arg(
Arg::new("bytes_total")
.short('b')
.long("bytes_total")
.value_parser(value_parser!(usize))
.help("Total number of bytes that has to be generated"),
)
.get_matches();
// Ugly hack to be able to use UnixSeeder
#[cfg(all(
not(target_os = "macos"),
not(feature = "seeder_x86_64_rdseed"),
feature = "seeder_unix"
))]
let new_seeder = || ActivatedSeeder::new(0);
#[cfg(not(all(
not(target_os = "macos"),
not(feature = "seeder_x86_64_rdseed"),
feature = "seeder_unix"
)))]
let new_seeder = || ActivatedSeeder;
let mut seeder = new_seeder();
let seed = seeder.seed();
// Don't print on std out
eprintln!("seed={seed:?}");
let mut generator = ActivatedRandomGenerator::new(seed);
let stdout = stdout();
let mut buffer = [0u8; 16];
// lock stdout as there is a single thread running
let mut stdout = stdout.lock();
match matches.get_one::<usize>("bytes_total") {
Some(&total) => {
bytes_generation(total, &mut buffer, &mut generator, &mut stdout);
}
None => {
infinite_bytes_generation(&mut buffer, &mut generator, &mut stdout);
}
};
}

View File

@@ -0,0 +1,20 @@
use crate::generators::aes_ctr::index::AesIndex;
use crate::generators::aes_ctr::BYTES_PER_BATCH;
/// Represents a key used in the AES block cipher.
#[derive(Clone, Copy)]
pub struct AesKey(pub u128);
/// A trait for AES block ciphers.
///
/// Note:
/// -----
///
/// The block cipher is used in a batched manner (to reduce amortized cost on special hardware).
/// For this reason we only expose a `generate_batch` method.
pub trait AesBlockCipher: Clone + Send + Sync {
/// Instantiate a new generator from a secret key.
fn new(key: AesKey) -> Self;
/// Generates the batch corresponding to the given index.
fn generate_batch(&mut self, index: AesIndex) -> [u8; BYTES_PER_BATCH];
}

View File

@@ -0,0 +1,379 @@
use crate::generators::aes_ctr::block_cipher::{AesBlockCipher, AesKey};
use crate::generators::aes_ctr::index::TableIndex;
use crate::generators::aes_ctr::states::{BufferPointer, ShiftAction, State};
use crate::generators::aes_ctr::BYTES_PER_BATCH;
use crate::generators::{ByteCount, BytesPerChild, ChildrenCount, ForkError};
// Usually, to work with iterators and parallel iterators, we would use opaque types such as
// `impl Iterator<..>`. Unfortunately, it is not yet possible to return existential types in
// traits, which we would need for `RandomGenerator`. For this reason, we have to use the
// full type name where needed. Hence the following trait aliases definition:
/// A type alias for the children iterator closure type.
pub type ChildrenClosure<BlockCipher> =
fn((usize, (Box<BlockCipher>, TableIndex, BytesPerChild))) -> AesCtrGenerator<BlockCipher>;
/// A type alias for the children iterator type.
pub type ChildrenIterator<BlockCipher> = std::iter::Map<
std::iter::Zip<
std::ops::Range<usize>,
std::iter::Repeat<(Box<BlockCipher>, TableIndex, BytesPerChild)>,
>,
ChildrenClosure<BlockCipher>,
>;
/// A type implementing the `RandomGenerator` api using the AES block cipher in counter mode.
#[derive(Clone)]
pub struct AesCtrGenerator<BlockCipher: AesBlockCipher> {
// The block cipher used in the background
pub(crate) block_cipher: Box<BlockCipher>,
// The state corresponding to the latest outputted byte.
pub(crate) state: State,
// The last legal index. This makes bound check faster.
pub(crate) last: TableIndex,
// The buffer containing the current batch of aes calls.
pub(crate) buffer: [u8; BYTES_PER_BATCH],
}
#[allow(unused)] // to please clippy when tests are not activated
impl<BlockCipher: AesBlockCipher> AesCtrGenerator<BlockCipher> {
/// Generates a new csprng.
///
/// Note :
/// ------
///
/// The `start_index` given as input, points to the first byte that will be outputted by the
/// generator. If not given, this one is automatically set to the second table index. The
/// first table index is not used to prevent an edge case from happening: since `state` is
/// supposed to contain the index of the previous byte, the initial value must be decremented.
/// Using the second value prevents wrapping to the max index, which would make the bound
/// checking fail.
///
/// The `bound_index` given as input, points to the first byte that can __not__ be legally
/// outputted by the generator. If not given, the bound is automatically set to the last
/// table index.
pub fn new(
key: AesKey,
start_index: Option<TableIndex>,
bound_index: Option<TableIndex>,
) -> AesCtrGenerator<BlockCipher> {
AesCtrGenerator::from_block_cipher(
Box::new(BlockCipher::new(key)),
start_index.unwrap_or(TableIndex::SECOND),
bound_index.unwrap_or(TableIndex::LAST),
)
}
/// Generates a csprng from an existing block cipher.
pub fn from_block_cipher(
block_cipher: Box<BlockCipher>,
start_index: TableIndex,
bound_index: TableIndex,
) -> AesCtrGenerator<BlockCipher> {
assert!(start_index < bound_index);
let last = bound_index.decremented();
let buffer = [0u8; BYTES_PER_BATCH];
let state = State::new(start_index);
AesCtrGenerator {
block_cipher,
state,
last,
buffer,
}
}
/// Returns the table index related to the previous random byte.
pub fn table_index(&self) -> TableIndex {
self.state.table_index()
}
/// Returns the bound of the generator if any.
///
/// The bound is the table index of the first byte that can not be outputted by the generator.
pub fn get_bound(&self) -> TableIndex {
self.last.incremented()
}
/// Returns whether the generator is bounded or not.
pub fn is_bounded(&self) -> bool {
self.get_bound() != TableIndex::LAST
}
/// Computes the number of bytes that can still be outputted by the generator.
///
/// Note :
/// ------
///
/// Note that `ByteCount` uses the `u128` datatype to store the byte count. Unfortunately, the
/// number of remaining bytes is in ⟦0;2¹³² -1⟧. When the number is greater than 2¹²⁸ - 1,
/// we saturate the count at 2¹²⁸ - 1.
pub fn remaining_bytes(&self) -> ByteCount {
TableIndex::distance(&self.last, &self.state.table_index()).unwrap()
}
/// Outputs the next random byte.
pub fn generate_next(&mut self) -> u8 {
self.next()
.expect("Tried to generate a byte after the bound.")
}
/// Tries to fork the current generator into `n_child` generators each able to output
/// `child_bytes` random bytes.
pub fn try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<ChildrenIterator<BlockCipher>, ForkError> {
if n_children.0 == 0 {
return Err(ForkError::ZeroChildrenCount);
}
if n_bytes.0 == 0 {
return Err(ForkError::ZeroBytesPerChild);
}
if !self.is_fork_in_bound(n_children, n_bytes) {
return Err(ForkError::ForkTooLarge);
}
// The state currently stored in the parent generator points to the table index of the last
// generated byte. The first index to be generated is the next one:
let first_index = self.state.table_index().incremented();
let output = (0..n_children.0)
.zip(std::iter::repeat((
self.block_cipher.clone(),
first_index,
n_bytes,
)))
.map(
// This map is a little weird because we need to cast the closure to a fn pointer
// that matches the signature of `ChildrenIterator<BlockCipher>`.
// Unfortunately, the compiler does not manage to coerce this one
// automatically.
(|(i, (block_cipher, first_index, n_bytes))| {
// The first index to be outputted by the child is the `first_index` shifted by
// the proper amount of `child_bytes`.
let child_first_index = first_index.increased(n_bytes.0 * i);
// The bound of the child is the first index of its next sibling.
let child_bound_index = first_index.increased(n_bytes.0 * (i + 1));
AesCtrGenerator::from_block_cipher(
block_cipher,
child_first_index,
child_bound_index,
)
}) as ChildrenClosure<BlockCipher>,
);
// The parent next index is the bound of the last child.
let next_index = first_index.increased(n_bytes.0 * n_children.0);
self.state = State::new(next_index);
Ok(output)
}
pub(crate) fn is_fork_in_bound(
&self,
n_child: ChildrenCount,
child_bytes: BytesPerChild,
) -> bool {
let mut end = self.state.table_index();
end.increase(n_child.0 * child_bytes.0);
end <= self.last
}
}
impl<BlockCipher: AesBlockCipher> Iterator for AesCtrGenerator<BlockCipher> {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
if self.state.table_index() >= self.last {
None
} else {
match self.state.increment() {
ShiftAction::OutputByte(BufferPointer(ptr)) => Some(self.buffer[ptr]),
ShiftAction::RefreshBatchAndOutputByte(aes_index, BufferPointer(ptr)) => {
self.buffer = self.block_cipher.generate_batch(aes_index);
Some(self.buffer[ptr])
}
}
}
}
}
#[cfg(test)]
pub mod aes_ctr_generic_test {
#![allow(unused)] // to please clippy when tests are not activated
use super::*;
use crate::generators::aes_ctr::index::{AesIndex, ByteIndex};
use crate::generators::aes_ctr::BYTES_PER_AES_CALL;
use rand::{thread_rng, Rng};
const REPEATS: usize = 1_000_000;
pub fn any_table_index() -> impl Iterator<Item = TableIndex> {
std::iter::repeat_with(|| {
TableIndex::new(
AesIndex(thread_rng().gen()),
ByteIndex(thread_rng().gen::<usize>() % BYTES_PER_AES_CALL),
)
})
}
pub fn any_usize() -> impl Iterator<Item = usize> {
std::iter::repeat_with(|| thread_rng().gen())
}
pub fn any_children_count() -> impl Iterator<Item = ChildrenCount> {
std::iter::repeat_with(|| ChildrenCount(thread_rng().gen::<usize>() % 2048 + 1))
}
pub fn any_bytes_per_child() -> impl Iterator<Item = BytesPerChild> {
std::iter::repeat_with(|| BytesPerChild(thread_rng().gen::<usize>() % 2048 + 1))
}
pub fn any_key() -> impl Iterator<Item = AesKey> {
std::iter::repeat_with(|| AesKey(thread_rng().gen()))
}
/// Output a valid fork:
/// a table index t,
/// a number of children nc,
/// a number of bytes per children nb
/// and a positive integer i such that:
/// increase(t, nc*nb+i) < MAX with MAX the largest table index.
///
/// Put differently, if we initialize a parent generator at t and fork it with (nc, nb), our
/// parent generator current index gets shifted to an index, distant of at least i bytes of
/// the max index.
pub fn any_valid_fork(
) -> impl Iterator<Item = (TableIndex, ChildrenCount, BytesPerChild, usize)> {
any_table_index()
.zip(any_children_count())
.zip(any_bytes_per_child())
.zip(any_usize())
.map(|(((t, nc), nb), i)| (t, nc, nb, i))
.filter(|(t, nc, nb, i)| {
TableIndex::distance(&TableIndex::LAST, t).unwrap().0 > (nc.0 * nb.0 + i) as u128
})
}
/// Check the property:
/// On a valid fork, the table index of the first child is the same as the table index of
/// the parent before the fork.
pub fn prop_fork_first_state_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
let first_child = forked_generator.try_fork(nc, nb).unwrap().next().unwrap();
assert_eq!(original_generator.table_index(), first_child.table_index());
}
}
/// Check the property:
/// On a valid fork, the table index of the first byte outputted by the parent after the
/// fork, is the bound of the last child of the fork.
pub fn prop_fork_last_bound_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let mut parent_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let last_child = parent_generator.try_fork(nc, nb).unwrap().last().unwrap();
assert_eq!(
parent_generator.table_index().incremented(),
last_child.get_bound()
);
}
}
/// Check the property:
/// On a valid fork, the bound of the parent does not change.
pub fn prop_fork_parent_bound_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
forked_generator.try_fork(nc, nb).unwrap().last().unwrap();
assert_eq!(original_generator.get_bound(), forked_generator.get_bound());
}
}
/// Check the property:
/// On a valid fork, the parent table index is increased of the number of children
/// multiplied by the number of bytes per child.
pub fn prop_fork_parent_state_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
forked_generator.try_fork(nc, nb).unwrap().last().unwrap();
assert_eq!(
forked_generator.table_index(),
// Decrement accounts for the fact that the table index stored is the previous one
t.increased(nc.0 * nb.0).decremented()
);
}
}
/// Check the property:
/// On a valid fork, the bytes outputted by the children in the fork order form the same
/// sequence the parent would have had yielded no fork had happened.
pub fn prop_fork<G: AesBlockCipher>() {
for _ in 0..1000 {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let bytes_to_go = nc.0 * nb.0;
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
let initial_output: Vec<u8> = original_generator.take(bytes_to_go).collect();
let forked_output: Vec<u8> = forked_generator
.try_fork(nc, nb)
.unwrap()
.flat_map(|child| child.collect::<Vec<_>>())
.collect();
assert_eq!(initial_output, forked_output);
}
}
/// Check the property:
/// On a valid fork, all children got a number of remaining bytes equals to the number of
/// bytes per child given as fork input.
pub fn prop_fork_children_remaining_bytes<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let mut generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
assert!(generator
.try_fork(nc, nb)
.unwrap()
.all(|c| c.remaining_bytes().0 == nb.0 as u128));
}
}
/// Check the property:
/// On a valid fork, the number of remaining bybtes of the parent is reduced by the number
/// of children multiplied by the number of bytes per child.
pub fn prop_fork_parent_remaining_bytes<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let bytes_to_go = nc.0 * nb.0;
let mut generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let before_remaining_bytes = generator.remaining_bytes();
let _ = generator.try_fork(nc, nb).unwrap();
let after_remaining_bytes = generator.remaining_bytes();
assert_eq!(
before_remaining_bytes.0 - after_remaining_bytes.0,
bytes_to_go as u128
);
}
}
}

View File

@@ -0,0 +1,389 @@
use crate::generators::aes_ctr::BYTES_PER_AES_CALL;
use crate::generators::ByteCount;
use std::cmp::Ordering;
/// A structure representing an [aes index](#coarse-grained-pseudo-random-table-lookup).
#[derive(Clone, Copy, Debug, PartialOrd, Ord, PartialEq, Eq)]
pub struct AesIndex(pub u128);
/// A structure representing a [byte index](#fine-grained-pseudo-random-table-lookup).
#[derive(Clone, Copy, Debug, PartialOrd, Ord, PartialEq, Eq)]
pub struct ByteIndex(pub usize);
/// A structure representing a [table index](#fine-grained-pseudo-random-table-lookup)
#[derive(Clone, Copy, Debug)]
pub struct TableIndex {
pub(crate) aes_index: AesIndex,
pub(crate) byte_index: ByteIndex,
}
impl TableIndex {
/// The first table index.
pub const FIRST: TableIndex = TableIndex {
aes_index: AesIndex(0),
byte_index: ByteIndex(0),
};
/// The second table index.
pub const SECOND: TableIndex = TableIndex {
aes_index: AesIndex(0),
byte_index: ByteIndex(1),
};
/// The last table index.
pub const LAST: TableIndex = TableIndex {
aes_index: AesIndex(u128::MAX),
byte_index: ByteIndex(BYTES_PER_AES_CALL - 1),
};
/// Creates a table index from an aes index and a byte index.
#[allow(unused)] // to please clippy when tests are not activated
pub fn new(aes_index: AesIndex, byte_index: ByteIndex) -> Self {
assert!(byte_index.0 < BYTES_PER_AES_CALL);
TableIndex {
aes_index,
byte_index,
}
}
/// Shifts the table index forward of `shift` bytes.
pub fn increase(&mut self, shift: usize) {
// Compute full shifts to avoid overflows
let full_aes_shifts = shift / BYTES_PER_AES_CALL;
let shift_remainder = shift % BYTES_PER_AES_CALL;
// Get the additional shift if any
let new_byte_index = self.byte_index.0 + shift_remainder;
let full_aes_shifts = full_aes_shifts + new_byte_index / BYTES_PER_AES_CALL;
// Store the reaminder in the byte index
self.byte_index.0 = new_byte_index % BYTES_PER_AES_CALL;
self.aes_index.0 = self.aes_index.0.wrapping_add(full_aes_shifts as u128);
}
/// Shifts the table index backward of `shift` bytes.
pub fn decrease(&mut self, shift: usize) {
let remainder = shift % BYTES_PER_AES_CALL;
if remainder <= self.byte_index.0 {
self.aes_index.0 = self
.aes_index
.0
.wrapping_sub((shift / BYTES_PER_AES_CALL) as u128);
self.byte_index.0 -= remainder;
} else {
self.aes_index.0 = self
.aes_index
.0
.wrapping_sub((shift / BYTES_PER_AES_CALL) as u128 + 1);
self.byte_index.0 += BYTES_PER_AES_CALL - remainder;
}
}
/// Shifts the table index forward of one byte.
pub fn increment(&mut self) {
self.increase(1)
}
/// Shifts the table index backward of one byte.
pub fn decrement(&mut self) {
self.decrease(1)
}
/// Returns the table index shifted forward by `shift` bytes.
pub fn increased(mut self, shift: usize) -> Self {
self.increase(shift);
self
}
/// Returns the table index shifted backward by `shift` bytes.
#[allow(unused)] // to please clippy when tests are not activated
pub fn decreased(mut self, shift: usize) -> Self {
self.decrease(shift);
self
}
/// Returns the table index to the next byte.
pub fn incremented(mut self) -> Self {
self.increment();
self
}
/// Returns the table index to the previous byte.
pub fn decremented(mut self) -> Self {
self.decrement();
self
}
/// Returns the distance between two table indices in bytes.
///
/// Note:
/// -----
///
/// This method assumes that the `larger` input is, well, larger than the `smaller` input. If
/// this is not the case, the method returns `None`. Also, note that `ByteCount` uses the
/// `u128` datatype to store the byte count. Unfortunately, the number of bytes between two
/// table indices is in ⟦0;2¹³² -1⟧. When the distance is greater than 2¹²⁸ - 1, we saturate
/// the count at 2¹²⁸ - 1.
pub fn distance(larger: &Self, smaller: &Self) -> Option<ByteCount> {
match std::cmp::Ord::cmp(larger, smaller) {
Ordering::Less => None,
Ordering::Equal => Some(ByteCount(0)),
Ordering::Greater => {
let mut result = larger.aes_index.0 - smaller.aes_index.0;
result = result.saturating_mul(BYTES_PER_AES_CALL as u128);
result = result.saturating_add(larger.byte_index.0 as u128);
result = result.saturating_sub(smaller.byte_index.0 as u128);
Some(ByteCount(result))
}
}
}
}
impl Eq for TableIndex {}
impl PartialEq<Self> for TableIndex {
fn eq(&self, other: &Self) -> bool {
matches!(self.partial_cmp(other), Some(Ordering::Equal))
}
}
impl PartialOrd<Self> for TableIndex {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Ord for TableIndex {
fn cmp(&self, other: &Self) -> Ordering {
match self.aes_index.cmp(&other.aes_index) {
Ordering::Equal => self.byte_index.cmp(&other.byte_index),
other => other,
}
}
}
#[cfg(test)]
mod test {
use super::*;
use rand::{thread_rng, Rng};
const REPEATS: usize = 1_000_000;
fn any_table_index() -> impl Iterator<Item = TableIndex> {
std::iter::repeat_with(|| {
TableIndex::new(
AesIndex(thread_rng().gen()),
ByteIndex(thread_rng().gen::<usize>() % BYTES_PER_AES_CALL),
)
})
}
fn any_usize() -> impl Iterator<Item = usize> {
std::iter::repeat_with(|| thread_rng().gen())
}
#[test]
#[should_panic]
/// Verifies that the constructor of `TableIndex` panics when the byte index is too large.
fn test_table_index_new_panic() {
TableIndex::new(AesIndex(12), ByteIndex(144));
}
#[test]
/// Verifies that the `TableIndex` wraps nicely with predecessor
fn test_table_index_predecessor_edge() {
assert_eq!(TableIndex::FIRST.decremented(), TableIndex::LAST);
}
#[test]
/// Verifies that the `TableIndex` wraps nicely with successor
fn test_table_index_successor_edge() {
assert_eq!(TableIndex::LAST.incremented(), TableIndex::FIRST);
}
#[test]
/// Check that the table index distance saturates nicely.
fn prop_table_index_distance_saturates() {
assert_eq!(
TableIndex::distance(&TableIndex::LAST, &TableIndex::FIRST)
.unwrap()
.0,
u128::MAX
)
}
#[test]
/// Check the property:
/// For all table indices t,
/// distance(t, t) = Some(0).
fn prop_table_index_distance_zero() {
for _ in 0..REPEATS {
let t = any_table_index().next().unwrap();
assert_eq!(TableIndex::distance(&t, &t), Some(ByteCount(0)));
}
}
#[test]
/// Check the property:
/// For all table indices t1, t2 such that t1 < t2,
/// distance(t1, t2) = None.
fn prop_table_index_distance_wrong_order_none() {
for _ in 0..REPEATS {
let (t1, t2) = any_table_index()
.zip(any_table_index())
.find(|(t1, t2)| t1 < t2)
.unwrap();
assert_eq!(TableIndex::distance(&t1, &t2), None);
}
}
#[test]
/// Check the property:
/// For all table indices t1, t2 such that t1 > t2,
/// distance(t1, t2) = Some(v) where v is strictly positive.
fn prop_table_index_distance_some_positive() {
for _ in 0..REPEATS {
let (t1, t2) = any_table_index()
.zip(any_table_index())
.find(|(t1, t2)| t1 > t2)
.unwrap();
assert!(matches!(TableIndex::distance(&t1, &t2), Some(ByteCount(v)) if v > 0));
}
}
#[test]
/// Check the property:
/// For all table indices t, positive i such that i < distance (MAX, t) with MAX the largest
/// table index,
/// distance(t.increased(i), t) = Some(i).
fn prop_table_index_distance_increase() {
for _ in 0..REPEATS {
let (t, inc) = any_table_index()
.zip(any_usize())
.find(|(t, inc)| {
(*inc as u128) < TableIndex::distance(&TableIndex::LAST, t).unwrap().0
})
.unwrap();
assert_eq!(
TableIndex::distance(&t.increased(inc), &t).unwrap().0 as usize,
inc
);
}
}
#[test]
/// Check the property:
/// For all table indices t, t =? t = true.
fn prop_table_index_equality() {
for _ in 0..REPEATS {
let t = any_table_index().next().unwrap();
assert_eq!(
std::cmp::PartialOrd::partial_cmp(&t, &t),
Some(std::cmp::Ordering::Equal)
);
}
}
#[test]
/// Check the property:
/// For all table indices t, positive i such that i < distance (MAX, t) with MAX the largest
/// table index,
/// t.increased(i) >? t = true.
fn prop_table_index_greater() {
for _ in 0..REPEATS {
let (t, inc) = any_table_index()
.zip(any_usize())
.find(|(t, inc)| {
(*inc as u128) < TableIndex::distance(&TableIndex::LAST, t).unwrap().0
})
.unwrap();
assert_eq!(
std::cmp::PartialOrd::partial_cmp(&t.increased(inc), &t),
Some(std::cmp::Ordering::Greater),
);
}
}
#[test]
/// Check the property:
/// For all table indices t, positive i such that i < distance (t, 0) with MAX the largest
/// table index,
/// t.decreased(i) <? t = true.
fn prop_table_index_less() {
for _ in 0..REPEATS {
let (t, inc) = any_table_index()
.zip(any_usize())
.find(|(t, inc)| {
(*inc as u128) < TableIndex::distance(t, &TableIndex::FIRST).unwrap().0
})
.unwrap();
assert_eq!(
std::cmp::PartialOrd::partial_cmp(&t.decreased(inc), &t),
Some(std::cmp::Ordering::Less)
);
}
}
#[test]
/// Check the property:
/// For all table indices t,
/// successor(predecessor(t)) = t.
fn prop_table_index_decrement_increment() {
for _ in 0..REPEATS {
let t = any_table_index().next().unwrap();
assert_eq!(t.decremented().incremented(), t);
}
}
#[test]
/// Check the property:
/// For all table indices t,
/// predecessor(successor(t)) = t.
fn prop_table_index_increment_decrement() {
for _ in 0..REPEATS {
let t = any_table_index().next().unwrap();
assert_eq!(t.incremented().decremented(), t);
}
}
#[test]
/// Check the property:
/// For all table indices t, positive integer i,
/// increase(decrease(t, i), i) = t.
fn prop_table_index_increase_decrease() {
for _ in 0..REPEATS {
let (t, i) = any_table_index().zip(any_usize()).next().unwrap();
assert_eq!(t.increased(i).decreased(i), t);
}
}
#[test]
/// Check the property:
/// For all table indices t, positive integer i,
/// decrease(increase(t, i), i) = t.
fn prop_table_index_decrease_increase() {
for _ in 0..REPEATS {
let (t, i) = any_table_index().zip(any_usize()).next().unwrap();
assert_eq!(t.decreased(i).increased(i), t);
}
}
#[test]
/// Check that a big increase does not overflow
fn prop_table_increase_max_no_overflow() {
let first = TableIndex::FIRST;
// Increase so that ByteIndex is at 1usize
let second = first.increased(1);
// Now increase by usize::MAX, as the underlying byte index stores a usize this may overflow
// depending on implementation, ensure it does not overflow
let big_increase = second.increased(usize::MAX);
let total_full_aes_shifts = (1u128 + usize::MAX as u128) / BYTES_PER_AES_CALL as u128;
assert_eq!(
big_increase,
TableIndex::new(AesIndex(total_full_aes_shifts), ByteIndex(0))
);
}
}

View File

@@ -0,0 +1,223 @@
//! A module implementing the random generator api with batched aes calls.
//!
//! This module provides a generic [`AesCtrGenerator`] structure which implements the
//! [`super::RandomGenerator`] api using the AES block cipher in counter mode. That is, the
//! generator holds a state (i.e. counter) which is incremented iteratively, to produce the stream
//! of random values:
//! ```ascii
//! state=0 state=1 state=2
//! ╔══↧══╗ ╔══↧══╗ ╔══↧══╗
//! key ↦ AES ║ key ↦ AES ║ key ↦ AES ║ ...
//! ╚══↧══╝ ╚══↧══╝ ╚══↧══╝
//! output0 output1 output2
//!
//! t=0 t=1 t=2
//! ```
//!
//! The [`AesCtrGenerator`] structure is generic over the AES block ciphers, which are
//! represented by the [`AesBlockCipher`] trait. Consequently, implementers only need to implement
//! the `AesBlockCipher` trait, to benefit from the whole api of the `AesCtrGenerator` structure.
//!
//! In the following section, we give details on the implementation of this generic generator.
//!
//! Coarse-grained pseudo-random lookup table
//! =========================================
//!
//! To generate random values, we use the AES block cipher in counter mode. If we denote f the aes
//! encryption function, we have:
//! ```ascii
//! f: ⟦0;2¹²⁸ -1⟧ X ⟦0;2¹²⁸ -1⟧ ↦ ⟦0;2¹²⁸ -1⟧
//! f(secret_key, input) ↦ output
//! ```
//! If we fix the secret key to a value k, we have a function fₖ from ⟦0;2¹²⁸ -1⟧ to ⟦0;2¹²⁸-1⟧,
//! transforming the state of the counter into a pseudo random value. Essentially, this fₖ
//! function can be considered as a the following lookup table, containing 2¹²⁸ pseudo-random
//! values:
//! ```ascii
//! ╭──────────────┬──────────────┬─────┬──────────────╮
//! │ 0 │ 1 │ │ 2¹²⁸ -1 │
//! ├──────────────┼──────────────┼─────┼──────────────┤
//! │ fₖ(0) │ fₖ(1) │ │ fₖ(2¹²⁸ -1) │
//! ╔═══════↧══════╦═══════↧══════╦═════╦═══════↧══════╗
//! ║┏━━━━━━━━━━━━┓║┏━━━━━━━━━━━━┓║ ║┏━━━━━━━━━━━━┓║
//! ║┃ u128 ┃║┃ u128 ┃║ ... ║┃ u128 ┃║
//! ║┗━━━━━━━━━━━━┛║┗━━━━━━━━━━━━┛║ ║┗━━━━━━━━━━━━┛║
//! ╚══════════════╩══════════════╩═════╩══════════════╝
//! ```
//!
//! An input to the fₖ function is called an _aes index_ (also called state or counter in the
//! standards) of the pseudo-random table. The [`AesIndex`] structure defined in this module
//! represents such an index in the code.
//!
//! Fine-grained pseudo-random table lookup
//! =======================================
//!
//! Since we want to deliver the pseudo-random bytes one by one, we have to come with a finer
//! grained indexing. Fortunately, each `u128` value outputted by fₖ can be seen as a table of 16
//! `u8`:
//! ```ascii
//! ╭──────────────┬──────────────┬─────┬──────────────╮
//! │ 0 │ 1 │ │ 2¹²⁸ -1 │
//! ├──────────────┼──────────────┼─────┼──────────────┤
//! │ fₖ(0) │ fₖ(1) │ │ fₖ(2¹²⁸ -1) │
//! ╔═══════↧══════╦═══════↧══════╦═════╦═══════↧══════╗
//! ║┏━━━━━━━━━━━━┓║┏━━━━━━━━━━━━┓║ ║┏━━━━━━━━━━━━┓║
//! ║┃ u128 ┃║┃ u128 ┃║ ║┃ u128 ┃║
//! ║┣━━┯━━┯━━━┯━━┫║┣━━┯━━┯━━━┯━━┫║ ... ║┣━━┯━━┯━━━┯━━┫║
//! ║┃u8│u8│...│u8┃║┃u8│u8│...│u8┃║ ║┃u8│u8│...│u8┃║
//! ║┗━━┷━━┷━━━┷━━┛║┗━━┷━━┷━━━┷━━┛║ ║┗━━┷━━┷━━━┷━━┛║
//! ╚══════════════╩══════════════╩═════╩══════════════╝
//! ```
//!
//! We introduce a second function to select a chunk of 8 bits:
//! ```ascii
//! g: ⟦0;2¹²⁸ -1⟧ X ⟦0;15⟧ ↦ ⟦0;2⁸ -1⟧
//! g(big_int, index) ↦ byte
//! ```
//!
//! If we fix the `u128` value to a value e, we have a function gₑ from ⟦0;15⟧ to ⟦0;2⁸ -1⟧
//! transforming an index into a pseudo-random byte:
//! ```ascii
//! ┏━━━━━━━━┯━━━━━━━━┯━━━┯━━━━━━━━┓
//! ┃ u8 │ u8 │...│ u8 ┃
//! ┗━━━━━━━━┷━━━━━━━━┷━━━┷━━━━━━━━┛
//! │ gₑ(0) │ gₑ(1) │ │ gₑ(15) │
//! ╰────────┴─────-──┴───┴────────╯
//! ```
//!
//! We call this input to the gₑ function, a _byte index_ of the pseudo-random table. The
//! [`ByteIndex`] structure defined in this module represents such an index in the code.
//!
//! By using both the g and the fₖ functions, we can define a new function l which allows to index
//! any byte of the pseudo-random table:
//! ```ascii
//! l: ⟦0;2¹²⁸ -1⟧ X ⟦0;15⟧ ↦ ⟦0;2⁸ -1⟧
//! l(aes_index, byte_index) ↦ g(fₖ(aes_index), byte_index)
//! ```
//!
//! In this sense, any member of ⟦0;2¹²⁸ -1⟧ X ⟦0;15⟧ uniquely defines a byte in this pseudo-random
//! table:
//! ```ascii
//! e = fₖ(a)
//! ╔══════════════╦═══════↧══════╦═════╦══════════════╗
//! ║┏━━━━━━━━━━━━┓║┏━━━━━━━━━━━━┓║ ║┏━━━━━━━━━━━━┓║
//! ║┃ u128 ┃║┃ u128 ┃║ ║┃ u128 ┃║
//! ║┣━━┯━━┯━━━┯━━┫║┣━━┯━━┯━━━┯━━┫║ ... ║┣━━┯━━┯━━━┯━━┫║
//! ║┃u8│u8│...│u8┃║┃u8│u8│...│u8┃║ ║┃u8│u8│...│u8┃║
//! ║┗━━┷━━┷━━━┷━━┛║┗━━┷↥━┷━━━┷━━┛║ ║┗━━┷━━┷━━━┷━━┛║
//! ║ ║│ gₑ(b) │║ ║ ║
//! ║ ║╰───-────────╯║ ║ ║
//! ╚══════════════╩══════════════╩═════╩══════════════╝
//! ```
//!
//! We call this input to the l function, a _table index_ of the pseudo-random table. The
//! [`TableIndex`] structure defined in this module represents such an index in the code.
//!
//! Prngs current table index
//! =========================
//!
//! When created, a prng is given an initial _table index_, denoted (a₀, b₀), which identifies the
//! first byte of the table to be outputted by the prng. Then, each time the prng is queried for a
//! new value, the byte corresponding to the current _table index_ is returned, and the current
//! _table index_ is incremented:
//! ```ascii
//! e = fₖ(a₀) e = fₖ(a₁)
//! ╔═════↧═════╦═══════════╦═════╦═══════════╗ ╔═══════════╦═════↧═════╦═════╦═══════════╗
//! ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║┏━┯━┯━━━┯━┓║ ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║┏━┯━┯━━━┯━┓║
//! ║┃ │ │...│ ┃║┃ │ │...│ ┃║ ║┃ │ │...│ ┃║ ║┃ │ │...│ ┃║┃ │ │...│ ┃║ ║┃ │ │...│ ┃║
//! ║┗━┷━┷━━━┷↥┛║┗━┷━┷━━━┷━┛║ ║┗━┷━┷━━━┷━┛║ → ║┗━┷━┷━━━┷━┛║┗↥┷━┷━━━┷━┛║ ║┗━┷━┷━━━┷━┛║
//! ║│ gₑ(b₀) │║ ║ ║ ║ ║ ║│ gₑ(b₁) │║ ║ ║
//! ║╰─────────╯║ ║ ║ ║ ║ ║╰─────────╯║ ║ ║
//! ╚═══════════╩═══════════╩═════╩═══════════╝ ╚═══════════╩═══════════╩═════╩═══════════╝
//! ```
//!
//! Prng bound
//! ==========
//!
//! When created, a prng is also given a _bound_ (aₘ, bₘ) , that is a table index which it is not
//! allowed to exceed:
//! ```ascii
//! e = fₖ(a₀)
//! ╔═════↧═════╦═══════════╦═════╦═══════════╗
//! ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║┏━┯━┯━━━┯━┓║
//! ║┃ │ │...│ ┃║┃ │╳│...│╳┃║ ║┃╳│╳│...│╳┃║
//! ║┗━┷━┷━━━┷↥┛║┗━┷━┷━━━┷━┛║ ║┗━┷━┷━━━┷━┛║ The current byte can be returned.
//! ║│ gₑ(b₀) │║ ║ ║ ║
//! ║╰─────────╯║ ║ ║ ║
//! ╚═══════════╩═══════════╩═════╩═══════════╝
//!
//! e = fₖ(aₘ)
//! ╔═══════════╦═════↧═════╦═════╦═══════════╗
//! ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║┏━┯━┯━━━┯━┓║
//! ║┃ │ │...│ ┃║┃ │╳│...│╳┃║ ║┃╳│╳│...│╳┃║ The table index reached the bound,
//! ║┗━┷━┷━━━┷━┛║┗━┷↥┷━━━┷━┛║ ║┗━┷━┷━━━┷━┛║ the current byte can not be
//! ║ ║│ gₑ(bₘ) │║ ║ ║ returned.
//! ║ ║╰─────────╯║ ║ ║
//! ╚═══════════╩═══════════╩═════╩═══════════╝
//! ```
//!
//! Buffering
//! =========
//!
//! Calling the aes function every time we need to output a single byte would be a huge waste of
//! resources. In practice, we call aes 8 times in a row, for 8 successive values of aes index, and
//! store the results in a buffer. For platforms which have a dedicated aes chip, this allows to
//! fill the unit pipeline and reduces the amortized cost of the aes function.
//!
//! Together with the current table index of the prng, we also store a pointer p (initialized at
//! p₀=b₀) to the current byte in the buffer. If we denote v the lookup function we have :
//! ```ascii
//! e = fₖ(a₀) Buffer(length=128)
//! ╔═════╦═══════════╦═════↧═════╦═══════════╦═════╗ ┏━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓
//! ║ ... ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║ ┃▓│▓│▓│▓│▓│▓│▓│▓│...│▓┃
//! ║ ║┃ │ │...│ ┃║┃▓│▓│...│▓┃║┃▓│▓│...│▓┃║ ║ ┗━┷↥┷━┷━┷━┷━┷━┷━┷━━━┷━┛
//! ║ ║┗━┷━┷━━━┷━┛║┗━┷↥┷━━━┷━┛║┗━┷━┷━━━┷━┛║ ║ │ v(p₀) │
//! ║ ║ ║│ gₑ(b₀) │║ ║ ║ ╰─────────────────────╯
//! ║ ║ ║╰─────────╯║ ║ ║
//! ╚═════╩═══════════╩═══════════╩═══════════╩═════╝
//! ```
//!
//! We call this input to the v function, a _buffer pointer_. The [`BufferPointer`] structure
//! defined in this module represents such a pointer in the code.
//!
//! When the table index is incremented, the buffer pointer is incremented alongside:
//! ```ascii
//! e = fₖ(a) Buffer(length=128)
//! ╔═════╦═══════════╦═════↧═════╦═══════════╦═════╗ ┏━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓
//! ║ ... ║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║┏━┯━┯━━━┯━┓║ ... ║ ┃▓│▓│▓│▓│▓│▓│▓│▓│...│▓┃
//! ║ ║┃ │ │...│ ┃║┃▓│▓│...│▓┃║┃▓│▓│...│▓┃║ ║ ┗━┷━┷↥┷━┷━┷━┷━┷━┷━━━┷━┛
//! ║ ║┗━┷━┷━━━┷━┛║┗━┷━┷↥━━┷━┛║┗━┷━┷━━━┷━┛║ ║ │ v(p) │
//! ║ ║ ║│ gₑ(b) │║ ║ ║ ╰─────────────────────╯
//! ║ ║ ║╰─────────╯║ ║ ║
//! ╚═════╩═══════════╩═══════════╩═══════════╩═════╝
//! ```
//!
//! When the buffer pointer is incremented it is checked against the size of the buffer, and if
//! necessary, a new batch of aes index values is generated.
pub const AES_CALLS_PER_BATCH: usize = 8;
pub const BYTES_PER_AES_CALL: usize = 128 / 8;
pub const BYTES_PER_BATCH: usize = BYTES_PER_AES_CALL * AES_CALLS_PER_BATCH;
/// A module containing structures to manage table indices.
mod index;
pub use index::*;
/// A module containing structures to manage table indices and buffer pointers together properly.
mod states;
pub use states::*;
/// A module containing an abstraction for aes block ciphers.
mod block_cipher;
pub use block_cipher::*;
/// A module containing a generic implementation of a random generator.
mod generic;
pub use generic::*;
/// A module extending `generic` to the `rayon` paradigm.
#[cfg(feature = "parallel")]
mod parallel;
#[cfg(feature = "parallel")]
pub use parallel::*;

View File

@@ -0,0 +1,222 @@
use crate::generators::aes_ctr::{
AesBlockCipher, AesCtrGenerator, ChildrenClosure, State, TableIndex,
};
use crate::generators::{BytesPerChild, ChildrenCount, ForkError};
/// A type alias for the parallel children iterator type.
pub type ParallelChildrenIterator<BlockCipher> = rayon::iter::Map<
rayon::iter::Zip<
rayon::range::Iter<usize>,
rayon::iter::RepeatN<(Box<BlockCipher>, TableIndex, BytesPerChild)>,
>,
fn((usize, (Box<BlockCipher>, TableIndex, BytesPerChild))) -> AesCtrGenerator<BlockCipher>,
>;
impl<BlockCipher: AesBlockCipher> AesCtrGenerator<BlockCipher> {
/// Tries to fork the current generator into `n_child` generators each able to output
/// `child_bytes` random bytes as a parallel iterator.
///
/// # Notes
///
/// This method necessitate the "multithread" feature.
pub fn par_try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<ParallelChildrenIterator<BlockCipher>, ForkError>
where
BlockCipher: Send + Sync,
{
use rayon::prelude::*;
if n_children.0 == 0 {
return Err(ForkError::ZeroChildrenCount);
}
if n_bytes.0 == 0 {
return Err(ForkError::ZeroBytesPerChild);
}
if !self.is_fork_in_bound(n_children, n_bytes) {
return Err(ForkError::ForkTooLarge);
}
// The state currently stored in the parent generator points to the table index of the last
// generated byte. The first index to be generated is the next one :
let first_index = self.state.table_index().incremented();
let output = (0..n_children.0)
.into_par_iter()
.zip(rayon::iter::repeatn(
(self.block_cipher.clone(), first_index, n_bytes),
n_children.0,
))
.map(
// This map is a little weird because we need to cast the closure to a fn pointer
// that matches the signature of `ChildrenIterator<BlockCipher>`. Unfortunately,
// the compiler does not manage to coerce this one automatically.
(|(i, (block_cipher, first_index, n_bytes))| {
// The first index to be outputted by the child is the `first_index` shifted by
// the proper amount of `child_bytes`.
let child_first_index = first_index.increased(n_bytes.0 * i);
// The bound of the child is the first index of its next sibling.
let child_bound_index = first_index.increased(n_bytes.0 * (i + 1));
AesCtrGenerator::from_block_cipher(
block_cipher,
child_first_index,
child_bound_index,
)
}) as ChildrenClosure<BlockCipher>,
);
// The parent next index is the bound of the last child.
let next_index = first_index.increased(n_bytes.0 * n_children.0);
self.state = State::new(next_index);
Ok(output)
}
}
#[cfg(test)]
pub mod aes_ctr_parallel_generic_tests {
use super::*;
use crate::generators::aes_ctr::aes_ctr_generic_test::{any_key, any_valid_fork};
use rayon::prelude::*;
const REPEATS: usize = 1_000_000;
/// Check the property:
/// On a valid fork, the table index of the first child is the same as the table index of
/// the parent before the fork.
pub fn prop_fork_first_state_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
let first_child = forked_generator
.par_try_fork(nc, nb)
.unwrap()
.find_first(|_| true)
.unwrap();
assert_eq!(original_generator.table_index(), first_child.table_index());
}
}
/// Check the property:
/// On a valid fork, the table index of the first byte outputted by the parent after the
/// fork, is the bound of the last child of the fork.
pub fn prop_fork_last_bound_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let mut parent_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let last_child = parent_generator
.par_try_fork(nc, nb)
.unwrap()
.find_last(|_| true)
.unwrap();
assert_eq!(
parent_generator.table_index().incremented(),
last_child.get_bound()
);
}
}
/// Check the property:
/// On a valid fork, the bound of the parent does not change.
pub fn prop_fork_parent_bound_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
forked_generator
.par_try_fork(nc, nb)
.unwrap()
.find_last(|_| true)
.unwrap();
assert_eq!(original_generator.get_bound(), forked_generator.get_bound());
}
}
/// Check the property:
/// On a valid fork, the parent table index is increased of the number of children
/// multiplied by the number of bytes per child.
pub fn prop_fork_parent_state_table_index<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
forked_generator
.par_try_fork(nc, nb)
.unwrap()
.find_last(|_| true)
.unwrap();
assert_eq!(
forked_generator.table_index(),
// Decrement accounts for the fact that the table index stored is the previous one
t.increased(nc.0 * nb.0).decremented()
);
}
}
/// Check the property:
/// On a valid fork, the bytes outputted by the children in the fork order form the same
/// sequence the parent would have had outputted no fork had happened.
pub fn prop_fork<G: AesBlockCipher>() {
for _ in 0..1000 {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let bytes_to_go = nc.0 * nb.0;
let original_generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let mut forked_generator = original_generator.clone();
let initial_output: Vec<u8> = original_generator.take(bytes_to_go).collect();
let forked_output: Vec<u8> = forked_generator
.par_try_fork(nc, nb)
.unwrap()
.flat_map(|child| child.collect::<Vec<_>>())
.collect();
assert_eq!(initial_output, forked_output);
}
}
/// Check the property:
/// On a valid fork, all children got a number of remaining bytes equals to the number of
/// bytes per child given as fork input.
pub fn prop_fork_children_remaining_bytes<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let mut generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
assert!(generator
.par_try_fork(nc, nb)
.unwrap()
.all(|c| c.remaining_bytes().0 == nb.0 as u128));
}
}
/// Check the property:
/// On a valid fork, the number of remaining bytes of the parent is reduced by the
/// number of children multiplied by the number of bytes per child.
pub fn prop_fork_parent_remaining_bytes<G: AesBlockCipher>() {
for _ in 0..REPEATS {
let (t, nc, nb, i) = any_valid_fork().next().unwrap();
let k = any_key().next().unwrap();
let bytes_to_go = nc.0 * nb.0;
let mut generator =
AesCtrGenerator::<G>::new(k, Some(t), Some(t.increased(nc.0 * nb.0 + i)));
let before_remaining_bytes = generator.remaining_bytes();
let _ = generator.par_try_fork(nc, nb).unwrap();
let after_remaining_bytes = generator.remaining_bytes();
assert_eq!(
before_remaining_bytes.0 - after_remaining_bytes.0,
bytes_to_go as u128
);
}
}
}

View File

@@ -0,0 +1,176 @@
use crate::generators::aes_ctr::index::{AesIndex, TableIndex};
use crate::generators::aes_ctr::BYTES_PER_BATCH;
/// A pointer to the next byte to be outputted by the generator.
#[derive(Clone, Copy, Debug, PartialOrd, Ord, PartialEq, Eq)]
pub struct BufferPointer(pub usize);
/// A structure representing the current state of generator using batched aes-ctr approach.
#[derive(Debug, Clone, Copy)]
pub struct State {
table_index: TableIndex,
buffer_pointer: BufferPointer,
}
/// A structure representing the action to be taken by the generator after shifting its state.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum ShiftAction {
/// Outputs the byte pointed to by the 0-th field.
OutputByte(BufferPointer),
/// Refresh the buffer starting from the 0-th field, and output the byte pointed to by the 0-th
/// field.
RefreshBatchAndOutputByte(AesIndex, BufferPointer),
}
impl State {
/// Creates a new state from the initial table index.
///
/// Note :
/// ------
///
/// The `table_index` input, is the __first__ table index that will be outputted on the next
/// call to `increment`. Put differently, the current table index of the newly created state
/// is the predecessor of this one.
pub fn new(table_index: TableIndex) -> Self {
// We ensure that the table index is not the first one, to prevent wrapping on `decrement`,
// and outputting `RefreshBatchAndOutputByte(AesIndex::MAX, ...)` on the first increment
// (which would lead to loading a non continuous batch).
assert_ne!(table_index, TableIndex::FIRST);
State {
// To ensure that the first outputted table index is the proper one, we decrement the
// table index.
table_index: table_index.decremented(),
// To ensure that the first `ShiftAction` will be a `RefreshBatchAndOutputByte`, we set
// the buffer to the last allowed value.
buffer_pointer: BufferPointer(BYTES_PER_BATCH - 1),
}
}
/// Shifts the state forward of `shift` bytes.
pub fn increase(&mut self, shift: usize) -> ShiftAction {
self.table_index.increase(shift);
let total_batch_index = self.buffer_pointer.0 + shift;
if total_batch_index > BYTES_PER_BATCH - 1 {
self.buffer_pointer.0 = self.table_index.byte_index.0;
ShiftAction::RefreshBatchAndOutputByte(self.table_index.aes_index, self.buffer_pointer)
} else {
self.buffer_pointer.0 = total_batch_index;
ShiftAction::OutputByte(self.buffer_pointer)
}
}
/// Shifts the state forward of one byte.
pub fn increment(&mut self) -> ShiftAction {
self.increase(1)
}
/// Returns the current table index.
pub fn table_index(&self) -> TableIndex {
self.table_index
}
}
impl Default for State {
fn default() -> Self {
State::new(TableIndex::FIRST)
}
}
#[cfg(test)]
mod test {
use super::*;
use crate::generators::aes_ctr::index::ByteIndex;
use crate::generators::aes_ctr::BYTES_PER_AES_CALL;
use rand::{thread_rng, Rng};
const REPEATS: usize = 1_000_000;
fn any_table_index() -> impl Iterator<Item = TableIndex> {
std::iter::repeat_with(|| {
TableIndex::new(
AesIndex(thread_rng().gen()),
ByteIndex(thread_rng().gen::<usize>() % BYTES_PER_AES_CALL),
)
})
}
fn any_usize() -> impl Iterator<Item = usize> {
std::iter::repeat_with(|| thread_rng().gen())
}
#[test]
/// Check the property:
/// For all table indices t,
/// State::new(t).increment() = RefreshBatchAndOutputByte(t.aes_index, t.byte_index)
fn prop_state_new_increment() {
for _ in 0..REPEATS {
let (t, mut s) = any_table_index()
.map(|t| (t, State::new(t)))
.next()
.unwrap();
assert!(matches!(
s.increment(),
ShiftAction::RefreshBatchAndOutputByte(t_, BufferPointer(p_)) if t_ == t.aes_index && p_ == t.byte_index.0
))
}
}
#[test]
/// Check the property:
/// For all states s, table indices t, positive integer i
/// if s = State::new(t), then t.increased(i) = s.increased(i-1).table_index().
fn prop_state_increase_table_index() {
for _ in 0..REPEATS {
let (t, mut s, i) = any_table_index()
.zip(any_usize())
.map(|(t, i)| (t, State::new(t), i))
.next()
.unwrap();
s.increase(i);
assert_eq!(s.table_index(), t.increased(i - 1))
}
}
#[test]
/// Check the property:
/// For all table indices t, positive integer i such as t.byte_index + i < 127,
/// if s = State::new(t), and s.increment() was executed, then
/// s.increase(i) = OutputByte(t.byte_index + i).
fn prop_state_increase_small() {
for _ in 0..REPEATS {
let (t, mut s, i) = any_table_index()
.zip(any_usize())
.map(|(t, i)| (t, State::new(t), i % BYTES_PER_BATCH))
.find(|(t, _, i)| t.byte_index.0 + i < BYTES_PER_BATCH - 1)
.unwrap();
s.increment();
assert!(matches!(
s.increase(i),
ShiftAction::OutputByte(BufferPointer(p_)) if p_ == t.byte_index.0 + i
));
}
}
#[test]
/// Check the property:
/// For all table indices t, positive integer i such as t.byte_index + i >= 127,
/// if s = State::new(t), and s.increment() was executed, then
/// s.increase(i) = RefreshBatchAndOutputByte(
/// t.increased(i).aes_index,
/// t.increased(i).byte_index).
fn prop_state_increase_large() {
for _ in 0..REPEATS {
let (t, mut s, i) = any_table_index()
.zip(any_usize())
.map(|(t, i)| (t, State::new(t), i))
.find(|(t, _, i)| t.byte_index.0 + i >= BYTES_PER_BATCH - 1)
.unwrap();
s.increment();
assert!(matches!(
s.increase(i),
ShiftAction::RefreshBatchAndOutputByte(t_, BufferPointer(p_))
if t_ == t.increased(i).aes_index && p_ == t.increased(i).byte_index.0
));
}
}
}

View File

@@ -0,0 +1,184 @@
use crate::generators::aes_ctr::{AesBlockCipher, AesIndex, AesKey, BYTES_PER_BATCH};
use core::arch::aarch64::{
uint8x16_t, vaeseq_u8, vaesmcq_u8, vdupq_n_u32, vdupq_n_u8, veorq_u8, vgetq_lane_u32,
vreinterpretq_u32_u8, vreinterpretq_u8_u32,
};
use std::arch::is_aarch64_feature_detected;
use std::mem::transmute;
const RCONS: [u32; 10] = [0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1B, 0x36];
const NUM_WORDS_IN_KEY: usize = 4;
const NUM_ROUNDS: usize = 10;
const NUM_ROUND_KEYS: usize = NUM_ROUNDS + 1;
/// An aes block cipher implementation which uses `neon` and `aes` instructions.
#[derive(Clone)]
pub struct ArmAesBlockCipher {
round_keys: [uint8x16_t; NUM_ROUND_KEYS],
}
impl AesBlockCipher for ArmAesBlockCipher {
fn new(key: AesKey) -> ArmAesBlockCipher {
let aes_detected = is_aarch64_feature_detected!("aes");
let neon_detected = is_aarch64_feature_detected!("neon");
if !(aes_detected && neon_detected) {
panic!(
"The ArmAesBlockCipher requires both aes and neon aarch64 CPU features.\n\
aes feature available: {}\nneon feature available: {}\n.",
aes_detected, neon_detected
)
}
let round_keys = unsafe { generate_round_keys(key) };
ArmAesBlockCipher { round_keys }
}
fn generate_batch(&mut self, AesIndex(aes_ctr): AesIndex) -> [u8; BYTES_PER_BATCH] {
#[target_feature(enable = "aes,neon")]
unsafe fn implementation(
this: &ArmAesBlockCipher,
AesIndex(aes_ctr): AesIndex,
) -> [u8; BYTES_PER_BATCH] {
let mut output = [0u8; BYTES_PER_BATCH];
// We want 128 bytes of output, the ctr gives 128 bit message (16 bytes)
for (i, out) in output.chunks_exact_mut(16).enumerate() {
// Safe because we prevent the user from creating the Generator
// on non-supported hardware
let encrypted = encrypt(aes_ctr + (i as u128), &this.round_keys);
out.copy_from_slice(&encrypted.to_ne_bytes());
}
output
}
// SAFETY: we checked for aes and neon availability in `Self::new`
unsafe { implementation(self, AesIndex(aes_ctr)) }
}
}
/// Does the AES SubWord operation for the Key Expansion step
///
/// # SAFETY
///
/// You must make sure the CPU's arch is`aarch64` and has
/// `neon` and `aes` features.
#[inline(always)]
unsafe fn sub_word(word: u32) -> u32 {
let data = vreinterpretq_u8_u32(vdupq_n_u32(word));
let zero_key = vdupq_n_u8(0u8);
let temp = vaeseq_u8(data, zero_key);
// vaeseq_u8 does SubBytes(ShiftRow(XOR(data, key))
// But because we used a zero aes key,the XOR did not alter data
// We now have temp = SubBytes(ShiftRow(data))
// Since in AES ShiftRow operation, the first row is not shifted
// We can just get that one to have our SubWord(word) result
vgetq_lane_u32::<0>(vreinterpretq_u32_u8(temp))
}
#[inline(always)]
fn uint8x16_t_to_u128(input: uint8x16_t) -> u128 {
unsafe { transmute(input) }
}
#[inline(always)]
fn u128_to_uint8x16_t(input: u128) -> uint8x16_t {
unsafe { transmute(input) }
}
#[target_feature(enable = "aes,neon")]
unsafe fn generate_round_keys(key: AesKey) -> [uint8x16_t; NUM_ROUND_KEYS] {
let mut round_keys: [uint8x16_t; NUM_ROUND_KEYS] = std::mem::zeroed();
round_keys[0] = u128_to_uint8x16_t(key.0);
let words = std::slice::from_raw_parts_mut(
round_keys.as_mut_ptr() as *mut u32,
NUM_ROUND_KEYS * NUM_WORDS_IN_KEY,
);
debug_assert_eq!(words.len(), 44);
// Skip the words of the first key, its already done
for i in NUM_WORDS_IN_KEY..words.len() {
if (i % NUM_WORDS_IN_KEY) == 0 {
words[i] = words[i - NUM_WORDS_IN_KEY]
^ sub_word(words[i - 1]).rotate_right(8)
^ RCONS[(i / NUM_WORDS_IN_KEY) - 1];
} else {
words[i] = words[i - NUM_WORDS_IN_KEY] ^ words[i - 1];
}
// Note: there is also a special thing to do when
// i mod SElf::NUM_WORDS_IN_KEY == 4 but it cannot happen on 128 bits keys
}
round_keys
}
/// Encrypts a 128-bit message
///
/// # SAFETY
///
/// You must make sure the CPU's arch is`aarch64` and has
/// `neon` and `aes` features.
#[inline(always)]
unsafe fn encrypt(message: u128, keys: &[uint8x16_t; NUM_ROUND_KEYS]) -> u128 {
// Notes:
// According the [ARM Manual](https://developer.arm.com/documentation/ddi0487/gb/):
// `vaeseq_u8` is the following AES operations:
// 1. AddRoundKey (XOR)
// 2. ShiftRows
// 3. SubBytes
// `vaesmcq_u8` is MixColumns
let mut data: uint8x16_t = u128_to_uint8x16_t(message);
for &key in keys.iter().take(NUM_ROUNDS - 1) {
data = vaesmcq_u8(vaeseq_u8(data, key));
}
data = vaeseq_u8(data, keys[NUM_ROUNDS - 1]);
data = veorq_u8(data, keys[NUM_ROUND_KEYS - 1]);
uint8x16_t_to_u128(data)
}
#[cfg(test)]
mod test {
use super::*;
// Test vector for aes128, from the FIPS publication 197
const CIPHER_KEY: u128 = u128::from_be(0x000102030405060708090a0b0c0d0e0f);
const KEY_SCHEDULE: [u128; 11] = [
u128::from_be(0x000102030405060708090a0b0c0d0e0f),
u128::from_be(0xd6aa74fdd2af72fadaa678f1d6ab76fe),
u128::from_be(0xb692cf0b643dbdf1be9bc5006830b3fe),
u128::from_be(0xb6ff744ed2c2c9bf6c590cbf0469bf41),
u128::from_be(0x47f7f7bc95353e03f96c32bcfd058dfd),
u128::from_be(0x3caaa3e8a99f9deb50f3af57adf622aa),
u128::from_be(0x5e390f7df7a69296a7553dc10aa31f6b),
u128::from_be(0x14f9701ae35fe28c440adf4d4ea9c026),
u128::from_be(0x47438735a41c65b9e016baf4aebf7ad2),
u128::from_be(0x549932d1f08557681093ed9cbe2c974e),
u128::from_be(0x13111d7fe3944a17f307a78b4d2b30c5),
];
const PLAINTEXT: u128 = u128::from_be(0x00112233445566778899aabbccddeeff);
const CIPHERTEXT: u128 = u128::from_be(0x69c4e0d86a7b0430d8cdb78070b4c55a);
#[test]
fn test_generate_key_schedule() {
// Checks that the round keys are correctly generated from the sample key from FIPS
let key = AesKey(CIPHER_KEY);
let keys = unsafe { generate_round_keys(key) };
for (expected, actual) in KEY_SCHEDULE.iter().zip(keys.iter()) {
assert_eq!(*expected, uint8x16_t_to_u128(*actual));
}
}
#[test]
fn test_encrypt_message() {
// Checks that encrypting many plaintext at the same time gives the correct output.
let message = PLAINTEXT;
let key = AesKey(CIPHER_KEY);
let keys = unsafe { generate_round_keys(key) };
let ciphertext = unsafe { encrypt(message, &keys) };
assert_eq!(CIPHERTEXT, ciphertext);
}
}

View File

@@ -0,0 +1,110 @@
use crate::generators::aes_ctr::{AesCtrGenerator, AesKey, ChildrenIterator};
use crate::generators::implem::aarch64::block_cipher::ArmAesBlockCipher;
use crate::generators::{ByteCount, BytesPerChild, ChildrenCount, ForkError, RandomGenerator};
use crate::seeders::Seed;
/// A random number generator using the `aesni` instructions.
pub struct NeonAesRandomGenerator(pub(super) AesCtrGenerator<ArmAesBlockCipher>);
/// The children iterator used by [`NeonAesRandomGenerator`].
///
/// Outputs children generators one by one.
pub struct ArmAesChildrenIterator(ChildrenIterator<ArmAesBlockCipher>);
impl Iterator for ArmAesChildrenIterator {
type Item = NeonAesRandomGenerator;
fn next(&mut self) -> Option<Self::Item> {
self.0.next().map(NeonAesRandomGenerator)
}
}
impl RandomGenerator for NeonAesRandomGenerator {
type ChildrenIter = ArmAesChildrenIterator;
fn new(seed: Seed) -> Self {
NeonAesRandomGenerator(AesCtrGenerator::new(AesKey(seed.0), None, None))
}
fn remaining_bytes(&self) -> ByteCount {
self.0.remaining_bytes()
}
fn try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ChildrenIter, ForkError> {
self.0
.try_fork(n_children, n_bytes)
.map(ArmAesChildrenIterator)
}
}
impl Iterator for NeonAesRandomGenerator {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
self.0.next()
}
}
#[cfg(test)]
mod test {
use crate::generators::aes_ctr::aes_ctr_generic_test;
use crate::generators::implem::aarch64::block_cipher::ArmAesBlockCipher;
use crate::generators::{generator_generic_test, NeonAesRandomGenerator};
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_generic_test::prop_fork_first_state_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_generic_test::prop_fork_last_bound_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_generic_test::prop_fork_parent_bound_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_generic_test::prop_fork_parent_state_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork() {
aes_ctr_generic_test::prop_fork::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_generic_test::prop_fork_children_remaining_bytes::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_generic_test::prop_fork_parent_remaining_bytes::<ArmAesBlockCipher>();
}
#[test]
fn test_roughly_uniform() {
generator_generic_test::test_roughly_uniform::<NeonAesRandomGenerator>();
}
#[test]
fn test_generator_determinism() {
generator_generic_test::test_generator_determinism::<NeonAesRandomGenerator>();
}
#[test]
fn test_fork() {
generator_generic_test::test_fork_children::<NeonAesRandomGenerator>();
}
#[test]
#[should_panic(expected = "expected test panic")]
fn test_bounded_panic() {
generator_generic_test::test_bounded_none_should_panic::<NeonAesRandomGenerator>();
}
}

View File

@@ -0,0 +1,16 @@
//! A module implementing a random number generator, using the aarch64 `neon` and `aes`
//! instructions.
//!
//! This module implements a cryptographically secure pseudorandom number generator
//! (CS-PRNG), using a fast block cipher. The implementation is based on the
//! [intel aesni white paper 323641-001 revision 3.0](https://www.intel.com/content/dam/doc/white-paper/advanced-encryption-standard-new-instructions-set-paper.pdf).
mod block_cipher;
mod generator;
pub use generator::*;
#[cfg(feature = "parallel")]
mod parallel;
#[cfg(feature = "parallel")]
pub use parallel::*;

View File

@@ -0,0 +1,95 @@
use super::*;
use crate::generators::aes_ctr::{AesCtrGenerator, ParallelChildrenIterator};
use crate::generators::implem::aarch64::block_cipher::ArmAesBlockCipher;
use crate::generators::{BytesPerChild, ChildrenCount, ForkError, ParallelRandomGenerator};
use rayon::iter::plumbing::{Consumer, ProducerCallback, UnindexedConsumer};
use rayon::prelude::*;
/// The parallel children iterator used by [`NeonAesRandomGenerator`].
///
/// Outputs the children generators one by one.
#[allow(clippy::type_complexity)]
pub struct ParallelArmAesChildrenIterator(
rayon::iter::Map<
ParallelChildrenIterator<ArmAesBlockCipher>,
fn(AesCtrGenerator<ArmAesBlockCipher>) -> NeonAesRandomGenerator,
>,
);
impl ParallelIterator for ParallelArmAesChildrenIterator {
type Item = NeonAesRandomGenerator;
fn drive_unindexed<C>(self, consumer: C) -> C::Result
where
C: UnindexedConsumer<Self::Item>,
{
self.0.drive_unindexed(consumer)
}
}
impl IndexedParallelIterator for ParallelArmAesChildrenIterator {
fn len(&self) -> usize {
self.0.len()
}
fn drive<C: Consumer<Self::Item>>(self, consumer: C) -> C::Result {
self.0.drive(consumer)
}
fn with_producer<CB: ProducerCallback<Self::Item>>(self, callback: CB) -> CB::Output {
self.0.with_producer(callback)
}
}
impl ParallelRandomGenerator for NeonAesRandomGenerator {
type ParChildrenIter = ParallelArmAesChildrenIterator;
fn par_try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ParChildrenIter, ForkError> {
self.0
.par_try_fork(n_children, n_bytes)
.map(|iterator| ParallelArmAesChildrenIterator(iterator.map(NeonAesRandomGenerator)))
}
}
#[cfg(test)]
mod test {
use crate::generators::aes_ctr::aes_ctr_parallel_generic_tests;
use crate::generators::implem::aarch64::block_cipher::ArmAesBlockCipher;
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_first_state_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_last_bound_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_bound_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_state_table_index::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_ttt() {
aes_ctr_parallel_generic_tests::prop_fork::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_children_remaining_bytes::<ArmAesBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_parent_remaining_bytes::<ArmAesBlockCipher>();
}
}

View File

@@ -0,0 +1,231 @@
use crate::generators::aes_ctr::{AesBlockCipher, AesIndex, AesKey, BYTES_PER_BATCH};
use std::arch::x86_64::{
__m128i, _mm_aesenc_si128, _mm_aesenclast_si128, _mm_aeskeygenassist_si128, _mm_shuffle_epi32,
_mm_slli_si128, _mm_store_si128, _mm_xor_si128,
};
use std::mem::transmute;
/// An aes block cipher implementation which uses `aesni` instructions.
#[derive(Clone)]
pub struct AesniBlockCipher {
// The set of round keys used for the aes encryption
round_keys: [__m128i; 11],
}
impl AesBlockCipher for AesniBlockCipher {
fn new(key: AesKey) -> AesniBlockCipher {
let aes_detected = is_x86_feature_detected!("aes");
let sse2_detected = is_x86_feature_detected!("sse2");
if !(aes_detected && sse2_detected) {
panic!(
"The AesniBlockCipher requires both aes and sse2 x86 CPU features.\n\
aes feature available: {}\nsse2 feature available: {}\n.",
aes_detected, sse2_detected
)
}
// SAFETY: we checked for aes and sse2 availability
let round_keys = unsafe { generate_round_keys(key) };
AesniBlockCipher { round_keys }
}
fn generate_batch(&mut self, AesIndex(aes_ctr): AesIndex) -> [u8; BYTES_PER_BATCH] {
#[target_feature(enable = "sse2,aes")]
unsafe fn implementation(
this: &AesniBlockCipher,
AesIndex(aes_ctr): AesIndex,
) -> [u8; BYTES_PER_BATCH] {
si128arr_to_u8arr(aes_encrypt_many(
u128_to_si128(aes_ctr),
u128_to_si128(aes_ctr + 1),
u128_to_si128(aes_ctr + 2),
u128_to_si128(aes_ctr + 3),
u128_to_si128(aes_ctr + 4),
u128_to_si128(aes_ctr + 5),
u128_to_si128(aes_ctr + 6),
u128_to_si128(aes_ctr + 7),
&this.round_keys,
))
}
// SAFETY: we checked for aes and sse2 availability in `Self::new`
unsafe { implementation(self, AesIndex(aes_ctr)) }
}
}
#[target_feature(enable = "sse2,aes")]
unsafe fn generate_round_keys(key: AesKey) -> [__m128i; 11] {
let key = u128_to_si128(key.0);
let mut keys: [__m128i; 11] = [u128_to_si128(0); 11];
aes_128_key_expansion(key, &mut keys);
keys
}
// Uses aes to encrypt many values at once. This allows a substantial speedup (around 30%)
// compared to the naive approach.
#[allow(clippy::too_many_arguments)]
#[inline(always)]
fn aes_encrypt_many(
message_1: __m128i,
message_2: __m128i,
message_3: __m128i,
message_4: __m128i,
message_5: __m128i,
message_6: __m128i,
message_7: __m128i,
message_8: __m128i,
keys: &[__m128i; 11],
) -> [__m128i; 8] {
unsafe {
let mut tmp_1 = _mm_xor_si128(message_1, keys[0]);
let mut tmp_2 = _mm_xor_si128(message_2, keys[0]);
let mut tmp_3 = _mm_xor_si128(message_3, keys[0]);
let mut tmp_4 = _mm_xor_si128(message_4, keys[0]);
let mut tmp_5 = _mm_xor_si128(message_5, keys[0]);
let mut tmp_6 = _mm_xor_si128(message_6, keys[0]);
let mut tmp_7 = _mm_xor_si128(message_7, keys[0]);
let mut tmp_8 = _mm_xor_si128(message_8, keys[0]);
for key in keys.iter().take(10).skip(1) {
tmp_1 = _mm_aesenc_si128(tmp_1, *key);
tmp_2 = _mm_aesenc_si128(tmp_2, *key);
tmp_3 = _mm_aesenc_si128(tmp_3, *key);
tmp_4 = _mm_aesenc_si128(tmp_4, *key);
tmp_5 = _mm_aesenc_si128(tmp_5, *key);
tmp_6 = _mm_aesenc_si128(tmp_6, *key);
tmp_7 = _mm_aesenc_si128(tmp_7, *key);
tmp_8 = _mm_aesenc_si128(tmp_8, *key);
}
tmp_1 = _mm_aesenclast_si128(tmp_1, keys[10]);
tmp_2 = _mm_aesenclast_si128(tmp_2, keys[10]);
tmp_3 = _mm_aesenclast_si128(tmp_3, keys[10]);
tmp_4 = _mm_aesenclast_si128(tmp_4, keys[10]);
tmp_5 = _mm_aesenclast_si128(tmp_5, keys[10]);
tmp_6 = _mm_aesenclast_si128(tmp_6, keys[10]);
tmp_7 = _mm_aesenclast_si128(tmp_7, keys[10]);
tmp_8 = _mm_aesenclast_si128(tmp_8, keys[10]);
[tmp_1, tmp_2, tmp_3, tmp_4, tmp_5, tmp_6, tmp_7, tmp_8]
}
}
fn aes_128_assist(temp1: __m128i, temp2: __m128i) -> __m128i {
let mut temp3: __m128i;
let mut temp2 = temp2;
let mut temp1 = temp1;
unsafe {
temp2 = _mm_shuffle_epi32(temp2, 0xff);
temp3 = _mm_slli_si128(temp1, 0x4);
temp1 = _mm_xor_si128(temp1, temp3);
temp3 = _mm_slli_si128(temp3, 0x4);
temp1 = _mm_xor_si128(temp1, temp3);
temp3 = _mm_slli_si128(temp3, 0x4);
temp1 = _mm_xor_si128(temp1, temp3);
temp1 = _mm_xor_si128(temp1, temp2);
}
temp1
}
#[inline(always)]
fn aes_128_key_expansion(key: __m128i, keys: &mut [__m128i; 11]) {
let (mut temp1, mut temp2): (__m128i, __m128i);
temp1 = key;
unsafe {
_mm_store_si128(keys.as_mut_ptr(), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x01);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(1), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x02);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(2), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x04);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(3), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x08);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(4), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x10);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(5), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x20);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(6), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x40);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(7), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x80);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(8), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x1b);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(9), temp1);
temp2 = _mm_aeskeygenassist_si128(temp1, 0x36);
temp1 = aes_128_assist(temp1, temp2);
_mm_store_si128(keys.as_mut_ptr().offset(10), temp1);
}
}
#[inline(always)]
fn u128_to_si128(input: u128) -> __m128i {
unsafe { transmute(input) }
}
#[allow(unused)] // to please clippy when tests are not activated
fn si128_to_u128(input: __m128i) -> u128 {
unsafe { transmute(input) }
}
#[inline(always)]
fn si128arr_to_u8arr(input: [__m128i; 8]) -> [u8; BYTES_PER_BATCH] {
unsafe { transmute(input) }
}
#[cfg(test)]
mod test {
use super::*;
// Test vector for aes128, from the FIPS publication 197
const CIPHER_KEY: u128 = u128::from_be(0x000102030405060708090a0b0c0d0e0f);
const KEY_SCHEDULE: [u128; 11] = [
u128::from_be(0x000102030405060708090a0b0c0d0e0f),
u128::from_be(0xd6aa74fdd2af72fadaa678f1d6ab76fe),
u128::from_be(0xb692cf0b643dbdf1be9bc5006830b3fe),
u128::from_be(0xb6ff744ed2c2c9bf6c590cbf0469bf41),
u128::from_be(0x47f7f7bc95353e03f96c32bcfd058dfd),
u128::from_be(0x3caaa3e8a99f9deb50f3af57adf622aa),
u128::from_be(0x5e390f7df7a69296a7553dc10aa31f6b),
u128::from_be(0x14f9701ae35fe28c440adf4d4ea9c026),
u128::from_be(0x47438735a41c65b9e016baf4aebf7ad2),
u128::from_be(0x549932d1f08557681093ed9cbe2c974e),
u128::from_be(0x13111d7fe3944a17f307a78b4d2b30c5),
];
const PLAINTEXT: u128 = u128::from_be(0x00112233445566778899aabbccddeeff);
const CIPHERTEXT: u128 = u128::from_be(0x69c4e0d86a7b0430d8cdb78070b4c55a);
#[test]
fn test_generate_key_schedule() {
// Checks that the round keys are correctly generated from the sample key from FIPS
let key = u128_to_si128(CIPHER_KEY);
let mut keys: [__m128i; 11] = [u128_to_si128(0); 11];
aes_128_key_expansion(key, &mut keys);
for (expected, actual) in KEY_SCHEDULE.iter().zip(keys.iter()) {
assert_eq!(*expected, si128_to_u128(*actual));
}
}
#[test]
fn test_encrypt_many_messages() {
// Checks that encrypting many plaintext at the same time gives the correct output.
let message = u128_to_si128(PLAINTEXT);
let key = u128_to_si128(CIPHER_KEY);
let mut keys: [__m128i; 11] = [u128_to_si128(0); 11];
aes_128_key_expansion(key, &mut keys);
let ciphertexts = aes_encrypt_many(
message, message, message, message, message, message, message, message, &keys,
);
for ct in &ciphertexts {
assert_eq!(CIPHERTEXT, si128_to_u128(*ct));
}
}
}

View File

@@ -0,0 +1,110 @@
use crate::generators::aes_ctr::{AesCtrGenerator, AesKey, ChildrenIterator};
use crate::generators::implem::aesni::block_cipher::AesniBlockCipher;
use crate::generators::{ByteCount, BytesPerChild, ChildrenCount, ForkError, RandomGenerator};
use crate::seeders::Seed;
/// A random number generator using the `aesni` instructions.
pub struct AesniRandomGenerator(pub(super) AesCtrGenerator<AesniBlockCipher>);
/// The children iterator used by [`AesniRandomGenerator`].
///
/// Outputs children generators one by one.
pub struct AesniChildrenIterator(ChildrenIterator<AesniBlockCipher>);
impl Iterator for AesniChildrenIterator {
type Item = AesniRandomGenerator;
fn next(&mut self) -> Option<Self::Item> {
self.0.next().map(AesniRandomGenerator)
}
}
impl RandomGenerator for AesniRandomGenerator {
type ChildrenIter = AesniChildrenIterator;
fn new(seed: Seed) -> Self {
AesniRandomGenerator(AesCtrGenerator::new(AesKey(seed.0), None, None))
}
fn remaining_bytes(&self) -> ByteCount {
self.0.remaining_bytes()
}
fn try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ChildrenIter, ForkError> {
self.0
.try_fork(n_children, n_bytes)
.map(AesniChildrenIterator)
}
}
impl Iterator for AesniRandomGenerator {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
self.0.next()
}
}
#[cfg(test)]
mod test {
use crate::generators::aes_ctr::aes_ctr_generic_test;
use crate::generators::implem::aesni::block_cipher::AesniBlockCipher;
use crate::generators::{generator_generic_test, AesniRandomGenerator};
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_generic_test::prop_fork_first_state_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_generic_test::prop_fork_last_bound_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_generic_test::prop_fork_parent_bound_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_generic_test::prop_fork_parent_state_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork() {
aes_ctr_generic_test::prop_fork::<AesniBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_generic_test::prop_fork_children_remaining_bytes::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_generic_test::prop_fork_parent_remaining_bytes::<AesniBlockCipher>();
}
#[test]
fn test_roughly_uniform() {
generator_generic_test::test_roughly_uniform::<AesniRandomGenerator>();
}
#[test]
fn test_generator_determinism() {
generator_generic_test::test_generator_determinism::<AesniRandomGenerator>();
}
#[test]
fn test_fork() {
generator_generic_test::test_fork_children::<AesniRandomGenerator>();
}
#[test]
#[should_panic(expected = "expected test panic")]
fn test_bounded_panic() {
generator_generic_test::test_bounded_none_should_panic::<AesniRandomGenerator>();
}
}

View File

@@ -0,0 +1,15 @@
//! A module implementing a random number generator, using the x86_64 `aesni` instructions.
//!
//! This module implements a cryptographically secure pseudorandom number generator
//! (CS-PRNG), using a fast block cipher. The implementation is based on the
//! [intel aesni white paper 323641-001 revision 3.0](https://www.intel.com/content/dam/doc/white-paper/advanced-encryption-standard-new-instructions-set-paper.pdf).
mod block_cipher;
mod generator;
pub use generator::*;
#[cfg(feature = "parallel")]
mod parallel;
#[cfg(feature = "parallel")]
pub use parallel::*;

View File

@@ -0,0 +1,95 @@
use super::*;
use crate::generators::aes_ctr::{AesCtrGenerator, ParallelChildrenIterator};
use crate::generators::implem::aesni::block_cipher::AesniBlockCipher;
use crate::generators::{BytesPerChild, ChildrenCount, ForkError, ParallelRandomGenerator};
use rayon::iter::plumbing::{Consumer, ProducerCallback, UnindexedConsumer};
use rayon::prelude::*;
/// The parallel children iterator used by [`AesniRandomGenerator`].
///
/// Outputs the children generators one by one.
#[allow(clippy::type_complexity)]
pub struct ParallelAesniChildrenIterator(
rayon::iter::Map<
ParallelChildrenIterator<AesniBlockCipher>,
fn(AesCtrGenerator<AesniBlockCipher>) -> AesniRandomGenerator,
>,
);
impl ParallelIterator for ParallelAesniChildrenIterator {
type Item = AesniRandomGenerator;
fn drive_unindexed<C>(self, consumer: C) -> C::Result
where
C: UnindexedConsumer<Self::Item>,
{
self.0.drive_unindexed(consumer)
}
}
impl IndexedParallelIterator for ParallelAesniChildrenIterator {
fn len(&self) -> usize {
self.0.len()
}
fn drive<C: Consumer<Self::Item>>(self, consumer: C) -> C::Result {
self.0.drive(consumer)
}
fn with_producer<CB: ProducerCallback<Self::Item>>(self, callback: CB) -> CB::Output {
self.0.with_producer(callback)
}
}
impl ParallelRandomGenerator for AesniRandomGenerator {
type ParChildrenIter = ParallelAesniChildrenIterator;
fn par_try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ParChildrenIter, ForkError> {
self.0
.par_try_fork(n_children, n_bytes)
.map(|iterator| ParallelAesniChildrenIterator(iterator.map(AesniRandomGenerator)))
}
}
#[cfg(test)]
mod test {
use crate::generators::aes_ctr::aes_ctr_parallel_generic_tests;
use crate::generators::implem::aesni::block_cipher::AesniBlockCipher;
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_first_state_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_last_bound_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_bound_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_state_table_index::<AesniBlockCipher>();
}
#[test]
fn prop_fork_ttt() {
aes_ctr_parallel_generic_tests::prop_fork::<AesniBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_children_remaining_bytes::<AesniBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_parent_remaining_bytes::<AesniBlockCipher>();
}
}

View File

@@ -0,0 +1,14 @@
#[cfg(feature = "generator_x86_64_aesni")]
mod aesni;
#[cfg(feature = "generator_x86_64_aesni")]
pub use aesni::*;
#[cfg(feature = "generator_aarch64_aes")]
mod aarch64;
#[cfg(feature = "generator_aarch64_aes")]
pub use aarch64::*;
#[cfg(feature = "generator_fallback")]
mod soft;
#[cfg(feature = "generator_fallback")]
pub use soft::*;

View File

@@ -0,0 +1,114 @@
use crate::generators::aes_ctr::{
AesBlockCipher, AesIndex, AesKey, AES_CALLS_PER_BATCH, BYTES_PER_AES_CALL, BYTES_PER_BATCH,
};
use aes::cipher::generic_array::GenericArray;
use aes::cipher::{BlockEncrypt, KeyInit};
use aes::Aes128;
#[derive(Clone)]
pub struct SoftwareBlockCipher {
// Aes structure
aes: Aes128,
}
impl AesBlockCipher for SoftwareBlockCipher {
fn new(key: AesKey) -> SoftwareBlockCipher {
let key: [u8; BYTES_PER_AES_CALL] = key.0.to_ne_bytes();
let key = GenericArray::clone_from_slice(&key[..]);
let aes = Aes128::new(&key);
SoftwareBlockCipher { aes }
}
fn generate_batch(&mut self, AesIndex(aes_ctr): AesIndex) -> [u8; BYTES_PER_BATCH] {
aes_encrypt_many(
aes_ctr,
aes_ctr + 1,
aes_ctr + 2,
aes_ctr + 3,
aes_ctr + 4,
aes_ctr + 5,
aes_ctr + 6,
aes_ctr + 7,
&self.aes,
)
}
}
// Uses aes to encrypt many values at once. This allows a substantial speedup (around 30%)
// compared to the naive approach.
#[allow(clippy::too_many_arguments)]
fn aes_encrypt_many(
message_1: u128,
message_2: u128,
message_3: u128,
message_4: u128,
message_5: u128,
message_6: u128,
message_7: u128,
message_8: u128,
cipher: &Aes128,
) -> [u8; BYTES_PER_BATCH] {
let mut b1 = GenericArray::clone_from_slice(&message_1.to_ne_bytes()[..]);
let mut b2 = GenericArray::clone_from_slice(&message_2.to_ne_bytes()[..]);
let mut b3 = GenericArray::clone_from_slice(&message_3.to_ne_bytes()[..]);
let mut b4 = GenericArray::clone_from_slice(&message_4.to_ne_bytes()[..]);
let mut b5 = GenericArray::clone_from_slice(&message_5.to_ne_bytes()[..]);
let mut b6 = GenericArray::clone_from_slice(&message_6.to_ne_bytes()[..]);
let mut b7 = GenericArray::clone_from_slice(&message_7.to_ne_bytes()[..]);
let mut b8 = GenericArray::clone_from_slice(&message_8.to_ne_bytes()[..]);
cipher.encrypt_block(&mut b1);
cipher.encrypt_block(&mut b2);
cipher.encrypt_block(&mut b3);
cipher.encrypt_block(&mut b4);
cipher.encrypt_block(&mut b5);
cipher.encrypt_block(&mut b6);
cipher.encrypt_block(&mut b7);
cipher.encrypt_block(&mut b8);
let output_array: [[u8; BYTES_PER_AES_CALL]; AES_CALLS_PER_BATCH] = [
b1.into(),
b2.into(),
b3.into(),
b4.into(),
b5.into(),
b6.into(),
b7.into(),
b8.into(),
];
unsafe { *{ output_array.as_ptr() as *const [u8; BYTES_PER_BATCH] } }
}
#[cfg(test)]
mod test {
use super::*;
use std::convert::TryInto;
// Test vector for aes128, from the FIPS publication 197
const CIPHER_KEY: u128 = u128::from_be(0x000102030405060708090a0b0c0d0e0f);
const PLAINTEXT: u128 = u128::from_be(0x00112233445566778899aabbccddeeff);
const CIPHERTEXT: u128 = u128::from_be(0x69c4e0d86a7b0430d8cdb78070b4c55a);
#[test]
fn test_encrypt_many_messages() {
// Checks that encrypting many plaintext at the same time gives the correct output.
let key: [u8; BYTES_PER_AES_CALL] = CIPHER_KEY.to_ne_bytes();
let aes = Aes128::new(&GenericArray::from(key));
let ciphertexts = aes_encrypt_many(
PLAINTEXT, PLAINTEXT, PLAINTEXT, PLAINTEXT, PLAINTEXT, PLAINTEXT, PLAINTEXT, PLAINTEXT,
&aes,
);
let ciphertexts: [u8; BYTES_PER_BATCH] = ciphertexts[..].try_into().unwrap();
for i in 0..8 {
assert_eq!(
u128::from_ne_bytes(
ciphertexts[BYTES_PER_AES_CALL * i..BYTES_PER_AES_CALL * (i + 1)]
.try_into()
.unwrap()
),
CIPHERTEXT
);
}
}
}

View File

@@ -0,0 +1,110 @@
use crate::generators::aes_ctr::{AesCtrGenerator, AesKey, ChildrenIterator};
use crate::generators::implem::soft::block_cipher::SoftwareBlockCipher;
use crate::generators::{ByteCount, BytesPerChild, ChildrenCount, ForkError, RandomGenerator};
use crate::seeders::Seed;
/// A random number generator using a software implementation.
pub struct SoftwareRandomGenerator(pub(super) AesCtrGenerator<SoftwareBlockCipher>);
/// The children iterator used by [`SoftwareRandomGenerator`].
///
/// Outputs children generators one by one.
pub struct SoftwareChildrenIterator(ChildrenIterator<SoftwareBlockCipher>);
impl Iterator for SoftwareChildrenIterator {
type Item = SoftwareRandomGenerator;
fn next(&mut self) -> Option<Self::Item> {
self.0.next().map(SoftwareRandomGenerator)
}
}
impl RandomGenerator for SoftwareRandomGenerator {
type ChildrenIter = SoftwareChildrenIterator;
fn new(seed: Seed) -> Self {
SoftwareRandomGenerator(AesCtrGenerator::new(AesKey(seed.0), None, None))
}
fn remaining_bytes(&self) -> ByteCount {
self.0.remaining_bytes()
}
fn try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ChildrenIter, ForkError> {
self.0
.try_fork(n_children, n_bytes)
.map(SoftwareChildrenIterator)
}
}
impl Iterator for SoftwareRandomGenerator {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
self.0.next()
}
}
#[cfg(test)]
mod test {
use super::*;
use crate::generators::aes_ctr::aes_ctr_generic_test;
use crate::generators::generator_generic_test;
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_generic_test::prop_fork_first_state_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_generic_test::prop_fork_last_bound_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_generic_test::prop_fork_parent_bound_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_generic_test::prop_fork_parent_state_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork() {
aes_ctr_generic_test::prop_fork::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_generic_test::prop_fork_children_remaining_bytes::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_generic_test::prop_fork_parent_remaining_bytes::<SoftwareBlockCipher>();
}
#[test]
fn test_roughly_uniform() {
generator_generic_test::test_roughly_uniform::<SoftwareRandomGenerator>();
}
#[test]
fn test_fork() {
generator_generic_test::test_fork_children::<SoftwareRandomGenerator>();
}
#[test]
fn test_generator_determinism() {
generator_generic_test::test_generator_determinism::<SoftwareRandomGenerator>();
}
#[test]
#[should_panic(expected = "expected test panic")]
fn test_bounded_panic() {
generator_generic_test::test_bounded_none_should_panic::<SoftwareRandomGenerator>();
}
}

View File

@@ -0,0 +1,11 @@
//! A module using a software fallback implementation of random number generator.
mod block_cipher;
mod generator;
pub use generator::*;
#[cfg(feature = "parallel")]
mod parallel;
#[cfg(feature = "parallel")]
pub use parallel::*;

View File

@@ -0,0 +1,94 @@
use super::*;
use crate::generators::aes_ctr::{AesCtrGenerator, ParallelChildrenIterator};
use crate::generators::implem::soft::block_cipher::SoftwareBlockCipher;
use crate::generators::{BytesPerChild, ChildrenCount, ForkError, ParallelRandomGenerator};
use rayon::iter::plumbing::{Consumer, ProducerCallback, UnindexedConsumer};
use rayon::prelude::*;
/// The parallel children iterator used by [`SoftwareRandomGenerator`].
///
/// Outputs the children generators one by one.
#[allow(clippy::type_complexity)]
pub struct ParallelSoftwareChildrenIterator(
rayon::iter::Map<
ParallelChildrenIterator<SoftwareBlockCipher>,
fn(AesCtrGenerator<SoftwareBlockCipher>) -> SoftwareRandomGenerator,
>,
);
impl ParallelIterator for ParallelSoftwareChildrenIterator {
type Item = SoftwareRandomGenerator;
fn drive_unindexed<C>(self, consumer: C) -> C::Result
where
C: UnindexedConsumer<Self::Item>,
{
self.0.drive_unindexed(consumer)
}
}
impl IndexedParallelIterator for ParallelSoftwareChildrenIterator {
fn len(&self) -> usize {
self.0.len()
}
fn drive<C: Consumer<Self::Item>>(self, consumer: C) -> C::Result {
self.0.drive(consumer)
}
fn with_producer<CB: ProducerCallback<Self::Item>>(self, callback: CB) -> CB::Output {
self.0.with_producer(callback)
}
}
impl ParallelRandomGenerator for SoftwareRandomGenerator {
type ParChildrenIter = ParallelSoftwareChildrenIterator;
fn par_try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ParChildrenIter, ForkError> {
self.0
.par_try_fork(n_children, n_bytes)
.map(|iterator| ParallelSoftwareChildrenIterator(iterator.map(SoftwareRandomGenerator)))
}
}
#[cfg(test)]
mod test {
use super::*;
use crate::generators::aes_ctr::aes_ctr_parallel_generic_tests;
#[test]
fn prop_fork_first_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_first_state_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_last_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_last_bound_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_bound_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_bound_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_state_table_index() {
aes_ctr_parallel_generic_tests::prop_fork_parent_state_table_index::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork() {
aes_ctr_parallel_generic_tests::prop_fork::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_children_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_children_remaining_bytes::<SoftwareBlockCipher>();
}
#[test]
fn prop_fork_parent_remaining_bytes() {
aes_ctr_parallel_generic_tests::prop_fork_parent_remaining_bytes::<SoftwareBlockCipher>();
}
}

View File

@@ -0,0 +1,235 @@
//! A module containing random generators objects.
//!
//! See [crate-level](`crate`) explanations.
use crate::seeders::Seed;
use std::error::Error;
use std::fmt::{Display, Formatter};
/// The number of children created when a generator is forked.
#[derive(Debug, Copy, Clone)]
pub struct ChildrenCount(pub usize);
/// The number of bytes each child can generate, when a generator is forked.
#[derive(Debug, Copy, Clone)]
pub struct BytesPerChild(pub usize);
/// A structure representing the number of bytes between two table indices.
#[derive(Clone, Copy, Debug, PartialOrd, Ord, PartialEq, Eq)]
pub struct ByteCount(pub u128);
/// An error occurring during a generator fork.
#[derive(Debug)]
pub enum ForkError {
ForkTooLarge,
ZeroChildrenCount,
ZeroBytesPerChild,
}
impl Display for ForkError {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
ForkError::ForkTooLarge => {
write!(
f,
"The children generators would output bytes after the parent bound. "
)
}
ForkError::ZeroChildrenCount => {
write!(
f,
"The number of children in the fork must be greater than zero."
)
}
ForkError::ZeroBytesPerChild => {
write!(
f,
"The number of bytes per child must be greater than zero."
)
}
}
}
}
impl Error for ForkError {}
/// A trait for cryptographically secure pseudo-random generators.
///
/// See the [crate-level](#crate) documentation for details.
pub trait RandomGenerator: Iterator<Item = u8> {
/// The iterator over children generators, returned by `try_fork` in case of success.
type ChildrenIter: Iterator<Item = Self>;
/// Creates a new generator from a seed.
///
/// This operation is usually costly to perform, as the aes round keys need to be generated from
/// the seed.
fn new(seed: Seed) -> Self;
/// Returns the number of bytes that can still be outputted by the generator before reaching its
/// bound.
///
/// Note:
/// -----
///
/// A fresh generator can generate 2¹³² bytes. Unfortunately, no rust integer type in is able
/// to encode such a large number. Consequently [`ByteCount`] uses the largest integer type
/// available to encode this value: the `u128` type. For this reason, this method does not
/// effectively return the number of remaining bytes, but instead
/// `min(2¹²⁸-1, remaining_bytes)`.
fn remaining_bytes(&self) -> ByteCount;
/// Returns the next byte of the stream, if the generator did not yet reach its bound.
fn next_byte(&mut self) -> Option<u8> {
self.next()
}
/// Tries to fork the generator into an iterator of `n_children` new generators, each able to
/// output `n_bytes` bytes.
///
/// Note:
/// -----
///
/// To be successful, the number of remaining bytes for the parent generator must be larger than
/// `n_children*n_bytes`.
fn try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ChildrenIter, ForkError>;
}
/// A trait extending [`RandomGenerator`] to the parallel iterators of `rayon`.
#[cfg(feature = "parallel")]
pub trait ParallelRandomGenerator: RandomGenerator + Send {
/// The iterator over children generators, returned by `par_try_fork` in case of success.
type ParChildrenIter: rayon::prelude::IndexedParallelIterator<Item = Self>;
/// Tries to fork the generator into a parallel iterator of `n_children` new generators, each
/// able to output `n_bytes` bytes.
///
/// Note:
/// -----
///
/// To be successful, the number of remaining bytes for the parent generator must be larger than
/// `n_children*n_bytes`.
fn par_try_fork(
&mut self,
n_children: ChildrenCount,
n_bytes: BytesPerChild,
) -> Result<Self::ParChildrenIter, ForkError>;
}
mod aes_ctr;
mod implem;
pub use implem::*;
#[cfg(test)]
pub mod generator_generic_test {
#![allow(unused)] // to please clippy when tests are not activated
use super::*;
use rand::Rng;
const REPEATS: usize = 1_000;
fn any_seed() -> impl Iterator<Item = Seed> {
std::iter::repeat_with(|| Seed(rand::thread_rng().gen()))
}
fn some_children_count() -> impl Iterator<Item = ChildrenCount> {
std::iter::repeat_with(|| ChildrenCount(rand::thread_rng().gen::<usize>() % 16 + 1))
}
fn some_bytes_per_child() -> impl Iterator<Item = BytesPerChild> {
std::iter::repeat_with(|| BytesPerChild(rand::thread_rng().gen::<usize>() % 128 + 1))
}
/// Checks that the PRNG roughly generates uniform numbers.
///
/// To do that, we perform an histogram of the occurrences of each byte value, over a fixed
/// number of samples and check that the empirical probabilities of the bins are close to
/// the theoretical probabilities.
pub fn test_roughly_uniform<G: RandomGenerator>() {
// Number of bins to use for the histogram.
const N_BINS: usize = u8::MAX as usize + 1;
// Number of samples to use for the histogram.
let n_samples = 10_000_000_usize;
// Theoretical probability of a each bins.
let expected_prob: f64 = 1. / N_BINS as f64;
// Absolute error allowed on the empirical probabilities.
// This value was tuned to make the test pass on an arguably correct state of
// implementation. 10^-4 precision is arguably pretty fine for this rough test, but it would
// be interesting to improve this test.
let precision = 10f64.powi(-3);
for _ in 0..REPEATS {
// We instantiate a new generator.
let seed = any_seed().next().unwrap();
let mut generator = G::new(seed);
// We create a new histogram
let mut counts = [0usize; N_BINS];
// We fill the histogram.
for _ in 0..n_samples {
counts[generator.next_byte().unwrap() as usize] += 1;
}
// We check that the empirical probabilities are close enough to the theoretical one.
counts
.iter()
.map(|a| (*a as f64) / (n_samples as f64))
.for_each(|a| assert!((a - expected_prob).abs() < precision))
}
}
/// Checks that given a state and a key, the PRNG is determinist.
pub fn test_generator_determinism<G: RandomGenerator>() {
for _ in 0..REPEATS {
let seed = any_seed().next().unwrap();
let mut first_generator = G::new(seed);
let mut second_generator = G::new(seed);
for _ in 0..1024 {
assert_eq!(first_generator.next(), second_generator.next());
}
}
}
/// Checks that forks returns a bounded child, and that the proper number of bytes can be
/// generated.
pub fn test_fork_children<G: RandomGenerator>() {
for _ in 0..REPEATS {
let ((seed, n_children), n_bytes) = any_seed()
.zip(some_children_count())
.zip(some_bytes_per_child())
.next()
.unwrap();
let mut gen = G::new(seed);
let mut bounded = gen.try_fork(n_children, n_bytes).unwrap().next().unwrap();
assert_eq!(bounded.remaining_bytes(), ByteCount(n_bytes.0 as u128));
for _ in 0..n_bytes.0 {
bounded.next().unwrap();
}
// Assert we are at the bound
assert!(bounded.next().is_none());
}
}
/// Checks that a bounded prng returns none when exceeding the allowed number of bytes.
///
/// To properly check for panic use `#[should_panic(expected = "expected test panic")]` as an
/// attribute on the test function.
pub fn test_bounded_none_should_panic<G: RandomGenerator>() {
let ((seed, n_children), n_bytes) = any_seed()
.zip(some_children_count())
.zip(some_bytes_per_child())
.next()
.unwrap();
let mut gen = G::new(seed);
let mut bounded = gen.try_fork(n_children, n_bytes).unwrap().next().unwrap();
assert_eq!(bounded.remaining_bytes(), ByteCount(n_bytes.0 as u128));
for _ in 0..n_bytes.0 {
assert!(bounded.next().is_some());
}
// One call too many, should panic
bounded.next().ok_or("expected test panic").unwrap();
}
}

114
concrete-csprng/src/lib.rs Normal file
View File

@@ -0,0 +1,114 @@
#![deny(rustdoc::broken_intra_doc_links)]
//! Cryptographically secure pseudo random number generator.
//!
//! Welcome to the `concrete-csprng` documentation.
//!
//! This crate provides a fast cryptographically secure pseudo-random number generator, suited to
//! work in a multithreaded setting.
//!
//! Random Generators
//! =================
//!
//! The central abstraction of this crate is the [`RandomGenerator`](generators::RandomGenerator)
//! trait, which is implemented by different types, each supporting a different platform. In
//! essence, a type implementing [`RandomGenerator`](generators::RandomGenerator) is a type that
//! outputs a new pseudo-random byte at each call to
//! [`next_byte`](generators::RandomGenerator::next_byte). Such a generator `g` can be seen as
//! enclosing a growing index into an imaginary array of pseudo-random bytes:
//! ```ascii
//! 0 1 2 3 4 5 6 7 8 9 M-1 │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃ │ │ │ │ │ │ │ │ │ │...│ ┃ │
//! ┗↥┷━┷━┷━┷━┷━┷━┷━┷━┷━┷━━━┷━┛ │
//! g │
//! │
//! g.next_byte() │
//! │
//! 0 1 2 3 4 5 6 7 8 9 M-1 │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃╳│ │ │ │ │ │ │ │ │ │...│ ┃ │
//! ┗━┷↥┷━┷━┷━┷━┷━┷━┷━┷━┷━━━┷━┛ │
//! g │
//! │
//! g.next_byte() │ legend:
//! │ -------
//! 0 1 2 3 4 5 6 7 8 9 M-1 │ ↥ : next byte to be outputted by g
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │ │ │: byte not yet outputted by g
//! ┃╳│╳│ │ │ │ │ │ │ │ │...│ ┃ │ │╳│: byte already outputted by g
//! ┗━┷━┷↥┷━┷━┷━┷━┷━┷━┷━┷━━━┷━┛ │
//! g 🭭
//! ```
//!
//! While being large, this imaginary array is still bounded to M = 2¹³² bytes. Consequently, a
//! generator is always bounded to a maximal index. That is, there is always a max amount of
//! elements of this array that can be outputted by the generator. By default, generators created
//! via [`new`](generators::RandomGenerator::new) are always bounded to M-1.
//!
//! Tree partition of the pseudo-random stream
//! ==========================================
//!
//! One particularity of this implementation is that you can use the
//! [`try_fork`](generators::RandomGenerator::try_fork) method to create an arbitrary partition tree
//! of a region of this array. Indeed, calling `try_fork(nc, nb)` outputs `nc` new generators, each
//! able to output `nb` bytes. The `try_fork` method ensures that the states and bounds of the
//! parent and children generators are set so as to prevent the same substream to be outputted
//! twice:
//! ```ascii
//! 0 1 2 3 4 5 6 7 8 9 M │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃P│P│P│P│P│P│P│P│P│P│...│P┃ │
//! ┗↥┷━┷━┷━┷━┷━┷━┷━┷━┷━┷━━━┷━┛ │
//! p │
//! │
//! (a,b) = p.fork(2,4) │
//! │
//! 0 1 2 3 4 5 6 7 8 9 M │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃A│A│A│A│B│B│B│B│P│P│...│P┃ │
//! ┗↥┷━┷━┷━┷↥┷━┷━┷━┷↥┷━┷━━━┷━┛ │
//! a b p │
//! │ legend:
//! (c,d) = b.fork(2, 1) │ -------
//! │ ↥ : next byte to be outputted by p
//! 0 1 2 3 4 5 6 7 8 9 M │ │P│: byte to be outputted by p
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │ │╳│: byte already outputted
//! ┃A│A│A│A│C│D│B│B│P│P│...│P┃ │
//! ┗↥┷━┷━┷━┷↥┷↥┷↥┷━┷↥┷━┷━━━┷━┛ │
//! a c d b p 🭭
//! ```
//!
//! This makes it possible to consume the stream at different places. This is particularly useful in
//! a multithreaded setting, in which we want to use the same generator from different independent
//! threads:
//!
//! ```ascii
//! 0 1 2 3 4 5 6 7 8 9 M │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃A│A│A│A│C│D│B│B│P│P│...│P┃ │
//! ┗↥┷━┷━┷━┷↥┷↥┷↥┷━┷↥┷━┷━━━┷━┛ │
//! a c d b p │
//! │
//! a.next_byte() │
//! │
//! 0 1 2 3 4 5 6 7 8 9 M │
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │
//! ┃│A│A│A│C│D│B│B│P│P│...│P┃ │
//! ┗━┷↥┷━┷━┷↥┷↥┷↥┷━┷↥┷━┷━━━┷━┛ │
//! a c d b p │
//! │ legend:
//! b.next_byte() │ -------
//! │ ↥ : next byte to be outputted by p
//! 0 1 2 3 4 5 6 7 8 9 M │ │P│: byte to be outputted by p
//! ┏━┯━┯━┯━┯━┯━┯━┯━┯━┯━┯━━━┯━┓ │ │╳│: byte already outputted
//! ┃│A│A│A│C│D││B│P│P│...│P┃ │
//! ┗━┷↥┷━┷━┷↥┷↥┷━┷↥┷↥┷━┷━━━┷━┛ │
//! a c d b p 🭭
//! ```
//!
//! Implementation
//! ==============
//!
//! The implementation is based on the AES blockcipher used in counter (CTR) mode, as presented
//! in the ISO/IEC 18033-4 document.
pub mod generators;
pub mod seeders;

View File

@@ -0,0 +1,141 @@
use crate::seeders::{Seed, Seeder};
use libc;
use std::cmp::Ordering;
/// There is no `rseed` equivalent in the ARM specification until `ARMv8.5-A`.
/// However it seems that these instructions are not exposed in `core::arch::aarch64`.
///
/// Our primary interest for supporting aarch64 targets is AppleSilicon support
/// which for the M1 macs available, they are based on the `ARMv8.4-A` set.
///
/// So we fall back to using a function from Apple's API which
/// uses the [Secure Enclave] to generate cryptographically secure random bytes.
///
/// [Secure Enclave]: https://support.apple.com/fr-fr/guide/security/sec59b0b31ff/web
mod secure_enclave {
pub enum __SecRandom {}
pub type SecRandomRef = *const __SecRandom;
use libc::{c_int, c_void};
#[link(name = "Security", kind = "framework")]
extern "C" {
pub static kSecRandomDefault: SecRandomRef;
pub fn SecRandomCopyBytes(rnd: SecRandomRef, count: usize, bytes: *mut c_void) -> c_int;
}
pub fn generate_random_bytes(bytes: &mut [u8]) -> std::io::Result<()> {
// As per Apple's documentation:
// - https://developer.apple.com/documentation/security/randomization_services?language=objc
// - https://developer.apple.com/documentation/security/1399291-secrandomcopybytes?language=objc
//
// The `SecRandomCopyBytes` "Generate cryptographically secure random numbers"
unsafe {
let res = SecRandomCopyBytes(
kSecRandomDefault,
bytes.len(),
bytes.as_mut_ptr() as *mut c_void,
);
if res != 0 {
Err(std::io::Error::last_os_error())
} else {
Ok(())
}
}
}
}
/// A seeder which uses the `SecRandomCopyBytes` function from Apple's `Security` framework.
///
/// <https://developer.apple.com/documentation/security/1399291-secrandomcopybytes?language=objc>
pub struct AppleSecureEnclaveSeeder;
impl Seeder for AppleSecureEnclaveSeeder {
fn seed(&mut self) -> Seed {
// 16 bytes == 128 bits
let mut bytes = [0u8; 16];
secure_enclave::generate_random_bytes(&mut bytes)
.expect("Failure while using Apple secure enclave: {err:?}");
Seed(u128::from_le_bytes(bytes))
}
fn is_available() -> bool {
let os_version_sysctl_name = match std::ffi::CString::new("kern.osproductversion") {
Ok(c_str) => c_str,
_ => return false,
};
// Big enough buffer to get a version output as an ASCII string
const OUTPUT_BUFFER_SIZE: usize = 64;
let mut output_buffer_size = OUTPUT_BUFFER_SIZE;
let mut output_buffer = [0u8; OUTPUT_BUFFER_SIZE];
let res = unsafe {
libc::sysctlbyname(
os_version_sysctl_name.as_ptr() as *const _ as *const _,
&mut output_buffer as *mut _ as *mut _,
&mut output_buffer_size as *mut _ as *mut _,
std::ptr::null_mut(),
0,
)
};
if res != 0 {
return false;
}
let result_c_str =
match std::ffi::CStr::from_bytes_with_nul(&output_buffer[..output_buffer_size]) {
Ok(c_str) => c_str,
_ => return false,
};
let result_string = match result_c_str.to_str() {
Ok(str) => str,
_ => return false,
};
// Normally we get a major version and minor version
let split_string: Vec<&str> = result_string.split('.').collect();
let mut major = -1;
let mut minor = -1;
// Major part of the version string
if !split_string.is_empty() {
major = match split_string[0].parse() {
Ok(major_from_str) => major_from_str,
_ => return false,
};
}
// SecRandomCopyBytes is available starting with mac OS 10.7
// https://developer.apple.com/documentation/security/1399291-secrandomcopybytes?language=objc
// This match pattern is recommended by clippy, so we oblige here
match major.cmp(&10) {
Ordering::Greater => true,
Ordering::Equal => {
// Minor part of the version string
if split_string.len() >= 2 {
minor = match split_string[1].parse() {
Ok(minor_from_str) => minor_from_str,
_ => return false,
};
}
minor >= 7
}
Ordering::Less => false,
}
}
}
#[cfg(test)]
mod test {
use super::*;
use crate::seeders::generic_tests::check_seeder_fixed_sequences_different;
#[test]
fn check_bounded_sequence_difference() {
check_seeder_fixed_sequences_different(|_| AppleSecureEnclaveSeeder);
}
}

View File

@@ -0,0 +1,14 @@
#[cfg(target_os = "macos")]
mod apple_secure_enclave_seeder;
#[cfg(target_os = "macos")]
pub use apple_secure_enclave_seeder::AppleSecureEnclaveSeeder;
#[cfg(feature = "seeder_x86_64_rdseed")]
mod rdseed;
#[cfg(feature = "seeder_x86_64_rdseed")]
pub use rdseed::RdseedSeeder;
#[cfg(feature = "seeder_unix")]
mod unix;
#[cfg(feature = "seeder_unix")]
pub use unix::UnixSeeder;

View File

@@ -0,0 +1,51 @@
use crate::seeders::{Seed, Seeder};
/// A seeder which uses the `rdseed` x86_64 instruction.
///
/// The `rdseed` instruction allows to deliver seeds from a hardware source of entropy see
/// <https://www.felixcloutier.com/x86/rdseed> .
pub struct RdseedSeeder;
impl Seeder for RdseedSeeder {
fn seed(&mut self) -> Seed {
Seed(unsafe { rdseed_random_m128() })
}
fn is_available() -> bool {
is_x86_feature_detected!("rdseed")
}
}
// Generates a random 128 bits value from rdseed
#[target_feature(enable = "rdseed")]
unsafe fn rdseed_random_m128() -> u128 {
let mut rand1: u64 = 0;
let mut rand2: u64 = 0;
let mut output_bytes = [0u8; 16];
unsafe {
loop {
if core::arch::x86_64::_rdseed64_step(&mut rand1) == 1 {
break;
}
}
loop {
if core::arch::x86_64::_rdseed64_step(&mut rand2) == 1 {
break;
}
}
}
output_bytes[0..8].copy_from_slice(&rand1.to_ne_bytes());
output_bytes[8..16].copy_from_slice(&rand2.to_ne_bytes());
u128::from_ne_bytes(output_bytes)
}
#[cfg(test)]
mod test {
use super::*;
use crate::seeders::generic_tests::check_seeder_fixed_sequences_different;
#[test]
fn check_bounded_sequence_difference() {
check_seeder_fixed_sequences_different(|_| RdseedSeeder);
}
}

View File

@@ -0,0 +1,72 @@
use crate::seeders::{Seed, Seeder};
use std::fs::File;
use std::io::Read;
/// A seeder which uses the `/dev/random` source on unix-like systems.
pub struct UnixSeeder {
counter: u128,
secret: u128,
file: File,
}
impl UnixSeeder {
/// Creates a new seeder from a user defined secret.
///
/// Important:
/// ----------
///
/// This secret is used to ensure the quality of the seed in scenarios where `/dev/random` may
/// be compromised.
///
/// The attack hypotheses are as follow:
/// - `/dev/random` output can be predicted by a process running on the machine by just
/// observing various states of the machine
/// - The attacker cannot read data from the process where `concrete-csprng` is running
///
/// Using a secret in `concrete-csprng` allows to generate values that the attacker cannot
/// predict, making this seeder secure on systems were `/dev/random` outputs can be
/// predicted.
pub fn new(secret: u128) -> UnixSeeder {
let file = std::fs::File::open("/dev/random").expect("Failed to open /dev/random .");
let counter = std::time::UNIX_EPOCH
.elapsed()
.expect("Failed to initialize unix seeder.")
.as_nanos();
UnixSeeder {
secret,
counter,
file,
}
}
}
impl Seeder for UnixSeeder {
fn seed(&mut self) -> Seed {
let output = self.secret ^ self.counter ^ dev_random(&mut self.file);
self.counter = self.counter.wrapping_add(1);
Seed(output)
}
fn is_available() -> bool {
cfg!(target_family = "unix")
}
}
fn dev_random(random: &mut File) -> u128 {
let mut buf = [0u8; 16];
random
.read_exact(&mut buf[..])
.expect("Failed to read from /dev/random .");
u128::from_ne_bytes(buf)
}
#[cfg(test)]
mod test {
use super::*;
use crate::seeders::generic_tests::check_seeder_fixed_sequences_different;
#[test]
fn check_bounded_sequence_difference() {
check_seeder_fixed_sequences_different(UnixSeeder::new);
}
}

View File

@@ -0,0 +1,50 @@
//! A module containing seeders objects.
//!
//! When initializing a generator, one needs to provide a [`Seed`], which is then used as key to the
//! AES blockcipher. As a consequence, the quality of the outputs of the generator is directly
//! conditioned by the quality of this seed. This module proposes different mechanisms to deliver
//! seeds that can accommodate varying scenarios.
/// A seed value, used to initialize a generator.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub struct Seed(pub u128);
/// A trait representing a seeding strategy.
pub trait Seeder {
/// Generates a new seed.
fn seed(&mut self) -> Seed;
/// Check whether the seeder can be used on the current machine. This function may check if some
/// required CPU features are available or if some OS features are available for example.
fn is_available() -> bool
where
Self: Sized;
}
mod implem;
// This import statement can be empty if seeder features are disabled, rustc's behavior changed to
// warn of empty modules, we know this can happen, so allow it.
#[allow(unused_imports)]
pub use implem::*;
#[cfg(test)]
mod generic_tests {
use crate::seeders::Seeder;
/// Naively verifies that two fixed-size sequences generated by repeatedly calling the seeder
/// are different.
#[allow(unused)] // to please clippy when tests are not activated
pub fn check_seeder_fixed_sequences_different<S: Seeder, F: Fn(u128) -> S>(
construct_seeder: F,
) {
const SEQUENCE_SIZE: usize = 500;
const REPEATS: usize = 10_000;
for i in 0..REPEATS {
let mut seeder = construct_seeder(i as u128);
let orig_seed = seeder.seed();
for _ in 0..SEQUENCE_SIZE {
assert_ne!(seeder.seed(), orig_seed);
}
}
}
}

View File

@@ -11,6 +11,7 @@ RUN sed -i 's|^deb http://archive.ubuntu.com/ubuntu/|deb http://mirror.ubuntu.ik
ENV CARGO_TARGET_DIR=/root/tfhe-rs-target
ARG RUST_TOOLCHAIN="stable"
ARG NODE_VERSION
WORKDIR /tfhe-wasm-tests
@@ -28,14 +29,12 @@ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > install-rustup.s
chmod +x install-rustup.sh && \
./install-rustup.sh -y --default-toolchain "${RUST_TOOLCHAIN}" \
-c rust-src -t wasm32-unknown-unknown && \
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf > install-wasm-pack.sh && \
chmod +x install-wasm-pack.sh && \
. "$HOME/.cargo/env" && \
./install-wasm-pack.sh -y && \
cargo install wasm-pack && \
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh > install-node.sh && \
chmod +x install-node.sh && \
./install-node.sh && \
. "$HOME/.nvm/nvm.sh" && \
bash -i -c 'nvm install node && nvm use node'
bash -i -c 'nvm install ${NODE_VERSION} && nvm use ${NODE_VERSION}'
WORKDIR /tfhe-wasm-tests/tfhe-rs/

Some files were not shown because too many files have changed in this diff Show More