Commit Graph

129 Commits

Author SHA1 Message Date
Arthur Meyre
30e00df977 chore(tools): update pylint target to lint dirs separately
- avoids triggering duplicate code detection between hdk tests and
benchmarks
- update helper script serialize_targets.sh to avoid printing dir
2021-08-24 16:00:42 +02:00
Umut
1d5be5a1e7 refactor: re-organize data types and values 2021-08-24 16:13:36 +03:00
Umut
4b3fb772b8 chore: integrate checking whether notebooks are sanitized into the pcc target 2021-08-24 10:56:54 +03:00
Umut
2b5f7f3118 refactor: rename ConstantInput to Constant to reduce confusion 2021-08-24 10:48:57 +03:00
Arthur Meyre
4e658c15cb feat: add TensorValue 2021-08-24 09:31:17 +02:00
Arthur Meyre
985cf973d8 refactor: values.py
- change import for BaseDataType
- create helper functions rather than partials to have docstrings in IDE
hints
2021-08-24 09:31:17 +02:00
Arthur Meyre
0e1534637b fix: BaseValue now deepcopies the data_type it receives
- avoids weird situations with refs to a type that gets modified
2021-08-24 09:31:17 +02:00
Arthur Meyre
e41f77349f refactor: make BaseDataType __eq__ abstract
- update test files with dummy dtypes
2021-08-24 09:31:17 +02:00
Arthur Meyre
7a0f11b1b0 refactor: change IR and BaseTracer to support any input mixing function
- prepares support for tensor types
- introduce _n_in and n_in() and requires_mix_values_func() for
IntermediateNode to know if they require a function to mix input values and
determine output value
- update BaseTracer to pass the class _mix_values_func to IntermediateNodes
that need it
- update NPTracer to stay consistent with current behavior regarding values
mixing
2021-08-23 14:10:41 +02:00
Arthur Meyre
78480e5da7 refactor: make IR agnostic of clear/encypted values 2021-08-23 14:10:41 +02:00
Arthur Meyre
9a0c108d4b refactor: refactor ConstantInput to be flexible
- refactor to take a function to generate the propore BaseValue to store in
its output
- refactor BaseTracer to force inheriting tracers to indicate how to build
a ConstantInput tracer
- remove "as import" for intermediate in hnumpy/tracing.py
- update compile to manage python dtypes
2021-08-23 14:10:41 +02:00
Arthur Meyre
c528d72e62 refactor: remove Scalars type hint
- add a helper function to determine BaseDataType of a constant python
scalar, int or float in dtype_helpers.py
- make BaseTracer type agnostic
- make ConstantInput type agnostic
2021-08-23 14:10:41 +02:00
Arthur Meyre
5e258ca443 refactor: change type hints in IntermediateNode evaluate 2021-08-23 14:10:41 +02:00
Arthur Meyre
5048992707 refactor: properly manage unhandled operands in BaseTracer and NPTracer
- use the pythonic way of signaling unhandled operands
- change typing hint in NPTracer
2021-08-23 14:10:41 +02:00
Arthur Meyre
a4181afe4d refactor: refactor np_dtype_helpers.py
- rename convert_numpy_dtype_to_common_dtype to
convert_numpy_dtype_to_base_data_type
- change names of constants to be clearer
2021-08-23 14:10:41 +02:00
Arthur Meyre
4f103e604a refactor: update OPGraph to be able to update bounds with foreign types
- get BaseDataType for values being checked in update_values_with_bounds
- rename SUPPORTED_TYPES to BASE_DATA_TYPES
2021-08-23 14:10:41 +02:00
Arthur Meyre
60daf31981 refactor: refactor make_integer_to_hold_ints
- rename make_integer_to_hold_ints to make_integer_to_hold
- accept any values as input as we don't know which type this function will
be called with
- rename get_bits_to_represent_int to
get_bits_to_represent_value_as_integer
2021-08-23 14:10:41 +02:00
Arthur Meyre
0ff3ae4795 refactor: refactor BaseValue and mix_values_determine_holding_dtype
- add _is_encrypted to BaseValue
- remove EncryptedValue and ClearValue classes
- add a ScalarValue class
- add two helpers EncryptedValue and ClearValue which create a ScalarValue
either encrypted or not when passed a data_type
- rename to mix_scalar_values_determine_holding_dtype
- change typing
2021-08-23 14:10:41 +02:00
Arthur Meyre
371ecae801 tests: do not test BaseValue __repr__ anymore 2021-08-23 14:10:41 +02:00
Arthur Meyre
f91092ac38 tools: update pytest make target to have a report on coverage
- allows to have coverage information without using make coverage
- make coverage remains recommended to check a commit's content
2021-08-20 09:53:02 +02:00
Arthur Meyre
6959fdf7e7 chore(tools): only make setup_env in dev docker if venv does not exist 2021-08-20 09:46:33 +02:00
Umut
94fef5e202 feat(debugging): provide a way to export the drawn graph as a png file 2021-08-18 17:33:19 +03:00
Arthur Meyre
0c5d5b0fd0 docs: change ordering of attributes and methods to be grouped
- alphabetical makes the docs hard to read
2021-08-18 14:30:37 +02:00
Umut
a367d68c6e feat(compilation-configuration): make compilation customizable 2021-08-18 15:10:58 +03:00
Umut
1b33cd7307 feat(Makefile): enable jupyter notebook support in docker environment 2021-08-18 14:49:09 +03:00
Arthur Meyre
5d26ad499a fix(build): properly serialize calls to mypy
- add an helper script to serialize make commands
- serialize mypy commands as they may overwrite each others cache
- format coverage command to avoid being ridiculously long
2021-08-18 12:23:53 +02:00
Arthur Meyre
5038502327 chore(test): make printable graph tests less verbose but keep debug info 2021-08-18 11:03:14 +02:00
Umut
c79e232d0f fix(Dockerfile): add LD_PRELOAD export to .bashrc of the docker environment 2021-08-18 11:31:06 +03:00
Umut
2dda00aeef chore(Makefile): add -s flag to pytest to improve the output of compiler tests 2021-08-18 11:31:06 +03:00
Arthur Meyre
ed66981ccd chore(reqs): move dev requirements to dev category 2021-08-18 09:36:03 +02:00
Arthur Meyre
6d14b4b318 fix(build): manage refs for push events 2021-08-18 09:08:49 +02:00
youben11
788e94bfa3 feat: end to end compilation and execution 2021-08-18 08:08:19 +01:00
Arthur Meyre
4e40982f5a feat(float-fusing): fuse float parts of an OPGraph during compilation
- this allows to be compatible with the current compiler and squash float
domains into a single int to int ArbitraryFunction
2021-08-17 18:27:31 +02:00
Arthur Meyre
d48c4dba32 dev(NPTracer): add support for sin
- re-organize numpy tracing tests

refs #126
2021-08-17 18:27:31 +02:00
Arthur Meyre
825d6422d0 dev(NPTracer): add op_name for traced functions, deepcopy kwargs
- ir.ArbitraryFunction does not deepcopy op_args and op_kwargs by default
anymore to let the control to the developer instantiating it
2021-08-17 18:27:31 +02:00
Arthur Meyre
0eebbfcd26 dev(opgraph): add facilities to OPGraph
- allow to construct graph from an existing networkx MultiDiGraph
- add a function to remove nodes unreachable from the outputs of the graph
- return the evaluated output when calling the OPGraph
2021-08-17 18:27:31 +02:00
Arthur Meyre
4976855c1d fix(debug): manage all get_color cases and unsorted arg names 2021-08-17 18:27:31 +02:00
Arthur Meyre
63eac35a43 build: launch pcc targets in parallel to speed up checks
- change call in github actions as proper flags are in the Makefile
- change mypy_ci target to avoid cache issues with multiple mypy instances
2021-08-17 18:27:31 +02:00
Umut
553f77f19b chore(ci): implement publishing docs to hdk.zama.ai on push to main 2021-08-17 18:52:27 +03:00
Umut
8df212ff49 bench: create the benchmarking infrastructure 2021-08-17 13:17:28 +03:00
Arthur Meyre
8dfed58829 chore(tools): add flake8 linter, flake8-bugbear plugin and fix issues
- remove detected `from hdk imports` (use relative imports instead) we ARE
the package
- change the way Float32 and Float64 are defined
2021-08-17 09:22:39 +02:00
youben11
3922bfe9b4 feat(mlir): support constant inputs in mlir conversion 2021-08-16 14:58:56 +01:00
youben11
8fbe5dab4d fix(mlir): unsigned are considered signless in compiler
+ changed the name of compiled func to main, as it's the default name to
  be executed later
2021-08-16 14:58:56 +01:00
Umut
479176e368 doc(examples): create quantized linear and logistic regression examples 2021-08-16 15:57:18 +03:00
Arthur Meyre
048bb61e8e fix(tools): add git to the docker image to be able to make coverage
- move comment in Makefile and make it silent
2021-08-13 18:11:43 +02:00
Benoit Chevallier
3245d3e673 Feat/user friendly arbitrary function name 144 (#149)
* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

* feat: let the dev give a useful name for ArbitraryFunction

might be useful to debug or understand what happens
closes #144

Co-authored-by: Benoit Chevallier-Mames <benoitchevalliermames@zama.ai>
2021-08-13 18:10:35 +02:00
Umut
5961d1630e refactor(compilation-artifacts): enable showing data types during export 2021-08-13 15:15:54 +03:00
Ayoub Benaissa
f6c9618b5a feat(mlir): MLIR Conversion (#103)
* feat(mlir): conversion from HDKIR to MLIR

* feat(mlir): support ir.Sub and ir.Mul

- better type conversion from HDK to MLIR
- Context management inside the converter class
- better handling of input type in conversion functions

* refactor(mlir): use input and output from OPGraph

Co-authored-by: Arthur Meyre <arthur.meyre@zama.ai>

* feat(mlir): eint-int subtractions

* feat(mlir): adhere to spec for supported ops

* feat(OPGraph): getters for ordered inputs/outputs

+ formatting

* tests(mlir): test converion via compiler roundtrip

* fix(mlir): flip operands on int_eint sym ops

* feat(mlir): check that the outputs are unsigned

* feat(mlir): set bit_width of all nodes to the max

This is currently required as the compiler is already assuming this.
Could be removed from HDK when the compiler can do it on its own

* feat: value_is_integer + CRs

disable some linting errors

* tests: update compile tests + coverage

* refactor: reorganize mlir package + better doc

* doc: conformance with pydocstyle

Co-authored-by: Arthur Meyre <arthur.meyre@zama.ai>
2021-08-13 12:50:31 +01:00
Umut
2c3c080923 test(direct-tlu): create debugging and compilation tests for direct table lookup 2021-08-12 17:41:06 +03:00
Benoit Chevallier
4afc373a6b Update GETTING-STARTED.md
Adding the group name
2021-08-12 15:51:35 +02:00