diff --git a/docs/dev/explanation/COMPILATION.md b/docs/dev/explanation/COMPILATION.md index 0dd09ab7d..dfa9ca971 100644 --- a/docs/dev/explanation/COMPILATION.md +++ b/docs/dev/explanation/COMPILATION.md @@ -1,3 +1,9 @@ + +```{warning} +FIXME(umut): update with the new API +``` + + # Compilation Pipeline In Depth ## What is **concretefhe**? @@ -54,6 +60,10 @@ Once the MLIR is prepared, the rest of the stack, which you can learn more about Here is the visual representation of the pipeline: +```{warning} +FIXME(arthur): check the graph, update with what is missing, notably: torch to numpy, quantization. Maybe an independant graph for quantization may be clearer. +``` + ![Frontend Flow](../../_static/compilation-pipeline/frontend_flow.svg) ## Tracing diff --git a/docs/dev/explanation/FLOAT-FUSING.md b/docs/dev/explanation/FLOAT-FUSING.md index 20e6104bf..63bc2699d 100644 --- a/docs/dev/explanation/FLOAT-FUSING.md +++ b/docs/dev/explanation/FLOAT-FUSING.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Arthur): explain recent updates on the fusing +``` + # Fusing Floating Point Operations ## Why is it needed? diff --git a/docs/dev/explanation/QUANTIZATION.md b/docs/dev/explanation/QUANTIZATION.md new file mode 100644 index 000000000..e9c136bbf --- /dev/null +++ b/docs/dev/explanation/QUANTIZATION.md @@ -0,0 +1,8 @@ +```{warning} +FIXME(Jordan/Andrei): do this +``` + +# Quantization in Depth + +(as a researcher) + diff --git a/docs/dev/explanation/TERMINOLOGY_AND_STRUCTURE.md b/docs/dev/explanation/TERMINOLOGY_AND_STRUCTURE.md index f1934029c..914ddf281 100644 --- a/docs/dev/explanation/TERMINOLOGY_AND_STRUCTURE.md +++ b/docs/dev/explanation/TERMINOLOGY_AND_STRUCTURE.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Benoit): to check this is still valid +``` + # Terminology and Structure ## Terminology diff --git a/docs/dev/howto/CONTRIBUTING.md b/docs/dev/howto/CONTRIBUTING.md index 128ce56b6..7c6c5fcf4 100644 --- a/docs/dev/howto/CONTRIBUTING.md +++ b/docs/dev/howto/CONTRIBUTING.md @@ -1,6 +1,10 @@ # Contributing +```{warning} +FIXME(alex): to see if something here needs some update +``` + ```{important} There are two ways to contribute to **concretefhe**: - you can open issues to report bugs, typos and suggest ideas @@ -51,6 +55,10 @@ The last requirement is to make sure you get a hundred percent code coverage. Yo make coverage ``` +```{warning} +FIXME(arthur): is this still valid? (that `make pytest` will not return an error) +``` + Remark that only calling `make pytest` will give you information about the coverage, at the end of the execution, but the test will not return a failure if the coverage is not a hundred percent, as opposed to a call to `make coverage`. Note that this will compare the coverage with `origin/main`. If you want to set a custom base branch, you can specify `BB` environment variable like so `BB=$YOUR_BASE_BRANCH make coverage`. @@ -73,7 +81,7 @@ git commit -m "feat(debugging): add an helper function to draw intermediate repr git commit -m "fix(tracing): fix a bug that crashed pytorch tracer" ``` -To learn more about conventional commits, check [this](https://www.conventionalcommits.org/en/v1.0.0/) page. +To learn more about conventional commits, check [this](https://www.conventionalcommits.org/en/v1.0.0/) page. Remark that commit messages are checked in the comformance step, and rejected if they don't follow the rules. ## Before creating pull request diff --git a/docs/dev/howto/DOCKER.md b/docs/dev/howto/DOCKER.md index fbbe5ebcc..5ade12039 100644 --- a/docs/dev/howto/DOCKER.md +++ b/docs/dev/howto/DOCKER.md @@ -1,3 +1,10 @@ +```{warning} +FIXME(arthur): to update if needed +``` +```{warning} +FIXME(arthur): to add a new .md about pypy if needed +``` + # Docker ## Setting up docker and X forwarding diff --git a/docs/dev/howto/PROJECT_SETUP.md b/docs/dev/howto/PROJECT_SETUP.md index affd6afca..af39039c6 100644 --- a/docs/dev/howto/PROJECT_SETUP.md +++ b/docs/dev/howto/PROJECT_SETUP.md @@ -1,3 +1,6 @@ +```{warning} +FIXME(Arthur): to check what needs to be updated here +``` # Project Setup diff --git a/docs/dev/howto/RELEASING.md b/docs/dev/howto/RELEASING.md index 98955c483..b822262f6 100644 --- a/docs/dev/howto/RELEASING.md +++ b/docs/dev/howto/RELEASING.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(arthur): to update with the new workflow +``` + # Creating A Release On GitHub ## Release Candidate cycle diff --git a/docs/dev/index.rst b/docs/dev/index.rst index b59c79c31..9d8cd2b71 100644 --- a/docs/dev/index.rst +++ b/docs/dev/index.rst @@ -19,3 +19,4 @@ Developer Guide explanation/TERMINOLOGY_AND_STRUCTURE.md explanation/FLOAT-FUSING.md explanation/MLIR.md + explanation/QUANTIZATION.md \ No newline at end of file diff --git a/docs/user/README.md b/docs/user/README.md index a5c35013e..584de7408 100644 --- a/docs/user/README.md +++ b/docs/user/README.md @@ -17,11 +17,16 @@ FHE is also a killer feature regarding data breaches: as anything done on the se In the first version of Concrete, there is a single frontend, called homomorphic numpy (or hnp), which is the equivalent of numpy. With our toolchain, a data scientist can convert a numpy program into an FHE program, without any a-priori knowledge on cryptography. ``` +```{note} +On top of the numpy frontend, we are adding an alpha-version of a torch compiler, which basically transforms a subset of torch modules into numpy, and then use numpy frontend and the compiler. This is an early version of a more stable torch compiler which will be released later in the year. +``` + ## Organization of the documentation Basically, we have divided our documentation into several parts: - one about basic elements, notably description of the installation, that you are currently reading - one dedicated to _users_ of **Concrete**, with tutorials, how-to's and deeper explanations +- one detailing the API's of the different functions of the frontend, directly done by parsing its source code - and finally, one dedicated to _developers_ of **Concrete**, who could be internal or external contributors to the framework ## A work in progress @@ -33,8 +38,13 @@ Concrete is a work in progress, and is currently limited to a certain number of The main _current_ limits are: - **Concrete** is only supporting unsigned integers - **Concrete** needs the integer to be less than 7 bits (included) -- **Concrete** is mostly restricted to scalars (by opposition to tensors). The only exception is the `dot` operator, which can dot a tensor of encrypted values with a tensor of constant values -The first two limits can be taken care of with the use of quantization, as explained a bit further in [this](explanation/QUANTIZATION.md) and [this](howto/REDUCE_NEEDED_PRECISION.md) parts of the documentation. +These limits can be taken care of with the use of quantization, as explained a bit further in [this](explanation/QUANTIZATION.md) and [this](howto/REDUCE_NEEDED_PRECISION.md) parts of the documentation. -The scalar limitation is mainly an engineering issue, and will be fixed in the next release. Today, one needs to split all the tensors into small scalars, which is inconvenient and will be no more needed very soon. +```{warning} +FIXME(Jordan): speak about our quantization framework +``` + +```{warning} +FIXME(Jordan/Andrei): add an .md about the repository of FHE-friendly models, and ideally .ipynb's +``` diff --git a/docs/user/advanced_examples/QuantizedGeneralizedLinearModel.ipynb b/docs/user/advanced_examples/QuantizedGeneralizedLinearModel.ipynb new file mode 100644 index 000000000..2090d4b8a --- /dev/null +++ b/docs/user/advanced_examples/QuantizedGeneralizedLinearModel.ipynb @@ -0,0 +1,775 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b760a0f6", + "metadata": {}, + "source": [ + "# FIXME(Andrei): To be done with 979\n", + "\n", + "FIXME(Andrei): to be done with 979!" + ] + }, + { + "cell_type": "markdown", + "id": "253288cf", + "metadata": {}, + "source": [ + "### Let's start by importing some libraries to develop our linear regression model" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "6200ab62", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "markdown", + "id": "f43e2387", + "metadata": {}, + "source": [ + "### And some helpers for visualization" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "d104c8df", + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "import matplotlib.pyplot as plt\n", + "from IPython.display import display" + ] + }, + { + "cell_type": "markdown", + "id": "53e676b8", + "metadata": {}, + "source": [ + "### We need an inputset, a handcrafted one for simplicity" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d451e829", + "metadata": {}, + "outputs": [], + "source": [ + "x = np.array([[69], [130], [110], [100], [145], [160], [185], [200], [80], [50]], dtype=np.float32)\n", + "y = np.array([181, 325, 295, 268, 400, 420, 500, 520, 220, 120], dtype=np.float32)" + ] + }, + { + "cell_type": "markdown", + "id": "75f4fdb7", + "metadata": {}, + "source": [ + "### Let's visualize our inputset to get a grasp of it" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "2a124a62", + "metadata": {}, + "outputs": [], + "source": [ + "plt.ioff()\n", + "fig, ax = plt.subplots(1)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "edcd361b", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAW+klEQVR4nO3dfZBddX3H8feHp4iKXCArjUnaoMY62BlDusY4WIuhKqBjcMY66ViJlE7UiZ3L6ihEZ+o6U2bEp3WZtjiRKEEpSBFLhsGpCKGOfxC6gRASImUVMFkDWYSNUqapYb/94/y2Obvsw717n86e/bxm7txzfufc3e+e5H733O/+HhQRmJlZuRzX6QDMzKz5nNzNzErIyd3MrISc3M3MSsjJ3cyshE7odAAACxcujGXLlnU6DDOzOWXnzp3PRETXZMcKkdyXLVvGwMBAp8MwM5tTJD051TGXZczMSsjJ3cyshJzczcxKyMndzKyEnNzNzDph4rxeTZ7ny8ndzKzdenuhp+dYQo/I9nt7m/YtnNzNzNopAkZGoL//WILv6cn2R0aadgdfiH7uZmbzhgR9fdl2f3/2AKhWs3apOd+mCPO5d3d3hwcxmdm8EgHH5Yono6N1J3ZJOyOie7JjLsuYmbXbWCkmL1+DbwIndzOzdsrX2KvV7I69Wh1fg28C19zNzNpJgkplfI19rAZfqbjmbmY2p0WMT+QT92vgmruZWdFMTORNumMf4+RuZvNXi0eJdlJNyV3SE5IelrRL0kBqO13SXZIeS8+npXZJukbSoKTdkla28gcwM5uVNowS7aR67tzfGRErcvWdK4G7I2I5cHfaB7gQWJ4eG4BrmxWsmVlTtGmUaCc10ltmLXBe2t4K3AtckdpviOwvtfdJqkhaFBEHGwnUzKxp2jRKtJNqvXMP4MeSdkrakNrOzCXsp4Az0/ZiYH/utQdS2ziSNkgakDQwPDw8i9DNzBqQT/BjSpLYofbk/vaIWElWctko6R35g+kuva7PMRGxOSK6I6K7q2vS9V3NzFqnDaNEO6mm5B4RQ+n5EPBDYBXwtKRFAOn5UDp9CFiae/mS1GZmVgxtGiXaSTMmd0mvkHTK2DbwbmAPsA1Yn05bD9yetrcBl6ReM6uBw663m1mhTDVKtFpt6ijRTppxhKqk15LdrUP2B9h/iYirJJ0B3AL8IfAk8KGIeFaSgH8ELgBeAC6NiGmHn3qEqpl1RBNGiXbSdCNUZ+wtExG/BN48SftvgPMnaQ9g4yziNDNrrxaPEu0kj1A1MyshJ3czsxJycjczKyEndzOzEnJyNzMrISd3M7MScnI3MyshJ3czsxJycjczKyEndzOzEnJyNzMrISd3M7MScnI3MyshJ3czsxJycjczKyEndzOzEqo5uUs6XtKDku5I+9dLelzSrvRYkdol6RpJg5J2S1rZotjNzGwKM67ElFMF9gGvyrV9JiJunXDehcDy9HgrcG16NjOzNqnpzl3SEuC9wHU1nL4WuCEy9wEVSYsaiNHMzOpUa1nmG8BngdEJ7Vel0kufpAWpbTGwP3fOgdQ2jqQNkgYkDQwPD9cZtpmZTWfG5C7pfcChiNg54dAm4I3AW4DTgSvq+cYRsTkiuiOiu6urq56XmpnZDGq5cz8XeL+kJ4CbgTWSvhcRB1Pp5QjwHWBVOn8IWJp7/ZLUZmZmbTJjco+ITRGxJCKWAeuAeyLir8fq6JIEXAzsSS/ZBlySes2sBg5HxMGWRG9mZpOqp7fMRDdK6gIE7AI+ntrvBC4CBoEXgEsbCdDMzOpXV3KPiHuBe9P2minOCWBjo4GZmdnseYSqmVkJObmbmZWQk7uZWQk5uZuZlZCTu5lZCTm5m1n9Iqbft45zcjez+vT2Qk/PsYQeke339nYyKpvAyd3MahcBIyPQ338swff0ZPsjI76DL5BGRqia2XwjQV9ftt3fnz0AqtWsXepcbDaOogC/abu7u2NgYKDTYZhZrSLguNwH/9FRJ/YOkLQzIronO+ayjJnVZ6wUk5evwVshOLmbWe3yNfZqNbtjr1bH1+CtEFxzN7PaSVCpjK+xj9XgKxWXZgrENXczq1/E+EQ+cd/awjV3M2uuiYncib1wak7uko6X9KCkO9L+WZJ2SBqU9H1JJ6X2BWl/MB1f1qLYzeY3jxK1adRz514F9uX2rwb6IuL1wHPAZan9MuC51N6XzjOzZvIoUZtBTcld0hLgvcB1aV/AGuDWdMpWsnVUAdamfdLx89P5ZtYMHiVqNai1t8w3gM8Cp6T9M4CRiDia9g8Ai9P2YmA/QEQclXQ4nf9MMwI2m/c8StRqMOOdu6T3AYciYmczv7GkDZIGJA0MDw8380ublV8+wY9xYrecWsoy5wLvl/QEcDNZOaYfqEgau/NfAgyl7SFgKUA6firwm4lfNCI2R0R3RHR3dXU19EOYzTseJWozmDG5R8SmiFgSEcuAdcA9EfFhYDvwwXTaeuD2tL0t7ZOO3xNF6ExvVhYeJWo1aGSE6hXAzZL+AXgQ2JLatwDflTQIPEv2C8HMmsWjRK0GHqFqNld5lOi85xGqZmXkUaI2DSd3M7MScnI3MyshJ3czsxJycjczKyEndzOzEnJyNzMrISd3M7MScnI3MyshJ3czsxJycjczKyEndzOzEnJyNzMrISd3M7MScnI3MyshJ3czsxKqZYHsl0m6X9JDkvZK+mJqv17S45J2pceK1C5J10galLRb0soW/wxmZjZBLcvsHQHWRMTzkk4EfibpR+nYZyLi1gnnXwgsT4+3AtemZzMza5NaFsiOiHg+7Z6YHtOtzbcWuCG97j6gImlR46GamVmtaqq5Szpe0i7gEHBXROxIh65KpZc+SQtS22Jgf+7lB1LbxK+5QdKApIHh4eHZ/wRmZvYSNSX3iHgxIlYAS4BVkv4E2AS8EXgLcDpwRT3fOCI2R0R3RHR3dXXVF7WZmU2rrt4yETECbAcuiIiDqfRyBPgOsCqdNgQszb1sSWozM7M2qaW3TJekSto+GXgX8POxOrokARcDe9JLtgGXpF4zq4HDEXGwBbGbmdkUauktswjYKul4sl8Gt0TEHZLukdQFCNgFfDydfydwETAIvABc2vSozcxsWjMm94jYDZwzSfuaKc4PYGPjoZmZ2Wx5hKqZWQk5uZuZlZCTu5lZCTm5mzUqYvp9sw5wcjdrRG8v9PQcS+gR2X5vbyejMnNyN5u1CBgZgf7+Ywm+pyfbHxnxHbx1VC393M1sMhL09WXb/f3ZA6BazdqlzsVm856iAHcX3d3dMTAw0OkwzGYnAo7LfQgeHXVit7aQtDMiuic75rKMWSPGSjF5+Rq8WYc4uZvNVr7GXq1md+zV6vgavFmHuOZuNlsSVCrja+xjNfhKxaUZ6yjX3M0aFTE+kU/cN2sR19zNWmliInditwJwcjczKyEndzOzEnJyNzMroVqW2XuZpPslPSRpr6QvpvazJO2QNCjp+5JOSu0L0v5gOr6sxT+DmZlNUMud+xFgTUS8GVgBXJDWRr0a6IuI1wPPAZel8y8Dnkvtfek8s9nxjItmszJjco/M82n3xPQIYA1wa2rfSrZINsDatE86fn5aRNusPp5x0WzWaqq5Szpe0i7gEHAX8AtgJCKOplMOAIvT9mJgP0A6fhg4Y5KvuUHSgKSB4eHhhn4IKyHPuGjWkJpGqEbEi8AKSRXgh8AbG/3GEbEZ2AzZIKZGv56VjGdcNGtIXb1lImIE2A68DahIGvvlsAQYSttDwFKAdPxU4DfNCNbmmXyCH+PEblaTWnrLdKU7diSdDLwL2EeW5D+YTlsP3J62t6V90vF7oghzHNjc4xkXzWatljv3RcB2SbuB/wTuiog7gCuAT0kaJKupb0nnbwHOSO2fAq5sfthWep5x0awhM9bcI2I3cM4k7b8EVk3S/j/AXzYlOpu/POOiWUM8K6QVm2dcNJuSZ4W0ucszLprNipO7mVkJObmbmZWQk7uZWQk5uZuZlZCTuzWXZ3E0KwQnd2sez+JoVhhO7tYcnsXRrFBqmhXSbEaexdGsUDxC1ZorAo7LfSAcHXViN2sRj1C19vAsjmaF4eRuzeFZHM0KxTV3aw7P4mhWKK65W3N5FkeztnHN3drHsziaFUIty+wtlbRd0iOS9kqqpvZeSUOSdqXHRbnXbJI0KOlRSe9p5Q9gZmYvVUvN/Sjw6Yh4QNIpwE5Jd6VjfRHx1fzJks4G1gFvAl4D/ETSGyLixWYGbmZmU5vxzj0iDkbEA2n7d2SLYy+e5iVrgZsj4khEPA4MMslyfGZm1jp11dwlLSNbT3VHavqkpN2Svi3ptNS2GNife9kBJvllIGmDpAFJA8PDw/VHbmZmU6o5uUt6JfAD4PKI+C1wLfA6YAVwEPhaPd84IjZHRHdEdHd1ddXzUjMzm0FNyV3SiWSJ/caIuA0gIp6OiBcjYhT4FsdKL0PA0tzLl6Q2MzNrk1p6ywjYAuyLiK/n2hflTvsAsCdtbwPWSVog6SxgOXB/80I2M7OZ1NJb5lzgI8DDknalts8BfyVpBRDAE8DHACJir6RbgEfIetpsdE8ZM7P2mjG5R8TPgMlGotw5zWuuAq5qIC4zM2uAR6iamZWQk7uZWQk5uZuZlZCTu5lZCTm5zyUTp2cuwHTNZlZMTu5zRW/v+BWNxlY+6u3tZFRmVlBO7nNBBIyMjF+ybmxJu5ER38Gb2Ut4mb25IL9kXX9/9oDxS9qZmeV4mb25JAKOy33YGh11Yjebx7zMXhmMlWLy8jV4M7McJ/e5IF9jr1azO/ZqdXwN3swsxzX3uUCCSmV8jX2sBl+puDRjZi/hmvtcEjE+kU/cN7N5xTX3spiYyJ3YzWwKTu5mZiVUy0pMSyVtl/SIpL2Sqqn9dEl3SXosPZ+W2iXpGkmDafHsla3+IczMbLxa7tyPAp+OiLOB1cBGSWcDVwJ3R8Ry4O60D3Ah2dJ6y4ENZAtpm5lZG82Y3CPiYEQ8kLZ/B+wDFgNrga3ptK3AxWl7LXBDZO4DKhPWWzUzsxarq+YuaRlwDrADODMiDqZDTwFnpu3FwP7cyw6ktolfa4OkAUkDw8PD9cZtZmbTqDm5S3ol8APg8oj4bf5YZP0p6+pTGRGbI6I7Irq7urrqeamZmc2gpuQu6USyxH5jRNyWmp8eK7ek50OpfQhYmnv5ktRmZmZtUktvGQFbgH0R8fXcoW3A+rS9Hrg9135J6jWzGjicK9+YmVkb1DL9wLnAR4CHJe1KbZ8DvgTcIuky4EngQ+nYncBFwCDwAnBpMwM2M7OZzZjcI+JnwFRDIc+f5PwANjYYl5mZNcAjVFvBa52aWYc5uTeb1zo1swJwcm8mr3VqZgXh+dybyWudmllBeD73VvBap2bWBp7PvZ281qmZFYCTezN5rVMzKwjX3JvJa52aWUG45t4KXuvUzNrANfd281qnZtZhTu5mZiXk5G5mVkJO7mZmJeTkbmZWQk7uZmYl5ORuZlZCtSyz921JhyTtybX1ShqStCs9Lsod2yRpUNKjkt7TqsDNzGxqtdy5Xw9cMEl7X0SsSI87ASSdDawD3pRe88+Sjm9WsGZmVpsZk3tE/BR4tsavtxa4OSKORMTjZOuormogPjMzm4VGau6flLQ7lW1OS22Lgf25cw6ktpeQtEHSgKSB4eHhBsIwM7OJZpvcrwVeB6wADgJfq/cLRMTmiOiOiO6urq5ZhmFmZpOZVXKPiKcj4sWIGAW+xbHSyxCwNHfqktRmZmZtNKvkLmlRbvcDwFhPmm3AOkkLJJ0FLAfubyxEMzOr14zzuUu6CTgPWCjpAPAF4DxJK4AAngA+BhAReyXdAjwCHAU2RsSLLYnczMym5PnczczmKM/nbmY2zzi5m5mVkJO7mVkJObmbmZWQk7uZWQnN3eQ+sZdPAXr9mJkVxdxM7r290NNzLKFHZPu9vZ2MysysMOZeco+AkRHo7z+W4Ht6sv2REd/Bm5lRwwjVwpGgry/b7u/PHgDVatYudS42M7OCmLsjVCPguNwHj9FRJ3Yzm1fKN0J1rBSTl6/Bm5nNc3Mvuedr7NVqdsderY6vwZuZzXNzs+ZeqYyvsY/V4CsVl2bMzJjrNfd8Ip+4b2ZWcuWrucNLE7kTu5nZ/5u7yd3MzKY0Y3KX9G1JhyTtybWdLukuSY+l59NSuyRdI2lQ0m5JK1sZvJmZTa6WO/frgQsmtF0J3B0Ry4G70z7AhWTrpi4HNgDXNidMMzOrx4zJPSJ+Cjw7oXktsDVtbwUuzrXfEJn7gMqExbTNzKwNZtsV8syIOJi2nwLOTNuLgf258w6ktoNMIGkD2d09wPOSHp1lLK2wEHim00FMo+jxQfFjLHp84BiboejxQWMx/tFUBxru5x4RIanu/pQRsRnY3Oj3bwVJA1N1LyqCoscHxY+x6PGBY2yGoscHrYtxtr1lnh4rt6TnQ6l9CFiaO29JajMzszaabXLfBqxP2+uB23Ptl6ReM6uBw7nyjZmZtcmMZRlJNwHnAQslHQC+AHwJuEXSZcCTwIfS6XcCFwGDwAvApS2IuR0KWS7KKXp8UPwYix4fOMZmKHp80KIYCzH9gJmZNZdHqJqZlZCTu5lZCc375C6pIulWST+XtE/S26aaXqGDMfZI2itpj6SbJL1M0lmSdqSpHr4v6aQ2x1ToaSmmiO8r6d95t6QfSqrkjm1K8T0q6T2tjm+qGHPHPi0pJC1M+4W4hqn979J13Cvpy7n2QlxDSSsk3Sdpl6QBSatSeyeu4VJJ2yU9kq5XNbW3/r0SEfP6QTbC9m/T9klABfgycGVquxK4uoPxLQYeB05O+7cAH03P61LbN4FPtDmudwArgT25tkmvG9kf2X8ECFgN7OhQfO8GTkjbV+fiOxt4CFgAnAX8Aji+EzGm9qXAv5N1VlhYsGv4TuAnwIK0/+qiXUPgx8CFuet2bwev4SJgZdo+BfivdK1a/l6Z13fukk4l+8+xBSAi/jciRph6eoVOOQE4WdIJwMvJRvyuAW5Nx9seYxR8WorJ4ouIH0fE0bR7H9k4jLH4bo6IIxHxOFlvr1WtjG+qGJM+4LNAvrdDIa4h8AngSxFxJJ0zNsalSNcwgFel7VOBX+dibPc1PBgRD6Tt3wH7yG7YWv5emdfJnewOYxj4jqQHJV0n6RVMPb1C20XEEPBV4FdkSf0wsBMYySWqsWkeOq3eaSk66W/I7pCgQPFJWgsMRcRDEw4VJcY3AH+WSoL/Iektqb0o8QFcDnxF0n6y986m1N7RGCUtA84BdtCG98p8T+4nkH2kuzYizgH+m2MzXALZ9AqMv4Nqq1SLW0v2i+g1wCt46SydhdPp6zYdSZ8HjgI3djqWPEkvBz4H/H2nY5nGCcDpZCWDz5CNdynaSjmfAHoiYinQQ/pk3kmSXgn8ALg8In6bP9aq98p8T+4HgAMRsSPt30qW7KeaXqET/gJ4PCKGI+L3wG3AuWQf18YGoRVlmofCT0sh6aPA+4APpzcVFCe+15H9En9I0hMpjgck/QHFifEAcFsqG9wPjJJNfFWU+CAbNX9b2v5XjpWHOhKjpBPJEvuNETEWV8vfK/M6uUfEU8B+SX+cms4HHmHq6RU64VfAakkvT3dIYzFuBz6Yzul0jGMKPS2FpAvIatnvj4gXcoe2AeskLZB0Ftl6BPe3O76IeDgiXh0RyyJiGVkiXZn+nxbiGgL/RvZHVSS9gawTwjMU5Bomvwb+PG2vAR5L222/huk9uwXYFxFfzx1q/Xul1X8tLvoDWAEMALvJ/uOeBpxBtgjJY2Q9A07vcIxfBH4O7AG+S9Yj4bVkb55BsruTBW2O6SayvwH8niwJXTbVdSP7y/8/kfWgeBjo7lB8g2T1zF3p8c3c+Z9P8T1K6mnRiRgnHH+CY71linINTwK+l/4vPgCsKdo1BN5O9neph8jq23/awWv4drKSy+7c/7uL2vFe8fQDZmYlNK/LMmZmZeXkbmZWQk7uZmYl5ORuZlZCTu5mZiXk5G5mVkJO7mZmJfR/VJCRfOQ+aeAAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ax.scatter(x[:, 0], y, marker=\"x\", color=\"red\")\n", + "display(fig)" + ] + }, + { + "cell_type": "markdown", + "id": "5c8310ab", + "metadata": {}, + "source": [ + "### Now, we need a model so let's define it\n", + "\n", + "The main purpose of this tutorial is not to train a linear regression model but to use it homomorphically. So we will not discuss about how the model is trained." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "91d4a1da", + "metadata": {}, + "outputs": [], + "source": [ + "class Model:\n", + " w = None\n", + " b = None\n", + "\n", + " def fit(self, x, y):\n", + " a = np.ones((x.shape[0], x.shape[1] + 1), dtype=np.float32)\n", + " a[:, 1:] = x\n", + "\n", + " regularization_contribution = np.identity(x.shape[1] + 1, dtype=np.float32)\n", + " regularization_contribution[0][0] = 0\n", + "\n", + " parameters = np.linalg.pinv(a.T @ a + regularization_contribution) @ a.T @ y\n", + "\n", + " self.b = parameters[0]\n", + " self.w = parameters[1:].reshape(-1, 1)\n", + "\n", + " return self\n", + "\n", + " def evaluate(self, x):\n", + " return x @ self.w + self.b" + ] + }, + { + "cell_type": "markdown", + "id": "faa5247c", + "metadata": {}, + "source": [ + "### And create one" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "682fb2d8", + "metadata": {}, + "outputs": [], + "source": [ + "model = Model().fit(x, y)" + ] + }, + { + "cell_type": "markdown", + "id": "084fb296", + "metadata": {}, + "source": [ + "### Time to make some predictions" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "4953b03e", + "metadata": {}, + "outputs": [], + "source": [ + "inputs = np.linspace(40, 210, 100).reshape(-1, 1)\n", + "predictions = model.evaluate(inputs)" + ] + }, + { + "cell_type": "markdown", + "id": "f28155cf", + "metadata": {}, + "source": [ + "### Let's visualize our predictions to see how our model performs" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "111574ed", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhn0lEQVR4nO3de5yWc/7H8deHsHJoSIgQK6uTDgZFysoxtrKHFrtbSwmlncZaxe4ydu1SZEykRK1yziGlg0oH6UdlOk8lFaJESQdEmub7++N73dwzzTTnua657/fz8bgfc93f+7qbT/fj9vHpc32v79ecc4iISGLZL+wARESk4im5i4gkICV3EZEEpOQuIpKAlNxFRBJQjbADADjqqKNc/fr1ww5DRKRaWbBgwZfOuTqFvRaJ5F6/fn2ys7PDDkNEpFoxs3VFvaa2jIhIAlJyFxFJQEruIiIJSMldRCQBKbmLiCQgJXcRkQSk5C4ikoCU3EVEQrBzJ/TrB+uKnKlePkruIiJVbOZMaNoUBg6ESZMq53couYuIVJFt26BnT7jwQthvP5g1C26+uXJ+l5K7iEgVGD8eGjeGESPg9tth6VJo167yfp+Su4hIJdq0Ca65Bjp1gtq1Ye5cGDAADj64cn+vkruISCVwDp59Fho1gldfhX//G7Kz4ayzqub3R2JVSBGRRPLpp76XPnEitGrlWzGNGlVtDKrcRUQqSF4eDB3qe+szZ0JmJsyZU/WJHVS5i4hUiNWroUcPmD0bLroIhg+Hk08OLx5V7iIi5ZCb6+ern3EGLFniWzBTp4ab2EGVu4hImS1ZAt27w4IF0LkzDBkCxx0XdlSeKncRkVLatQv++U9ITfUXT8eM8TNiopLYQZW7iEipvPuur9ZXroSuXeGhh/z89ahR5S4iUgLffANpaXDeefDttzB5MowaFc3EDkruIiLFmjbNL/Q1eDD07uXIyYHLLgtedC7U2Iqi5C4iUoStW+H66+GSS+DAA+Ht60bySI10Djs0SOjOQXo6ZGSEGmdhlNxFRAoxdqy/+Wj0aLjjDliy2NHm8KWQleUTeiyxZ2X55R4jVsHrgqqISJzPP4c+feDll6FZM7+EQMuWAOZvOQWf0LOy/HFamh83CyvkQqlyFxHBF96jR/tqffx4+M9/4L33Yok9YHEJPiaCiR2U3EVEWLcOOnSAbt2gYUN/c9Kdd8IBBxQ4MdaKiRdr0USMkruIJK28PH9XaZMm8Pbb8Mgj/ufppxdycnyPPS3NvzktLX8PPkLUcxeRpLRqlV/oa84cuPRSGDYM6tffxxvMICUlf4891qJJSYlca8ZcBP5vk5qa6rKzs8MOQ0SqK+fyJ9eCz+Ps3g0PPgj33AM1a/r83LVrKXJzKX5XZTOzBc651MJeK1Fbxsw+NrNlZrbYzLKDsSPNbJqZrQ5+HhGMm5kNNrM1ZrbUzFru+08XESmHjIz8bZF9zD1ftAjOOcf306+8Elas8H32UuXmgidHrGKPKU3P/ZfOueZx/5foD0x3zjUApgfPAS4HGgSPnsDQigpWRCQf5/wc82Lmnn//vU/oZ50Fn30Gr7zipzoee2yo0Veq8vTcOwEXBMejgFlAv2B8tPP9nrlmlmJmdZ1zG8sTqIjIXuL73kXMPZ8zx/fWV62C666DQYPgiCPCC7mqlLRyd8BUM1tgZj2DsWPiEvbnwDHB8fHAp3HvXR+M5WNmPc0s28yyN2/eXIbQRUQocu75198YffpA27a+cp8yBUaOTI7EDiVP7m2ccy3xLZfeZtY2/sWgSi/VlVnn3HDnXKpzLrVOnTqleauIyE8KmXs+5deP06SJY8gQf7dpTo5fHyaZlCi5O+c2BD83AWOBs4EvzKwuQPBzU3D6BuCEuLfXC8ZERCpWgbnnX32Zx58bzuOy126i5tebeHu2IysLDj007ECrXrHJ3cwOMbPDYsfAJUAOMB7oFpzWDRgXHI8HugazZloB29VvF5FKETf3/OXzMmnYyHh29Vn8/aypLOr1BOe1ieZMlqpQkguqxwBjzU/3qQE855x7w8zeA8aYWXdgHdAlOH8S0AFYA+wErqvwqEVEAhtvzOCWWxyvdjFatoQpU4zmzS4GS7I+TAHFJnfn3IdAs0LGtwDtCxl3QO8KiU5EpAjOwVNPwa23wvffGwMG+OMaNQCSt2KP0fIDIlLtfPQR9OwJb74J558PTz4Jp50WdlTRooXDRKTa2LPHb3XXpAnMnesX/Zo1S4m9MKrcRaRaWLkSuneHd9+Fyy/3C32deGLYUUWXKncRibTdu/3GGc2b+7tMn37a746kxL5vqtxFJLIWLPDV+pIl0KWLX2/96KPDjqp6UOUuIpHz3XfQv79fwXHTJr9Z9YsvKrGXhip3EYmU2bP9Ql+rV/ufDzzg71OS0lHlLiKRsGMH9OoF7dpBbq6f5vjEE0rsZaXkLiKhmzzZT28cNswvFbNsGbTf6xZJKQ21ZUQkNFu2QN++8Mwz0KgRvPMOtGoVdlSJQZW7iFQ552DMGGjYEF54Ae66CxYuVGKvSKrcRaRKffaZ762PGwepqb63fsYZYUeVeFS5i0iVcA5GjPDtlylTYOBAf7epEnvlUOUuIpXuww/9Ql/Tp/vZME8+CaeeGnZUiU2Vu4hUmj174OGHoWlTmD/fz4aZMUOJvSqocheRSrF8uV86YN48uOIKn9jr1Qs7quShyl1EKtQPP8C//gUtWsDatfDcc/D660rsVU2Vu4hUmPfe89X6smVw9dV+7fU6dcKOKjmpcheRctu5E/72Nz9P/auvYPx4eP55JfYwqXIXkXKZNcsv8LV2Ldx4IwwYALVqhR2VqHIXkTLZvt0n81/+0j+fMcNfNFVijwYldxEptQkToHFjP1/9tttg6dKfkrxEg5K7iJTY5s1w7bXwq1/BEUf4O0wfeABq1gw7MilIyV1EiuWcv0DaqBG8/DLcfbffAu/ss8OOTIqiC6oisk/r1/uFvl5/3SfzESP82usSbarcRSQ/5wDIy4Phw6FxY8ebb8KgQX69dSX26kHJXUR+kpEB6emsWe1o397Phjnz8NUs65HFrbfC/vuHHaCUlJK7iHjOkfvVDh7MqkHThrtZuNAxvP0LTF//C36+30c/VvRSPajnLiIALMsxus8dxHsYHfeM47EdvTh++meQlgaZmWAWdohSCqrcRZLcrl1+9kvLlvDxx8YLzzteozPH85k/QYm9WlJyF0li8+bBmWf6VRyvvhpWLHf8fm46+VJ5erpaMtWQkrtIEvr2W5+zW7f2ywhMnAhPj3Yc9Z90yMryrZi8PP8zK0sJvhpSz10kyUyfDjfcAB995Oev33cfHH44gEFKSv4ee2amf1NKiloz1YySu0iS2LbNrwMzYgQ0aABvvQVt2xY4KSPDV+ixRB5L8Ers1Y7aMiLVQcGWSClbJOPG+aUDnnoK+vWDJUsKSewxBRO5Enu1VOLkbmb7m9kiM5sQPD/ZzOaZ2Roze9HMDgzGDwqerwler19JsYskh+DGoh8TunP+eUZGsW/dtAl+/3vo3NlvnDFvHtx/Pxx8cGUGLFFQmso9DVgZ93wAkOmcOxXYCnQPxrsDW4PxzOA8ESkL53w/Jf6iZnpw0XPbtiIreOfg2Wd9tf7aa3DvvZCd7WfGSJJwzhX7AOoB04ELgQmAAV8CNYLXWwNTguMpQOvguEZwnu3rzz/zzDOdiBQhL8+5tDTnfM72j7Q0P16ITz5xrkMHf1qrVs4tX16l0UoVArJdEXm1pJX7w8DtQF7wvDawzTmXGzxfDxwfHB8PfBr8jyMX2B6cn4+Z9TSzbDPL3rx5cwnDEElC8bNWYgq5yJmXB0OH+k00Zs3yxf2cOb56l+RTbHI3syuBTc65BRX5i51zw51zqc651DraRVekaLFWTLwC884/+AAuuMBPbTznHMjJgb/8RQt9JbOSVO7nAR3N7GPgBXxrJgtIMbPYVMp6wIbgeANwAkDwei1gSwXGLJI84nvshdxYlLvbMXAgNGsGy5bByJEwdSqcfHLYgUvYip3n7py7A7gDwMwuAG5zzv3BzF4CfotP+N2AccFbxgfP3w1enxH0hkSktKzoG4uWfP8Lrm9lLFwIV10FQ4ZA3brhhivRUZ6bmPoBL5jZvcAiYEQwPgJ42szWAF8BV5cvRJEkV+DGol0/GPcemsn9Q4zatf22d7/5TbghSvSUKrk752YBs4LjD4G9dlB0zn0P/K4CYhORmCCxv/MOdO8O779vdO3qi/gjjww5Nokk3aEqUg18843vzLRpAzt3whtvwKhRSuxSNK0tIxJxU6dCz57wySc/LfR12GFhRyVRp8pdJKK2boXrr4dLL4Wf/Qxmz4ZHH1Vil5JRcheJoFdf9TcfjR4Nd9wBixf7loxISaktIxIhn38Ot9wCr7wCzZvDpEnQokXYUUl1pMpdJAKc8xdIGzWCCRPgP/+B+fOV2KXsVLmLhGzdOrjpJj8D5txz/WYap58edlRS3alyFwlJXp6/q7RJE3j7bRg82P9UYpeKoMpdJASrVkGPHn7Vxksvhccfh5NOCjsqSSSq3EWq0O7dfp56s2awfLnf9m7yZCV2qXiq3EWqyKJFfumARYvgt7+FRx6BY48NOypJVKrcRSrZ99/DnXfCWWfBxo1+muNLLymxS+VS5S5SiebM8b31Vavguutg0CA44oiwo5JkoMpdpBJ8/bW/Gen8833lPnWq30hDiV2qipK7SAWbMsVPb3zsMb+SY04OXHxx2FFJslFyF6kgW7ZAt25w2WVQs6ZvyTz8MBx6aNiRSTJSchcpJ+f8bkiNGsFzz8E//uEX+jr33LAjk2SmC6oi5bBxI/TuDWPHQsuWvrferFnYUYmochcpE+fgf//z1frkyTBgAMybp8Qu0aHKXaSUPv7Y74w0bZqfDfPkk3DaaWFHJZKfKneREtqzxy/u1aQJvPuunw0za5YSu0STKneREli50i8d8O67cPnlMGwYnHhi2FGJFE2Vu8g+7N7tN85o3tzfZfr00zBxYpDYnct/csHnIiFSchcpwoIFkJrqpzZ27uyr9z/+EcyAjAxIT/8poTvnn2dkhBewSBwld5ECvvsO+vWDc86BzZv9NMcXX4Sjjw5OcA62bYOsrJ8SfHq6f75tmyp4iQT13EXivPUW3HADrF7te+wPPggpKQVOMoPMTH+cleUf4NcayMwMSnuRcKlyFwF27ICbb4YLLoDcXHjzTT/Fca/EHhOf4GOU2CVClNwl6U2aBI0b+63u+vaFZcugffti3hRrxcSL78GLhEzJXZLWl1/6C6RXXAGHH+6nOWZmwiGHFPPG+B57Wprf6TotLX8PXiRk6rlL0nHOXyDt08df/7zrLr9T0kEHlfAPMPP9mvgee6xFk5Ki1oxEgrkIVBmpqakuOzs77DAkCWzYAL16wfjxfprjyJHQtGkZ/zDn8ifygs9FKpmZLXDOpRb2mtoykhScgyee8At9TZ0KAwf6NkyZEzvsnciV2CVC1JaRhPfhh35644wZ0K6dnwVz6qlhRyVSuVS5S8Las8e3wps0gexsvx7MjBlK7JIcik3uZvYzM5tvZkvMbLmZ3ROMn2xm88xsjZm9aGYHBuMHBc/XBK/Xr+S/g8hecnLgvPPg1lv9tMbly+HGG2E/lTOSJEryVd8FXOicawY0By4zs1bAACDTOXcqsBXoHpzfHdgajGcG54lUiR9+gHvu8bsirV3rt70bPx7q1Qs7MpGqVWxyd943wdMDgocDLgReDsZHAZ2D407Bc4LX25vpSpNUkH2sxPjee3DmmX7trt/9DlasgGuu0XVOSU4luqBqZvsDC4BTgSHAWmCbcy43OGU9cHxwfDzwKYBzLtfMtgO1gS8L/Jk9gZ4AJ2phbCmJjAw/MT02tzy4mWjnIXW4a9ffycyEunV9pf6rX4UdrEi4SpTcnXN7gOZmlgKMBU4v7y92zg0HhoOf517eP08SXPxKjOATfHo6s7IW06PWS6zd7nvqAwZArVqhRioSCaWaCumc22ZmM4HWQIqZ1Qiq93rAhuC0DcAJwHozqwHUArZUYMySjAqsxLg963/czkCG8zA/P8ox8zW/6JeIeCWZLVMnqNgxs4OBi4GVwEzgt8Fp3YBxwfH44DnB6zNcFG6DleovSPCvcyWNWMGT9OC2vzqWLjUldpECSjJbpi4w08yWAu8B05xzE4B+wK1mtgbfUx8RnD8CqB2M3wr0r/iwJRlt3uS49vSFdOR1arOFubTigdx0ah6s2kGkoGLbMs65pUCLQsY/BM4uZPx74HcVEp0Ivt3+/HOOv/T4lh3fN+WeVpPpP+syDux3Xv4evKbFiPxIyw9IpK1fDzfdBBMnGuccv40Rv3yGxqP7aSVGkWIouUsk5eX5hb5uv93vjJSZCX361GP//fr9lMhjCV6JXWQvSu4SOatX+4W+3nrLLx0wfDicckrsVa3EKFISWmlDIiM3129IfcYZsHixX71x2rT4xC4iJaXKXapGMRtbLF0K3bv71Rs7dYLHHoPjjgshTpEEocpdKl9GRv69RWN7kGZksGsX3H23XxNm3Tq//d3YsUrsIuWl5C6VK37ZgFiCDzaXnruyFi1bOv71L7j6ali5Erp0URtdpCKoLSOVq8CyAWRl8S01+WeLmTz8Ujvq1TMmToQOHcINUyTRqHKXyheX4KdzIU1ZRuaiC7j5ZiMnR4ldpDIouUvlc45tve6kB09wEdOpQS5v/XYwQx51HH542MGJJCa1ZaRyOcdrHUfSa0IfNtkx3H6bI2PnExw85EFI/1A3IYlUEiV3qTRffAF9+hgvTejOGUdt4PXJ+3FmqoEbCDV2a9kAkUqk5C4Vzjl45hno2xe++QbuvRdu/9txHHCglg0QqSpK7lKhPvnEL/Q1eTK0bu3vMm3UCLRsgEjV0gVVqRB5eTB0KDRuDLNnw+DB8PbbscQuIlVNlbuU2wcf+IW+Zs+Giy/2C33Vrx92VCLJTZW7lFluLgwcCM2a+bVhRo6EKVOU2EWiQJW7lMmSJXD99bBwIfz61/Doo1C3bthRiUiMKncplV274B//gNRU2LABXn4ZXnlFiV0kalS5S4m9845flvf996FbN3joITjyyLCjEpHCqHKXYn3zDaSlQZs2sHMnvPEGPPWUErtIlKlyl32aNg169vRrrffuDf/9Lxx2WNhRiUhxVLlLobZuheuug0sugYMO8tMcH3lEiV2kulByl728+qq/+ejpp+GOO/x+pm3ahB2ViJSG2jLyo88/h1tu8bNfmjeHSZOgRYuwoxKRslDlnqhi+5UW9bzAS6NG+Wp9wgTfV58/X4ldpDpTck9E+9iQuqB16+Cyy+DPf/bJffFi34o54IAqjFdEKpySe6LZx4bUbNv2Y8LPy/N3lTZu7OevP/qov2h6+umhRi8iFUQ990RTyIbUgJ+oHqyhvmqVvxnp//7PV+3DhsFJJ4UXsohUPFXuiSg+wcdkZrI717jvPr/Q18qVvs8+aZISu0giUnJPRLFWTJxFf3iQs8923HkndOwIK1ZA167aM0MkUSm5J5r4HntaGt99m8cdqdM46/l0Pv9gB6++4hgzBo45JuxARaQyKbknGjO/8XRaGnN+k0nzFsb92RfRtdECVtwylKt+rVJdJBnogmoC+vqvGdzR3zGkrVG/PkydChdfdDbYOWGHJiJVRJV7gpk82U9vfGyokZYGy5b5re/UXBdJLsUmdzM7wcxmmtkKM1tuZmnB+JFmNs3MVgc/jwjGzcwGm9kaM1tqZi0r+y8hsGWLv0DaoQMceqif5vjww/5YRJJPSSr3XOCvzrlGQCugt5k1AvoD051zDYDpwXOAy4EGwaMnMLTCo5YfOQcvveTvLn3+eb9L0qJF0Lp12JGJSJiKTe7OuY3OuYXB8dfASuB4oBMwKjhtFNA5OO4EjHbeXCDFzLQJWyXYuNHvX9qlC5xwAmRnw7//7ZfoFZHkVqqeu5nVB1oA84BjnHMbg5c+B2KT644HPo172/pgrOCf1dPMss0se/PmzaWNO6k5ByNHQsOGflekAQNg7lx/c5KICJQiuZvZocArQF/n3I7415xzDih62cFCOOeGO+dSnXOpderUKc1bk9pHH/kNNLp398l86VK4/XaooXlPIhKnRMndzA7AJ/ZnnXOvBsNfxNotwc9NwfgG4IS4t9cLxqQc9uzx9yU1aQLz5sHQoTBzJjRoEHZkIhJFJZktY8AIYKVz7qG4l8YD3YLjbsC4uPGuwayZVsD2uPaNlMGKFXD++dC3L7RrB8uXw003wX6ayCoiRSjJP+bPA/4ELDOzxcHYncD9wBgz6w6sA7oEr00COgBrgJ3AdRUZcMJyLv9cdOf4YbcxcKC/SHrYYfDMM3DttZqyLiLFKza5O+fmAEWlk/aFnO+A3uWMK7lkZPi11oMleXGO7GsfovvMP7D0i2P5/e9h8GA4+uiwAxWR6kKX4cIWv7kG8N1/M8loO4MHF/Tl2EO+5rWxjk6dVaqLSOkouYctbu312VkL6ZG1mtW054Ym7zBwdmtSjlBiF5HS0yW5CNjxtdFrVybtmM0e9mc6FzJ8qRK7iJSdknvIJk2Cxo0djz/uuJVBLKMpFzIz/wbXIiKlpOQeki+/hD/+Ea64Amp99znvuNYMSvuUmnnf+v1O4ze4FhEpJfXcq5hzMGYM9Onjr6PefTfcuedJDvy69U+zZWL7n6akaN6jiJSJknsV2rABevWC8ePhrLNgxAho2hTgn/nnuccSvBK7iJSR2jJVwDl44gm/LO+0afDgg/Duu7HEHiiYyJXYRaQcVLlXsrVr4YYb/DowF1zgk/ypp4YdlYgkOlXulWTPHnjoIV+dL1gAjz8O06crsYtI1VDlXglycvySvPPnw5VX+hUc69ULOyoRSSaq3CvQDz/APfdAy5bw4Yfw3HP+4qkSu4hUNVXuFWT+fF+t5+TANdf4aerag0REwqLKvZx27oTbbvMbUm/dCq+/7it2JXYRCZMq93KYORN69PAtmBtv9HuZ1qoVdlQiIqrcy2T7dp/ML7zQ74Y0cyYMG6bELiLRoeReSq+/7m9GevJJ+NvfYMkSP39dRCRKlNxLaPNmf6G0Y0eoXdtvUj1wINSsGXZkIiJ7U3IvhnP+AmnDhvDKK36qY3Y2pKaGHZmISNF0QXUfPv0Ubr4ZJk6EVq18K6Zx47CjEhEpnir3QuTl+QukjRv7i6WZmTBnjhK7iFQfqtwLWL3aL/T11lvQvj0MHw6nnBJ2VCIipaPKPZCbCw88AGecAYsX+7XWp01TYheR6kmVO7B0qV86IDsbOnWCxx6D444LOyoRkbJL6sp91y646y4480xYtw5efBHGjlViF5HqL2kr97lzfbW+YgX86U/+omnt2gQbUmsXJBGp3pKucv/2W0hPh3PPha/Xb2dix8cZPcr9lNjT0yEjI+wwRUTKJamS+/Tpfmekhx+Gm29y5Fz7XzqMv8kn9Fhiz8qCbduCCl5EpHpKirbMtm3w17/CyJHQoIGf5ti2rYG7Hw7a5RN6VpY/OS3N92i0QbWIVGMJX7m/9ppf6GvUKOjXzy/01bZt8KKZT+TxlNhFJAEkbHL/4gvo0gWuugqOPtov9HX//XDwwXEnxVox8WItGhGRaizhkrtz8PTTvlofNw7uvRfee89Pd9zrxFiPPS3NrzmQluafK8GLSDWXUD33Tz7xm2i88Ybf9m7ECL+aY6HMICUlf4891qJJSVFrRkSqNXMRqFBTU1NddnZ2md+flwdDh0L//r7gvu8+6N3b75JULOfyJ/KCz0VEIsrMFjjnCl2AvNj0Z2YjzWyTmeXEjR1pZtPMbHXw84hg3MxssJmtMbOlZtay4v4ahVu1Ctq1g1tu8dV6Tg706VPCxA57J3IldhFJACVJgU8BlxUY6w9Md841AKYHzwEuBxoEj57A0IoJs3AjR0KzZrB8OTz1FEyZAvXrV+ZvFBGpHopN7s652cBXBYY7AaOC41FA57jx0c6bC6SYWd0KinUvp50GV17plxDo1k1Ft4hITFkvqB7jnNsYHH8OHBMcHw98Gnfe+mBsIwWYWU98dc+JJ55YpiDatPEPERHJr9xTIZ2/Ilvqq7LOueHOuVTnXGqdOnXKG4aIiMQpa3L/ItZuCX5uCsY3ACfEnVcvGBMRkSpU1uQ+HugWHHcDxsWNdw1mzbQCtse1b0REpIoU23M3s+eBC4CjzGw9cDdwPzDGzLoD64AuwemTgA7AGmAncF0lxCwiIsUoNrk7564p4qX2hZzrgN7lDUpERMon4daWERERJXcRkYSk5C4ikoAisXCYmW3GX5gN01HAlyHHUFqKufJVt3hBMVeVKMR8knOu0BuFIpHco8DMsotaXS2qFHPlq27xgmKuKlGPWW0ZEZEEpOQuIpKAlNx/MjzsAMpAMVe+6hYvKOaqEumY1XMXEUlAqtxFRBKQkruISAJK2uRuZh+b2TIzW2xm2cFYoXvDhs3MfhHEGXvsMLO+ZpZhZhvixjuEHGek99stRcwPmNn7QVxjzSwlGK9vZt/Ffd7DIhRzkd8FM7sj+JxXmdmlEYr5xbh4PzazxcF46J+zmZ1gZjPNbIWZLTeztGA80t/nfJxzSfkAPgaOKjA2EOgfHPcHBoQdZyFx74/f/eokIAO4LeyY4mJrC7QEcor7TPGrh04GDGgFzItQzJcANYLjAXEx148/L2Kfc6HfBaARsAQ4CDgZWAvsH4WYC7w+CLgrKp8zUBdoGRwfBnwQfJaR/j7HP5K2ci9CUXvDRkl7YK1zLuw7evfiIrzfblEKi9k5N9U5lxs8nYvfdCYyivici9IJeME5t8s59xF+Oe6zKy24IuwrZjMz/LLhz1dpUPvgnNvonFsYHH8NrMRvGRrp73O8ZE7uDphqZguC/Vyh6L1ho+Rq8v9HcEvwz8CRUWkjFVDa/Xaj5np8RRZzspktMrO3zOz8sIIqQmHfherwOZ8PfOGcWx03FpnP2czqAy2AeVSj73MyJ/c2zrmWwOVAbzNrG/+i8//WitQ8UTM7EOgIvBQMDQV+DjTHb0I+KJzISiaKn+m+mNnfgVzg2WBoI3Cic64FcCvwnJkdHlZ8BVSr70IB15C/YInM52xmhwKvAH2dczviX4v69zlpk7tzbkPwcxMwFv9P1aL2ho2Ky4GFzrkvAJxzXzjn9jjn8oAnCOGf2yVQLffbNbM/A1cCfwj+IyZobWwJjhfg+9enhRZknH18F6L+OdcAfg28GBuLyudsZgfgE/uzzrlXg+Fq831OyuRuZoeY2WGxY/wFtByK3hs2KvJVOAV6elfh/w5RU+322zWzy4DbgY7OuZ1x43XMbP/g+BSgAfBhOFHmt4/vwnjgajM7yMxOxsc8v6rj24eLgPedc+tjA1H4nIPrACOAlc65h+Jeqj7f57Cv6IbxAE7BzyBYAiwH/h6M1wamA6uBN4Ejw441LuZDgC1Arbixp4FlwFL8l6tuyDE+j/8n9W58z7F7UZ8pflbBEHxVtgxIjVDMa/D908XBY1hw7m+C78tiYCHwqwjFXOR3Afh78DmvAi6PSszB+FPATQXODf1zBtrgWy5L474HHaL+fY5/aPkBEZEElJRtGRGRRKfkLiKSgJTcRUQSkJK7iEgCUnIXEUlASu4iIglIyV1EJAH9P9KjtXmE2MJ+AAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ax.plot(inputs, predictions, color=\"blue\")\n", + "display(fig)" + ] + }, + { + "cell_type": "markdown", + "id": "23852861", + "metadata": {}, + "source": [ + "### As a bonus let's inspect the model parameters" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "7877cb2e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[2.6698928]]\n", + "-3.2299957\n" + ] + } + ], + "source": [ + "print(model.w)\n", + "print(model.b)" + ] + }, + { + "cell_type": "markdown", + "id": "de63118c", + "metadata": {}, + "source": [ + "They are floating point numbers and we can't directly work with them!" + ] + }, + { + "cell_type": "markdown", + "id": "2d959640", + "metadata": {}, + "source": [ + "### So, let's abstract quantization\n", + "\n", + "Here is a quick summary of quantization. We have a range of values and we want to represent them using small number of bits (n). To do this, we split the range into 2^n sections and map each section to a value. Here is a visualization of the process!" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "9da2e1a4", + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": "
min(x)
min(x)
max(x)
max(x)
Map
to 0
Map...
Map
to 1
Map...
Distance
Between
Consecutive
Values
Distan...
Map
to 2
Map...
Map
to 3
Map...
(when n = 2)
(when n = 2)
0
0
= 1 / scale
= 1 / q
= 1 / scale...
x = (x   + zp  ) / q
x = (x   + zp  ) / q
q
q
x
x
x
x
zero point
zp = 2
zero point...
Viewer does not support full SVG 1.1
", + "text/plain": [ + "" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from IPython.display import SVG\n", + "SVG(filename=\"figures/QuantizationVisualized.svg\")" + ] + }, + { + "cell_type": "markdown", + "id": "45d12e7a", + "metadata": {}, + "source": [ + "If you want to learn more, head to https://intellabs.github.io/distiller/algo_quantization.html" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "2541cdb7", + "metadata": {}, + "outputs": [], + "source": [ + "class QuantizationParameters:\n", + " def __init__(self, q, zp, n):\n", + " # q = scale factor = 1 / distance between consecutive values\n", + " # zp = zero point which is used to determine the beginning of the quantized range\n", + " # (quantized 0 = the beginning of the quantized range = zp * distance between consecutive values)\n", + " # n = number of bits\n", + " \n", + " # e.g.,\n", + " \n", + " # n = 2\n", + " # zp = 2\n", + " # q = 0.66\n", + " # distance between consecutive values = 1 / q = 1.5151\n", + " \n", + " # quantized 0 = zp / q = zp * distance between consecutive values = 3.0303\n", + " # quantized 1 = quantized 0 + distance between consecutive values = 4.5454\n", + " # quantized 2 = quantized 1 + distance between consecutive values = 6.0606\n", + " # quantized 3 = quantized 2 + distance between consecutive values = 7.5757\n", + " \n", + " self.q = q\n", + " self.zp = zp\n", + " self.n = n\n", + "\n", + "class QuantizedArray:\n", + " def __init__(self, values, parameters):\n", + " # values = quantized values\n", + " # parameters = parameters used during quantization\n", + " \n", + " # e.g.,\n", + " \n", + " # values = [1, 0, 2, 1]\n", + " # parameters = QuantizationParameters(q=0.66, zp=2, n=2)\n", + " \n", + " # original array = [4.5454, 3.0303, 6.0606, 4.5454]\n", + " \n", + " self.values = np.array(values)\n", + " self.parameters = parameters\n", + "\n", + " @staticmethod\n", + " def of(x, n):\n", + " if not isinstance(x, np.ndarray):\n", + " x = np.array(x)\n", + "\n", + " min_x = x.min()\n", + " max_x = x.max()\n", + "\n", + " if min_x == max_x: # encoding single valued arrays\n", + " \n", + " if min_x == 0.0: # encoding 0s\n", + " \n", + " # dequantization = (x_q + zp_x) / q_x = 0 --> q_x = 1 && zp_x = 0 && x_q = 0\n", + " q_x = 1\n", + " zp_x = 0\n", + " x_q = np.zeros(x.shape, dtype=np.uint)\n", + " \n", + " elif min_x < 0.0: # encoding negative scalars\n", + " \n", + " # dequantization = (x_q + zp_x) / q_x = -x --> q_x = 1 / x & zp_x = -1 & x_q = 0\n", + " q_x = abs(1 / min_x)\n", + " zp_x = -1\n", + " x_q = np.zeros(x.shape, dtype=np.uint)\n", + " \n", + " else: # encoding positive scalars\n", + " \n", + " # dequantization = (x_q + zp_x) / q_x = x --> q_x = 1 / x & zp_x = 0 & x_q = 1\n", + " q_x = 1 / min_x\n", + " zp_x = 0\n", + " x_q = np.ones(x.shape, dtype=np.uint)\n", + " \n", + " else: # encoding multi valued arrays\n", + " \n", + " # distance between consecutive values = range of x / number of different quantized values = (max_x - min_x) / (2^n - 1)\n", + " # q = 1 / distance between consecutive values\n", + " q_x = (2**n - 1) / (max_x - min_x)\n", + " \n", + " # zp = what should be added to 0 to get min_x -> min_x = (0 + zp) / q -> zp = min_x * q\n", + " zp_x = int(round(min_x * q_x))\n", + " \n", + " # x = (x_q + zp) / q -> x_q = (x * q) - zp\n", + " x_q = ((q_x * x) - zp_x).round().astype(np.uint)\n", + "\n", + " return QuantizedArray(x_q, QuantizationParameters(q_x, zp_x, n))\n", + "\n", + " def dequantize(self):\n", + " # x = (x_q + zp) / q\n", + " # x = (x_q + zp) / q\n", + " return (self.values.astype(np.float32) + float(self.parameters.zp)) / self.parameters.q\n", + "\n", + " def affine(self, w, b, min_y, max_y, n_y):\n", + " # the formulas used in this method was derived from the following equations\n", + " #\n", + " # x = (x_q + zp_x) / q_x\n", + " # w = (w_q + zp_w) / q_w\n", + " # b = (b_q + zp_b) / q_b\n", + " #\n", + " # (x * w) + b = ((x_q + zp_x) / q_x) * ((w_q + zp_w) / q_w) + ((b_q + zp_b) / q_b)\n", + " # = y = (y_q + zp_y) / q_y\n", + " #\n", + " # So, ((x_q + zp_x) / q_x) * ((w_q + zp_w) / q_w) + ((b_q + zp_b) / q_b) = (y_q + zp_y) / q_y\n", + " # We can calculate zp_y and q_y from min_y, max_y, n_y. So, the only unknown is y_q and it can be solved.\n", + "\n", + " x_q = self.values\n", + " w_q = w.values\n", + " b_q = b.values\n", + "\n", + " q_x = self.parameters.q\n", + " q_w = w.parameters.q\n", + " q_b = b.parameters.q\n", + "\n", + " zp_x = self.parameters.zp\n", + " zp_w = w.parameters.zp\n", + " zp_b = b.parameters.zp\n", + "\n", + " q_y = (2**n_y - 1) / (max_y - min_y)\n", + " zp_y = int(round(min_y * q_y))\n", + "\n", + " y_q = (q_y / (q_x * q_w)) * ((x_q + zp_x) @ (w_q + zp_w) + (q_x * q_w / q_b) * (b_q + zp_b))\n", + " y_q -= min_y * q_y\n", + " y_q = y_q.round().clip(0, 2**n_y - 1).astype(np.uint)\n", + "\n", + " return QuantizedArray(y_q, QuantizationParameters(q_y, zp_y, n_y))\n", + "\n", + "class QuantizedFunction:\n", + " def __init__(self, table):\n", + " self.table = table\n", + "\n", + " @staticmethod\n", + " def of(f, input_bits, output_bits):\n", + " domain = np.array(range(2**input_bits), dtype=np.uint)\n", + " table = f(domain).round().clip(0, 2**output_bits - 1).astype(np.uint)\n", + " return QuantizedFunction(table)" + ] + }, + { + "cell_type": "markdown", + "id": "ab82ae87", + "metadata": {}, + "source": [ + "### Let's quantize our model parameters\n", + "\n", + "Since the parameters only consist of scalars, we can use a single bit quantization." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "c8b08ef4", + "metadata": {}, + "outputs": [], + "source": [ + "parameter_bits = 1\n", + "\n", + "w_q = QuantizedArray.of(model.w, parameter_bits)\n", + "b_q = QuantizedArray.of(model.b, parameter_bits)" + ] + }, + { + "cell_type": "markdown", + "id": "e2528092", + "metadata": {}, + "source": [ + "### And quantize our inputs" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "affe644e", + "metadata": {}, + "outputs": [], + "source": [ + "input_bits = 6\n", + "\n", + "x_q = QuantizedArray.of(inputs, input_bits)" + ] + }, + { + "cell_type": "markdown", + "id": "a5a50eb8", + "metadata": {}, + "source": [ + "### Time to make quantized inference" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "0fdfd3d9", + "metadata": {}, + "outputs": [], + "source": [ + "output_bits = 7\n", + "\n", + "min_y = predictions.min()\n", + "max_y = predictions.max()\n", + "y_q = x_q.affine(w_q, b_q, min_y, max_y, output_bits)\n", + "\n", + "quantized_predictions = y_q.dequantize()" + ] + }, + { + "cell_type": "markdown", + "id": "5fb15eb4", + "metadata": {}, + "source": [ + "### And visualize the results" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "8076a406", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnZ0lEQVR4nO3deXhU1f3H8feXRUBZooBIQYpU6/KrFjFFUNkXWQVRxArIKqCgMVRFsEik2oKIGCsiKC6oCAiyurQIKopESRBFFgsqKC6sSUjIQkLO74+50SEmECDhTiaf1/PMM3fOvTP5Zp7hw8m5Z8415xwiIhJeyvhdgIiIFD2Fu4hIGFK4i4iEIYW7iEgYUriLiIShcn4XAFCjRg1Xv359v8sQESlREhIS9jrnaua3LyTCvX79+sTHx/tdhohIiWJmOwrap2EZEZEwpHAXEQlDCncRkTCkcBcRCUMKdxGRMKRwFxEJQwp3EZEwpHAXEfFBamo2TZvew6effl8sr69wFxE5xT788DB16vQjLm4yjz/+VrH8DIW7iMgpkpoKI0YcpnnzARw4MJsBA/7JnDlDi+VnhcTyAyIi4erQoUO88cYbxMWl8PLLsH//CmAuY8f+g/HjRxfbz1W4i4gUk6ysLHr0uJk331x4RPtDDz3Egw/+vVh/tsJdRKQYZGdn07z5LcTFLcRsMiNG9CIqCqpVq0CNGjWK/ecr3EVEitj332dz1VV92LlzPnXqTGbZspE0bHhqa9AJVRGRIuIczJx5mD/8oR87d86lY8dH2b791Ac7KNxFRIrE9u3Qvv1hBg8eQFbWbKKj/8lbb91LOZ/GRzQsIyJygpxzbN78P15+OZvYWEdW1mPAy4wf/w/Gji2+mTCFoXAXETkBWVlZdOnyV/773wVHtMfExDB2bPHOhCkMhbuIyHFKT8+mcePefPnlAipW/DuDB19Ks2ZGrVpn06JFC7/LAxTuIiLH5ZNPsunQoQ9JSa/z5z8/zjvv3M0555jfZf2GTqiKiBRCejrcd99hmjbtR1LSXPr1m8T69dEhGeygcBcROaZVq+Cyyw4zaVJ/nJvNuHETePHFewI7nfO3uAIo3EVECnDgANxxB7RocZiffx4EvMIjTZsSM+6+wAHOQXQ0xMT4WWa+FO4iInlkZWXRps2tRERUZdq0qpQrV5XU1JcY36QJY9asCQR6brDHxkJSUsj14HVCVUQkyE8/ZdGkyS189918qlXrR8eOZ1G7NjRs2JBb+/b9NdBjYwNPiIqCKVPAQmvs3VwI/G8TGRnp4uPj/S5DREox5+C117IZOLA3mZnzaN/+cZYsiaZChXwOLBM06JGT41uwm1mCcy4yv30alhGRUu/HH6Fbt8P07n0rmZnzGDlyEv/5TwHBHh19ZFvuEE2IUbiLSKkVWOgLLr74MG++2R94jX/+cwKTJ9+T/8G5QzJRUYEee1RU4HEIBrzG3EWkVPrmG7jtNli5ModatQZx4MArPPLII4wePSr/J5hBRMSRY+xTpgT2RURozD0/GnMXkZPi3JHhmvdxkIyMLDp2HM2qVV9iBnXr7mPHjnjGjx/P2LFji/RnFbeTHnM3s+1mtsHM1ptZvNd2lpktN7Ot3v2ZXruZ2ZNmts3MvjCzRkX3q4iI5BETc+SwyFHmnq9fn0WdOrfw/vuTqVJlH5ddlsQ555Rl8uTJhQt2+G2Qh1iPPdfxjLm3cs41DPpf4n5ghXPuAmCF9xigI3CBdxsCTCuqYkVEjuBcYI558Lh3PnPPDx2CceOyadSoD/v3z6dPn8dJTFzLunVxxMXFMXLkSF9/jeJwMmPu3YCW3vZLwPvAKK99lguM98SZWYSZ1XbO/XQyhYqI/EbwuHeeueeZEybwr4ceYv36H1i1ChITtwIfMG7cJGJiogt8yXBRqDF3M/sWSAQcMN05N8PMkpxzEd5+AxKdcxFmtgyY4Jz7yNu3AhjlnIvP85pDCPTsqVev3hU7duwowl9LREqVPHPPD2VkcH2Pnrz11lKgNmXKGGedVZYxY6KJzjuVsQQ72ph7YXvu1zjnfjCzs4HlZrYleKdzzpnZcZ2Zdc7NAGZA4ITq8TxXROQXeeaeHwJan/cXVv+0AXiaoUNvZ+JEqFbNtwp9Uagxd+fcD979bmAh0BjYZWa1Abz73d7hPwDnBj29rtcmIlK08sw937s7kwurNmf1TxuoUfFfvLdyGM88U/qCHQoR7mZ2hplVyd0G2gNfAkuAft5h/YDF3vYS4FZv1kwTIFnj7SJSLILmni9q8Sjn1ruF7QdW0brucHaMPEzLVqE5k+VUKMywTC1gYWBYnXLAbOfcO2a2FphnZoOAHcBN3vFvAZ2AbUAaMKDIqxYR8ewZHsOdd2Yxt0cfYAHR0VN4fHJUyE5RPFWOGe7OuW+AP+fTvg9ok0+7A4YXSXUiIvnIzMxk4cJFvPfeQWbPhoMH3wTeYMKExxg16m6/ywsJWn5AREqUzMxMunS5kXffXXZE+4QJExg16m8+VRV6FO4iUmJkZBziyitv4osvllG+/JOMHt2N/v2hcuVK1KxZ0+/yQorCXURKhM2bs2je/Gb27l3CH//4FG+/PZwGDfyuKnRpyV8RCWnZ2TBxYhZ/+tMt7N27kL/+9Um2bFGwH4vCXURC1oYN0KRJNvff34ecnPmMG/c4s2ffWdonwhSKhmVEJKQ459i0aRtTpx5mxgwoV248MI9HH53EvfeGz9IBxU3hLiIhIzMzk7Zte/LRR0t/aTt8ODAT5t5787k6khRI4S4iISEx8RCRkb345pulVK06jmHDLuLyy+F3v/sdzZs397u8EkfhLiK++89/sujR42bS0hbTrNlTLFs2nKpV/a6qZNMJVRHxTVISDByYRYcOt5CWtpC77nqSVasU7EVBPXcR8cWiRXD77dns2tUHmM+jj07h3nvv9LussKGeu4icUrt2wU03wfXXZ5OR0Rfn5vHYY49x7713+11aWFHPXUROiYyMTFq1GsAnn7yNc3DaadkkJaUyceJE/vY3rQlT1BTuIlLstm07xNVX38Tu3UuoWbMfHTpU48wzoXHjxvTu3dvv8sKSwl1Eik1ODkydmkV09M0cPryEnj2nMmfOHcGXO5VionAXkSKVnp7OsGHDiIv7jJ07IS0tBdjOuHFPEhNzh9/llRoKdxEpMhkZGXTr1p13312OWWfKli1Po0Zw550x9O/f79gvIEVG4S4iRSKwdEAPVq/+LzCT7t0HMnUq1K7td2Wlk8JdRE5acnImkZE3sG3b21Sp8iwvvDCQG27wu6rSTac1ROSkfPDBIerWvYlt296kadNn2L59sII9BCjcReSEpKbCiBFZtGx5M6mpSxgxYioffzyUs87yuzIBDcuIyHFIT09nwoQJxMfvYtUqSE3dDKxi0qQnuecezYQJJQp3ESmUjIwMOnfuznvvLQfOpmxZqFmzHA8++G9GjBjhd3mSh8JdRI4pMzOTq67qwWef/RezmYwePZCxY6FiRb8rk4Io3EXkqHbsyKRJkxv5+ee3OffcZ1myZCANG/pdlRyLTqiKyJGc++XuuecOcf75Pfn552V07z6Nr78erGAvIRTuIvKrmBiIjmb7t4527bK47bZeZGcv5cHmPVm4cBjly/tdoBSWhmVEBICM9HQWfbKWZe8c4vWnZnKYZcBiYoG7Lv9doCtv5neZUkgKdxEhIyODtu26e0sHAIffxYApwF1RUTBlioK9hNGwjEgpl5KSwWWXXc/q1cs5/fRpTH7sa74GfgbuBgV7CaVwFynF1qzJpE6dG9m69R0iI5/lm6+HMvL7J2kAnJ17UHT0LydZpeRQuIuUQunpcO+9h7jqqp6kpLzJ7bdPZ+2nA6k1IRpiYyEqKnCljaiowGMFfImjMXeRUubDD2HgwCy2besFLGXSpKncc8+QwM6IiECg5w7FTJnya7uGZkoUcyHwv3FkZKSLj4/3uwyRsJWTk8MXX3zLpEk5zJ7tOP30MaSlLeDf/85n6YC8s2I0SyZkmVmCcy4yv33quYuUBCcRuBkZGVxzzfUkJLzzS1taGjzxxBP5rwmT93UV7CVSocPdzMoC8cAPzrkuZnYeMAeoDiQAfZ1zh8ysAjALuALYB/Ryzm0v8spFSouYGEhK+nWoxLnAGHhERGDfUfzwQyaNG/fgxx/foWbNhxg+/A+cfz7Uq1ePZs2anYLixS/H03OPAjYDVb3HE4Epzrk5ZvYMMAiY5t0nOufON7ObveN6FWHNIqWHc4Fgj40NPJ4yJRDsuSc98/TgnXNkZmbiHMybl8WQIX/l0KG36dLlWebPH0yFCv78GuID59wxb0BdYAXQGlgGGLAXKOftbwr8x9v+D9DU2y7nHWdHe/0rrrjCiUgBcnKci4pyLhDlgVtUVKA9SGpqqmvfvr0Djrj9/e/P+FK2FD8g3hWQq4XtuT8B3AdU8R5XB5Kcc9ne451AHW+7DvC99x9Htpkle8fvDX5BMxsCDIHAn4giUoDcWSu5vXf4zReL0tLS6Nq1K++//wGnnXYvOTln0a4dDB/ekM6dO/hQtPjtmOFuZl2A3c65BDNrWVQ/2Dk3A5gBgdkyRfW6ImEnd4w9WHT0LwGfnp5O+/bdWL36fWAWTZv24bnn4Pzz/ShWQkVhvsR0NXCdmW0ncAK1NRALRJhZ7n8OdYEfvO0fgHMBvP3VCJxYFZHjlRvsBXyx6GBqOpdf3p3Vq1dQseKLTJ/eh5UrFexSiJ67c240MBrA67nf45zrbWavAzcSCPx+wGLvKUu8x2u8/Su9sSEROV5mBX6xaF1afVrX7UFy8nL+/OeZLFt2K3Xr+luuhI6Tmec+CphjZg8DnwEzvfaZwMtmtg3YD9x8ciWKlHIxMUfMijmUZTxcZQIPP3kDzr3Dbbc9y/TpAzQdXY5wXOHunHsfeN/b/gZonM8xGUDPIqhNRDwH09IYOnQoa9du5LvvICMjCdjOY49N529/G+x3eRKC9A1VkRCXlpZGp05d+fDDD3CuIxUrlqVx43rcddfD9O7d2+/yJEQp3EVCWHp6Os2bdyMh4X3gZYYM6c2jj0K1an5XJqFO4S4SonbtyiAysjs7d67g7LNfZO7c3rRs6XdVUlJoPXeRELRgQQa///317Ny5nGuvncm3396qYJfjonAXCSF79kCvXpnceOMNZGa+wwMPPMs77wzg9NP9rkxKGg3LiPjs4MGDTJz4KHFxe/nwQ8jM/AL4iKlTp3PHHYP8Lk9KKIW7iI/S0tJo374rH3/8PlCdcuWgRo3yPPzwdIYMGeJ3eVKCKdxFfHLwYDqRkd3YsuUDypd/mYkTe3PXXVC2rN+VSThQuIv4YOPGDJo1605i4gouvvglli3rTYMGflcl4UQnVEVOoexs+Ne/MrjssutJTFxO//7Ps3FjXwW7FDn13EVOkQ0boH//TNatuwF4h8mTZzJyZH+/y5IwpXAXKUbp6eksXLiM11/PZMkSKFfuNeAtnnlmOkOHDvS7PAljCneRYpKWlkbz5l1ISHjvl7asLOPpp59m6FDNhJHipXAXKQZ796bTqNF1fP/9B5x55nNMmtSCFi2gSpUq1KpVy+/ypBRQuIsUgczMTD799FNycnKIj3f8/e//JCNjJW3avMQbb/SlalW/K5TSRuEucpJSUlLo0KEDH3/8cVCrcf/9z/Ovf/X1rS4p3RTuIichNTWVTp06ERf3CVWrTiM19UJ69YL77qtNw4YX+V2elGIKd5ETdPDgQdq370Jc3Bqce43zzuvJzJlwxRV+VyaiLzGJnJCDB9P4y1+6smbNh5Qp8yqPPNKTtWsV7BI61HMXOU7/+186V111Hfv2fcAFF8xiyZJeXKQRGAkx6rmLHINzjqysLDIzs3j88VQuuaQb+/atpE+fF9mypbeCXUKSeu4iR5GSkkL37t1ZuXJlUKsxadLz3HOPZsJI6FK4ixQgNTWVjh07sWbNGsqWvY/y5avSqRMMGfIXrr22PTgHZr8+Ie9jER8p3EXycfDgQVq06My6dWuA1+jRoydTp8I553gHxMRAUhJMmRIIdOcgOhoiIgL7RHymMXeRPPbvT+OSS7qwbt1HVKv2KvPn92TBgqBgdy4Q7LGxgUDPDfbY2EC7cz5WLxKgnrtIkBUr0rjuuq6kpa2iWbNZLFrUi7POynOQWaDHDoFAj40NbEdF/dqTF/GZeu4iQGoq3H57Om3bdiMt7T3uvfdFVq3q/dtgzxUc8LkU7BJC1HOXUislJYUhQ4aQkLCVHTvg0KF9wA6mTXueYcOOMRMmdygmWHS0Al5ChnruUiqlpqbSrl1H5s59na1bz6Z8+XO46qo/MWfOawwb1v/oTw4eY4+KgpycwH3wGLyIz9Rzl1Ln4MGDXHllZzZtiqNMmdcYPbonY8dCxYqFfAGzwKyY4DH23CGaiAj13CUkmAuBXkZkZKSLj4/3uwwpBb7+Oo0mTTqzd+8q6tefzcKFvWjY8ARfTPPcxWdmluCci8xvn4ZlpFRwDp55Jo2LLurK3r2ruOmmWfzvfycR7PDbIFewSwhRuEvY274d2rVL5/bbu5Od/R4TJ77I3Lm9KV/e78pEio/G3CUspaSkMGnSY3zwQSIffww5OQmYreG5555n4ECtCSPh75jhbmYVgVVABe/4+c65cWZ2HjAHqA4kAH2dc4fMrAIwC7gC2Af0cs5tL6b6RX4jNTWVVq06kZCwGoigfHmoXr0CEyfOZMCA/n6XJ3JKFGZYJhNo7Zz7M9AQ6GBmTYCJwBTn3PlAIjDIO34QkOi1T/GOEzklkpIOcumlnUlIWEPlynOZNWs/mZn72b37JwYMGOB3eSKnzDF77i4wnSbVe1jeuzmgNXCL1/4SEANMA7p52wDzgafMzFwoTMuRki/PjJTkpCRefuUV0tPT2bkTZs5cysGDq2nSZDaLFvWkVi0faxXxUaHG3M2sLIGhl/OBqcDXQJJzLts7ZCdQx9uuA3wP4JzLNrNkAkM3e/O85hBgCEC9evVO7reQ0iHPSoxJiYm0v/hi1u7aFXRQRaKjX+bxx3v5VKRIaCjUbBnn3GHnXEOgLtAYOOlrzzjnZjjnIp1zkTVr1jzZl5Nwl2clxgPJyXS45BI+27Wb2qe/AKRy662p7NqVzOOP33KsVxMJe8c1W8Y5l2Rm7wFNgQgzK+f13usCP3iH/QCcC+w0s3JANQInVkVOXNC3QFNiY2kX+2/igRwWUrHWdbz7rNGmjb8lioSSY/bczaymmUV425WAdsBm4D3gRu+wfsBib3uJ9xhv/0qNt0uRMCNl/HiacCafYuQwl7ujrmPDBgW7SF6F6bnXBl7yxt3LAPOcc8vMbBMwx8weBj4DZnrHzwReNrNtwH7g5mKoW0qhHdtTaHxJC3ZzgLpM4nUeowkfwelTAH07VCRYYWbLfAFcnk/7NwTG3/O2ZwA9i6Q6KdUyMjJISEggJ8exckUOj/zjAbJyvuCGP9zHq1/eTYX7d/x6oQwttStyBH1DVUJSUlIS7du3Z+3atUGtZZhw1U2M+uifWolR5BgU7hJyDhw4QIcOHVi3bj2VKk0nO7sB/fvDXXfV5U//d+GvQZ4b8Ap2kd9QuEtISUlJoWXLDqxfn4Bz82ncuBvPPQfnn1/AExTsIvlSuEvISEpKoVGjjnz77adUrDiP2NhuDB4MZbR2qchxU7jLqXGMC1usXZtKmzadSUmJo1GjOSxe3IO6dX2oUyRMqE8kxS8m5shrizqHu/tucsaNIyMjhwceSOXKK7uQkrKaESNeJT7+RgW7yElSz12KV/CyAQBTppB8xx10feYZPgQYPx4AszJMm/Yyw4ZpTRiRoqBwl+IVPGUxNpYDsbG0w0iwcuCiqVKlMl27wsCBV9NGXzMVKTK6QLacGs6RUqYMTanKRtKA+Qwd2o2JE6FaNb+LEymZjnaBbPXcpfg5x87BI4nkD+xiB7WI5bUe39NqmtNURpFiohOqUryc4/XO/+YPz3/MLrbTtetsvrnjW1q9ceeRJ1lFpEip5y7FZs8eGD48jdfffgOI55FHZjNmTE9wN0L5LC0bIFKMFO5SpJKSkhg2bBjx8d+yYwdkZ+/G7DteevFl+t7qzYTRsgEixU7hLkUmOTmZVq2u5fPPP8O51lSrZlx6aXWioyfTo0ePIw9WsIsUK4W7FImkpAM0atSBb79dx2mnzWfChG7cdReULet3ZSKlk8JdTsi+ffsYM2YM+/btIzUVPvxwM2lpX3HppfNYtKgbDRr4XaFI6aZwl+O2f/9+2rZty6ZNmzjzzAvYvRvMTuOOO17nqaeu14iLSAhQuMtxSUxMpF27dmzcuIn69RezdWsHrrsOpk2D3/3O7+pEJJfCXQotOTmZdu3a8/nnX+LcQpKSOjBnDtx0k86PioQahbsUyoEDB7j66mvZuPFzYAG9e3fiiSegRg2/KxOR/Cjc5Zh+/jmFRo068NNPCVSvPp9Zs7rSqZPfVYnI0Sjc5TcSExOZPXs2mZmZbN0KL7wwn8zMT7n22nnMm9eNqlX9rlBEjkXhLkfYv38/bdq0Yf369b+0mVXkoYfm8OCDPQp+ooiEFIW7/OLXmTCbOfPMZSQnN+POO2HcuNM488yKfpcnIsdB4S5AYE2YVq3as2HDl+TkLKRevU7MnAlXXOF3ZSJyIrTkb7jKu5TuUZbWTUpK5oorruXzzz+nTJkFPPxwJ9auVbCLlGTquYejmJjAdUtzV150LrB2ekREYF+QjRsPcPXVHUhOXseFF85n4cIuXHyxDzWLSJFSzz3cBF+QOvdiGNHRgcdJSWSkpxMXF8fq1Wu4556PueyyjiQnxzNo0Dw2buymYBcJE+q5h5s8F6QmNjawHRXF/rFjaXPVVXlmwpTl6afncvvt15/6WkWk2OgC2eHKOSjz6x9mifv20bpNW778chMwlQoV6jJ0KAwe/Hsuvvgi/+oUkROmC2SXNrlDMZ4k4OoLLmNL4h6cW8T113dk6lSoXdu3CkWkmGnMPdwEj7FHRbHrp0T+74wL2bx/N9VOe5n5r3fgjTcU7CLhTj33cGMWmBUTFcV/Oz5Et/M6kJHxNa3q/pP5t3zLWTdq+UaR0kDhHoZS74nhnr8dYHqHjkA8Y8fOY/xD3bUur0gponAPE/v376dLly6sWbPmlzazssyaNYc+fTQTRqS0OWa4m9m5wCygFuCAGc65WDM7C5gL1Ae2Azc55xLNzIBYoBOQBvR3zq0rnvIFAmvCtGqVOxPmPqpXr8h110Hfvi1p1aqV3+WJiA8K03PPBv7mnFtnZlWABDNbDvQHVjjnJpjZ/cD9wCigI3CBd7sSmObdSzFISkoiMrId33yzkTJlFjF6dEcefBAqap0vkVLtmOHunPsJ+MnbTjGzzUAdoBvQ0jvsJeB9AuHeDZjlAhPo48wswsxqe68jJ2n37t3cfPPNbN26lZwc2LMnlaysgzRo8AYLFnSkYUO/KxSRUHBcUyHNrD5wOfAJUCsosH8mMGwDgeD/PuhpO722vK81xMzizSx+z549x1t3qbRnzx5at25NXFwc9eu3Ze/e9hw+fAODB7/Nli1dFOwi8otCn1A1s8rAAuBu59wBC5p54ZxzZnZcX3V1zs0AZkDgG6rH89zSaO/evbRp04Zt277mT396k48+as0118Bzz8GFF/pdnYiEmkL13M2sPIFgf9U594bXvMvManv7awO7vfYfgHODnl7Xa5MTtG/fPtq2bcuWLVsxW8pXX7Xmqafggw8U7CKSv2OGuzf7ZSaw2Tn3eNCuJUA/b7sfsDio/VYLaAIka7z9xCUmJtKsWTu++GILWVmLaNmyLRs3wvDhRywdIyJyhMIMy1wN9AU2mNl6r20MMAGYZ2aDgB3ATd6+twhMg9xGYCrkgKIsOGw5d+SXjJxjz95kGjZsx48/bqRy5cU8/fS19Omj7yKJyLEVZrbMR0BBcdImn+MdMPwk6ypdvItr7B49mrEPPkhSUhKJa75i1Y9pZB7ezjXXLGTBgg6cfbbfhYpISaFvqPrNu7jGnthYWr/yCttSUznDzmZ/xhmUK1OBMWMW8Mgjnf2uUkRKGI3a+s2MvQ88QJvq1dm6L5EamdPZn/Edg//vefbs/YJHHunqd4UiUgKp5+6z5ORkWrdpx6akgxzmP5xGA96lDW02vKvBdRE5Yeq5+6xnz2g2bPiSw4ff4G42sIFLacPKX69/KiJyAhTuPtm3D9q0+Q/Ll79A9Yq3s4aHmBK1gzNyUiEq6sgLXIuIHCcNy5xizsH8+XDHHQfYu/c2qle/iK8Hn0m1jCaBC1sHX+A6IkJDMyJyQhTup9CPP8Idd8DixVCz5ijMdrJ06WqqNW165Dz33IBXsIvICVK4nwLOwZAhr/L88+PIyckiIgL27PmOkSNH0rRp08BBeYNcwS4iJ0HhXsy++Qa6dn2VTZv6UrlyI9q3v5SqVeGcc85h7NixfpcnImFK4V5MDh+Gf/8bRo16jUOHbuXCC1sSH7+MypVP97s0ESkFFO7FYONGGDQIPvlkHtCHJk2a8e67SznjDAW7iJwamgpZhA4dgvHj4fLLYdOm+ZQpcwvXXHMVy5cv44wzzvC7PBEpRRTuRWTtWoiMhHHj4MorF5Ke/leaNm3CW2+9ReXKlf0uT0RKGQ3LnKS0NLjllkUsXjyXSpXg6qsPExe3kL/85S+89dZbVKlSxe8SRaQUUrifhPffh169XmH37lupVKkWdepUZc8e6Ny5My+99BJVq1b1u0QRKaUU7icgORlGjYLp02cD/WjYsCWrVy/j9NN1wlREQoPC/TgsWbKEF1/8kOXLITX1IGbTueaaZrz99lIFu4iEFIV7IU2Z8hwjR94GVMCsLBUrQqtW1zJv3jzNhBGRkKNwPwbnYNiwF5gxYwhmHRgzZiEPPliR007zuzIRkYIp3I9i507o2nUW69cPolq1dqxcuZBGjSr6XZaIyDFpnns+cnJg+nS44IJXWL++P3/8Yxt27lykYBeREkPhnse2bdCmDQwbNpuMjH40bdqKzz5bTOXKlfwuTUSk0DQs4/nkkwSefvo7XnsNypb9FrN7adGiOcuWLdFMGBEpcRTuQEzMczz00G2/PM7KgmbNmrF06VLNhBGREqlUh3tmJvTs+QJLlw6hfPkO/OMf/+Laa40yZYxLLrmEcuVK9dsjIiVYqU2vuDi44YaX+PHHQdSu3Y61axdSp07FIy93JyJSQpW6E6oHD0J0NDRt+go//jiAhjUu5OttQcEeHQ0xMX6XKSJyUkpVuK9YAZdeCk88MRuzfjT73bms3ruFSmPG/BrssbGQlBR4LCJSQpWKcN+xI5lbbvmZtm1/Ji3tFcqU6UuLFs15+6uNnB4VFQj0MmUC91FRMGWKhmZEpEQzFwI91MjISBcfH18srz1ixEymTh0KHP6lrVmzZrz99tuBmTDOBYI9V06Ogl1ESgQzS3DORea3L2xPqO7aBV26vEB8/G1UrtyWESN68PvfQ6VKlbjxxht/Dfbo6COfGB2tnruIlHhhF+7OwSuvwO23v8TBg4O44IL2JCQsokqVir89MHeMPXcoJvcxKOBFpEQLq3D/7jsYOhTeeecVYABNm7ZhxYqFVKqUz5owZhARceQY+5QpgX0REQp2ESnRwmLMPScHnnkmcHWkrKzZHDrUl5YtW7JsWSEuopF3XrvmuYtICXG0MfdjzpYxs+fNbLeZfRnUdpaZLTezrd79mV67mdmTZrbNzL4ws0ZF92vk76uvoEULGD4c6tefS1ZWX29NmEJeHSlvkCvYRSQMFGYq5ItAhzxt9wMrnHMXACu8xwAdgQu82xBgWtGUmb9+/WZw0UUN+PjjBtSs2YDNm3tz9dVXs2yZrmcqIqXbMcfcnXOrzKx+nuZuQEtv+yXgfWCU1z7LBcZ64swswsxqO+d+KrKKg/zf/9WhXr1raNwYKlWCs88+m3HjxmmxLxEp9U70hGqtoMD+GajlbdcBvg86bqfX9ptwN7MhBHr31KtX74SKuO++ztx3X+cTeq6ISDg76W+oer304z4r65yb4ZyLdM5F1qxZ82TLEBGRICca7rvMrDaAd7/ba/8BODfouLpem4iInEInGu5LgH7edj9gcVD7rd6smSZAcnGNt4uISMGOOeZuZq8ROHlaw8x2AuOACcA8MxsE7ABu8g5/C+gEbAPSgAHFULOIiBxDYWbL/LWAXW3yOdYBw0+2KBEROTmlYslfEZHSRuEuIhKGFO4iImEoJBYOM7M9BE7M+qkGsNfnGo6Xai5+Ja1eUM2nSijU/HvnXL5fFAqJcA8FZhZf0OpqoUo1F7+SVi+o5lMl1GvWsIyISBhSuIuIhCGF+69m+F3ACVDNxa+k1Quq+VQJ6Zo15i4iEobUcxcRCUMKdxGRMFRqw93MtpvZBjNbb2bxXlu+14b1m5ld6NWZeztgZnebWYyZ/RDU3snnOkP6ervHUfMkM9vi1bXQzCK89vpmlh70fj8TQjUX+Fkws9He+/yVmV0bQjXPDap3u5mt99p9f5/N7Fwze8/MNpnZRjOL8tpD+vN8BOdcqbwB24EaedoeBe73tu8HJvpdZz51lyVw9avfAzHAPX7XFFRbc6AR8OWx3lMCq4e+DRjQBPgkhGpuD5TzticG1Vw/+LgQe5/z/SwAlwCfAxWA84CvgbKhUHOe/ZOBB0PlfQZqA4287SrA/7z3MqQ/z8G3UttzL0A3AteExbvv7l8pBWoDfO2c8/sbvb/hnFsF7M/TXNB7+sv1dp1zcUBE7gVgTqX8anbO/dc5l+09jCNw0ZmQUcD7XJBuwBznXKZz7lsCy3E3LrbiCnC0ms3MCCwb/topLeoonHM/OefWedspwGYClwwN6c9zsNIc7g74r5kleNdzhYKvDRtKbubIfwQjvD8Dnw+VYaQ8jvd6u6FmIIEeWa7zzOwzM/vAzJr5VVQB8vsslIT3uRmwyzm3NagtZN5nM6sPXA58Qgn6PJfmcL/GOdcI6AgMN7PmwTtd4G+tkJonamanAdcBr3tN04A/AA0JXIR8sj+VFU4ovqdHY2YPANnAq17TT0A959zlwEhgtplV9au+PErUZyGPv3JkhyVk3mczqwwsAO52zh0I3hfqn+dSG+7OuR+8+93AQgJ/qhZ0bdhQ0RFY55zbBeCc2+WcO+ycywGexYc/twuhRF5v18z6A12A3t4/YryhjX3edgKB8es/+lZkkKN8FkL9fS4H9ADm5raFyvtsZuUJBPurzrk3vOYS83kuleFuZmeYWZXcbQIn0L6k4GvDhoojejh5xvSuJ/A7hJoSd71dM+sA3Adc55xLC2qvaWZlve0GwAXAN/5UeaSjfBaWADebWQUzO49AzZ+e6vqOoi2wxTm3M7chFN5n7zzATGCzc+7xoF0l5/Ps9xldP25AAwIzCD4HNgIPeO3VgRXAVuBd4Cy/aw2q+QxgH1AtqO1lYAPwBYEPV22fa3yNwJ/UWQTGHAcV9J4SmFUwlUCvbAMQGUI1byMwfrreuz3jHXuD93lZD6wDuoZQzQV+FoAHvPf5K6BjqNTstb8IDMtzrO/vM3ANgSGXL4I+B51C/fMcfNPyAyIiYahUDsuIiIQ7hbuISBhSuIuIhCGFu4hIGFK4i4iEIYW7iEgYUriLiISh/wcnkJcMBAKRuwAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ax.plot(inputs, quantized_predictions, color=\"black\")\n", + "display(fig)" + ] + }, + { + "cell_type": "markdown", + "id": "af6bc89e", + "metadata": {}, + "source": [ + "### Now it's time to make the inference homomorphic" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "cbda8067", + "metadata": {}, + "outputs": [], + "source": [ + "q_y = (2**output_bits - 1) / (max_y - min_y)\n", + "zp_y = int(round(min_y * q_y))\n", + "\n", + "q_x = x_q.parameters.q\n", + "q_w = w_q.parameters.q\n", + "q_b = b_q.parameters.q\n", + "\n", + "zp_x = x_q.parameters.zp\n", + "zp_w = w_q.parameters.zp\n", + "zp_b = b_q.parameters.zp\n", + "\n", + "x_q = x_q.values\n", + "w_q = w_q.values\n", + "b_q = b_q.values" + ] + }, + { + "cell_type": "markdown", + "id": "b8e95e3d", + "metadata": {}, + "source": [ + "### Simplification to rescue!\n", + "\n", + "The `y_q` formula in `QuantizedArray.affine(...)` can be rewritten to make it easier to implement in homomorphically. Here is the breakdown.\n", + "```\n", + "(q_y / (q_x * q_w)) * ((x_q + zp_x) @ (w_q + zp_w) + (q_x * q_w / q_b) * (b_q + zp_b)) - (min_y * q_y)\n", + "^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^\n", + "constant (c1) can be done constant (c2) constant (c3) constant (c4)\n", + " on the circuit \n", + " \n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " can be done on the circuit\n", + " \n", + "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + "cannot be done on the circuit because of floating point operation so will be a single table lookup\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "c6e101ae", + "metadata": {}, + "source": [ + "### Let's import the Concrete numpy package now!" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "4da7aed5", + "metadata": {}, + "outputs": [], + "source": [ + "import concrete.numpy as hnp" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "d3816fa5", + "metadata": {}, + "outputs": [], + "source": [ + "c1 = q_y / (q_x * q_w)\n", + "c2 = w_q + zp_w\n", + "c3 = (q_x * q_w / q_b) * (b_q + zp_b)\n", + "c4 = min_y * q_y\n", + "\n", + "f = lambda intermediate: (c1 * (intermediate + c3)) - c4\n", + "f_q = QuantizedFunction.of(f, input_bits + parameter_bits, output_bits)\n", + "\n", + "table = hnp.LookupTable([int(entry) for entry in f_q.table])\n", + "\n", + "w_0 = int(c2.flatten()[0])\n", + "\n", + "def infer(x_0):\n", + " return table[(x_0 + zp_x) * w_0]" + ] + }, + { + "cell_type": "markdown", + "id": "01d67c28", + "metadata": {}, + "source": [ + "### Let's compile our quantized inference function to it's homomorphic equivalent" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "81304aca", + "metadata": {}, + "outputs": [], + "source": [ + "inputset = [int(x_i[0]) for x_i in x_q]\n", + "\n", + "circuit = hnp.compile_numpy_function(\n", + " infer,\n", + " {\"x_0\": hnp.EncryptedScalar(hnp.Integer(input_bits, is_signed=False))},\n", + " inputset,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "c62af039", + "metadata": {}, + "source": [ + "### Here are some representations of the fhe circuit" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "0c533af6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "%0 = Constant(1) # ClearScalar>\n", + "%1 = x_0 # EncryptedScalar>\n", + "%2 = Constant(15) # ClearScalar>\n", + "%3 = Add(1, 2) # EncryptedScalar>\n", + "%4 = Mul(3, 0) # EncryptedScalar>\n", + "%5 = TLU(4) # EncryptedScalar>\n", + "return(%5)\n", + "\n" + ] + } + ], + "source": [ + "print(circuit)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "c1fc0f48", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOMAAAGnCAYAAABFMOCCAABPy0lEQVR4nO2deXxURda/n84eSCCiAiHmJ4sECARBRBAMRGEcEQFlEREBcRlQcBmXEUVHR9xe35kRB15REBEcB0wQYYZFQZCEZUBBUFYJyL6phEBYsp/fH9Wd253ubKS7by/18OlP3773pu+5xf12VZ2qOsciIoJGozGbjBCzLdBoNAotRo3GR9Bi1Gh8hDAzL378OGzfrl4HDsDRo2rfyZOQmwulpZCXB8XFUKcOREZCVBTExUF8PCQkQJMmkJQEKSmQnAx165p5R/7FcWC79XUAOGrddxLIBUqBPKAYqANEAlFAHBAPJABNgCQgBUgGdPFfOhZvOXBKSmDrVsjMVK916+DUKfdew2JRwkxNhR49IC0NEhPdew1/pQTYCmRaX+sANxc/FpQwU4EeQBqgi7/aZHhUjIWFsHIlfPEFLFoEv/xS+fkhIdCoETRsCA0aQGgoxMZCWBhcuAAFBXDxIuTkqFr07Nmqbbj2Whg4EO66S9WewUQhsBL4AlgEVFH8hACNgIZAAyAUiEU1ny4ABcBFIAdVi1aj+LkWGAjchao9NRXiGTHu3QvTp8OsWfDbb87HQ0OVSDp1gnbtoG1baNUKGjdWwqsuFy7AoUOwcyfs2KGau+vXw5Ejrs/v2BHGjoV774WYmEu7N39gLzAdmAW4KH5CUSLpBLQD2gKtgMbUrN9yATgE7AR2oJq764EKip+OwFjgXiCAi/9Sca8YN2yASZNg2TIo/63NmsGdd0KvXnDTTVC/vruu6sy+fZCVpexYtgzOnXM8Xq8ejBkDzzyjauFAYQMwCVgGlP9PbQbcCfQCbgI8WPzsA7KsdiwDyhU/9YAxwDOoWlgDQAbiBjZtErntNhElQePVuLHIhAki33/vjqtcGhcviixaJHL33SIREY721a0r8swzIjk55tnnDjaJyG0iQrlXYxGZICImFr9cFJFFInK3iESIo311ReQZEfHz4ncX6bUS4+nTIuPGiYSGOj7k3buLpKeLFBa6yUw3ceKEyBtviMTHO9rbsKHIxx+LlJaabWHNOC0i40QkVBwf8u4iki4iPlb8ckJE3hCReHG0t6GIfCwiflb87ubSxbhkiUijRo4PdY8eIitXutM+z3Dhgsjkyc6iTEsTOXzYbOuqxxIRaSSOD3UPEfGD4pcLIjJZnEWZJiJ+UvyeoOZiLCwUefllkZAQ4yFu0kRk9mwPmOdhzp9X9xIZadxL/fqqVvdVCkXkZREJEeMhbiIiflj8cl7UvUSKcS/1RdXqQUjNxJiTI5Kaajy4FovIE0+InDvnIfO8xM6dIl26ON7Xa6+ZbZUzOSKSKsaDaxGRJ0TEz4tfdopIF3G8Lx8sfk9TfTEeOiTStq1jP2vpUk/a5l0KC0Wee86xxn/kEZHiYrMtUxwSkbbi2M8KoOKXQhF5Thxr/EdExEeK3xtUT4zHj4u0aGE8pCkp/tO3qilffCESHW3c68MPm+/YOS4iLcR4SFMkcPtWX4hItBj3+rAEjWOnajHm5op06GA8nD17Ki9qILN2rUiDBsY9T5xoni25ItJBjIezpygvaiCzVkQaiHHPJha/N6lajAMGGA9l167+3z+sLhs2qHFI273Pm2eOHQPEeCi7iv/3D6vLBlHjkLZ7N6n4vUl6pUuopk1Tc0oBWreGxYuDZ1VEly4wf74xPW/MGLWyxJtMQ80pBWgNLCZ4VkV0AeZjTM8bg1pZEshUKMYDB+Dpp9V2VBSkp8Pll3vJKh/httvgpZfU9pkz8OCD3rv2AcBa/EQB6UCQFT+3Adbi5wzgxeI3hQrFOHGiWiEB8NZbvrviYevWrfTt25e4uDhiY2Pp3bs369atc9v3T5wI3bur7VWrVOvAG0xErZAAeAvPrnhYunQpSUlJhFUxS/+mm27CYrG4fD355JMesW0iYC1+VqFaB4GKSzFu2QJz56rtdu1g/HhvmlR9Nm7cSLdu3YiNjWXXrl3s37+f5s2bk5aWxvLly91yjdBQmDJFLe8CmDDBeRK8u9kCWIufdoCnin/fvn3079+f559/npMnT3roKrUjFJiC8aBOwHkSfMDgqif58MOG42LxYm/3Y6tHSUmJtG3bVuLj4+XChQtl+4uLi6VVq1aSmJgo+fn5brve8OFGmXzzjdu+1iUPi+G48GTxDxs2TN58800pKiqShIQECQ0NrfT87t27y3fffedBiypmuBhl8o0pFngcZwfOxYuQkaG2mzeH22/39s9D9cjKymLHjh0MHjyY6Ojosv2hoaEMGzaMw4cPs9iNbcrHHze2P/rIbV/rxEXAWvw0BzxZ/DNnzmTChAlVNk99Abvix4PFbypOYlyxQsWfAeWwsFi8bFE1WbVqFQDXX3+90zHbvpUrV7rtejfcYPSbFy5UYUQ8wQpU/BlQDgtPFr/9j5ivcwNGv3khKoxIoOEkxrVrjW131YrlO/733XcfAL1793bYn2v7FagGu3fvBuCqq65yOpaQkADAnj17am+8HbbyyMuDH39061eXYVf8Hq0VL5VPPvmEDh06ULduXerXr09qair/+te/vHJtW3nkAR4qflNxEuPGjeo9JsZ9HtS1a9eydetW6taty7XXXssHH3wAwJIlS+jSpQtz585FRIiLi6v2d9qEW9fFwGeMNabG6dOna227Pd26GdsbNrj1q8uwFj8x+GbMmNOnT/PRRx/xyy+/8O2339KsWTOGDx/O4/bteA9hV/x4qPhNxUmMtvgxSUnKk+gurr32WmbNmsUPP/zAyJEjERHGjBlDr169uOeee9x3IUCs7k6Lm9vYycnGdkVxdmqL7WuTUJ5EX2Lt2rXMmTOH6667jrp169KqVSvmzJnDDTfcwJQpU9ho+yX3EHbFX2GcHX/GSYy2AFJXXun+iw0ZMoSJEyeyYMECbrrpJk6dOsWkSZMu6btstej58+edjtn21aSmrQ72kx5cBdpyB7av9UDxe4zBgwcD8J///Mej17Gf9OCh4jcVl95UAE/17SdNmkSXLl1Yv349Q4YMISTk0oKat27dGoAjLqqoo0ePApCUlHTphrrAvkXs4jfALdgG+v3HtQLx8fEA/FJVLM5aYt8h8VDxm4qTEi67TL27ubtVxurVqzlz5gwpKSk8+uij/PDDD5f0PTfffDMAmzdvdjpm29erV69LN9QF9kGXPTU10Fr8eKj4PcKxY8cAaOjhUHv2QZcDcWqgkxhtD5knJmTs37+fBx98kM8//5x///vfREdHM2DAAH799dcaf1fPnj1JTk5m/vz55Ofnl+0vKSlh3rx5JCYm0rdvX3ea7xCEuUEDt351GbaHzNfmw3z44Yd06tTJab+IkJ6eDkC/fv08aoN9veuh4jcVJzG2aqXe9+xRk6Pdxblz57jzzjuZPHkyycnJNG3alPnz53Ps2DEGDx5MUVFRjb4vJCSEmTNnkpOTw+jRozlx4gSnTp1i3LhxZGdnM2PGDKKiotx3A8B33xnbbdq49avLsBY/e1CTo32J77//nnHjxrF3717y8/P56aefGDFiBJs3b+axxx6jS5cuHr2+XfHjoeI3l/Jzcv73f41pX1995Z55PuPGjRPUlEIBZNu2bfLrr7867ANk0qRJNf7u77//Xvr06SP16tWTmJgYueWWW2Tt2rXuMbwco0cbZXPokEcuIf8rxrQvNxV/hfznP/9x+j+wvWbMmOFwbn5+vmRkZMhdd90lLVq0kMjISKlfv76kpaXJv/71Lw9bqhgtRtl4qPjNJN0povi336q1fKBm4Hz4oZd+FXyc/HyV9SonR0VH//lnz1znW9RaPlAzcHTxK/JRWa9yUNHRPVT8ZuKcubhzZzXGCPDZZ86h8YOVhQuVEAFGjPDcdTqjxhgBPsM5NH6wshAlRAAPFr+pOInRYoH771fb587Bu+962SIfpLQU3n5bbVssMGqU565lAe63bp8DdPGrPJHW4scCeLD4TcXlIN+YMcYQx9tvV53KzZ1UtHjV/vXKK694zyBgzhy1xhNg6FC1msWTjMEY4nibqlO5BTpzUGs8AYaiVrMEIhVmofrrX+HZZ9X2wIHw+efeNMt3OHlSpa87eRIiIlT6uRYtPH/dvwLW4mcgEKTFz0lU+rqTQAQq/ZwXit8MnPuMNsaPV0GoABYsgBkzvGWT71BaCsOHG2Ouf/yjd4QIanW/tfhZAARh8VMKDMcYc/0jAStERWW+1h9/FImKUq78iAj3DXX4C08+aQxldOokUlDg3ev/KCJRolz5EeL5oQ5f40kxhjI6iYiXi9/bVB6qMSXFcFwUFsKQIeBi9llA8uqrMHmy2r7sMpg3TzVTvUkKhuOiEBgCBEnx8yow2bp9GTAP1UwNaKoj2aefNmqImBiRL7/09I+EeZSWqsxUtvuNjhbJyjLXpqfFqCFiRCSAi19KRWWmst1vtIiYXPzeonq5NkpLRUaNMh7QyEj/TAFXFefPiwwbZtxnRITKemw2pSIySowHNFL8MwVcVZwXkWFi3GeEqKzHQUL1s1CVrzFAZMQIkbw8D5rnRXbuVAl97FsAy5aZbZVB+RoDERkhIgFS/LJTVEIf+xaADxW/N6h5stR33hEJCzMe2pYtRZYv94BpXiI/X+T11x0zTyUmimzZYrZlrnlHRMLEeGhbiogfF7/ki8jr4ph5KlFEtphok0lcWhrxdetErr7asZYcMkRk/373WudpliwRSUpyvI8BA0ROnTLbsspZJyJXi2MtOURE9ptn0iWxRESSxPE+BoiIjxe/p7g0MYqoLMYjR6osv7YHOTxc5KGHRH7+2Z02up9ly1RGLXsRxsWJTJtmfi7G6pIjIiNFZfm1PcjhIvKQiPh48csyURm17EUYJyLTJGhyMbri0sVoIyvLsa8FKvtv794i6em+k/n3zBmRDz5wzDVpX6ufOGG2hZdGljj2tRCV/be3iKSL72T+PSMiH4hjrkn7Wt1Pi9+d1F6MIiJFRSIffyxyzTXOD3rTpiLPPqvyHXq71jl3TiQjQ+SeexxzLYKq0fv1E9m0ybs2eYIiEflYRK4R5we9qYg8KyrfobdrnXMikiEi94hjrkVE1ej9RCQAit9dOK9nrA3FxSphztSpal1keRIS4JZboEcPSE01ogq4i4ICtRo/MxPWrIGsLCPAlo3ISBg0CJ56ClxEkfBrilEJc6ai1kWWJwG4BegBpGJEFXAXBajV+JnAGiALI8CWjUhgEPAUEGDFX1sy3CpGe7ZsgenTVcLRisIa1qunYpGmpKg1lPHxkJgIjRqpY1FRKiJbRIRazlVUBGfPqtfhw2rO6MGDavL29u2Qna1+EFzRrh2MHAmjR8MVV3jijn2LLcB0VMLRisIa1kPFIk1BraGMBxKBRtZjUaiIbBGo5VxFwFnr6zBqzuhB1OTt7UA26gfBFe2AkcBoIAiK/1LwnBhtlJSommrBAvjqK9i715NXOwPUByA8XC2U7tdPrTpxc9RGv6EEVVMtAL4CHIr/zBmoX98j1w1HLZTuh1p1EqTFXxM8L8byHDumxLl+varNtm1zDIF4KYSEQEzMeCIj9/Doo8tJTYWuXYMn5XlNOIYS58pffuHj5s2pu3AhZ3v3rtV3hgBNUTVsB1QTuCvBk/LcTXhfjK44cULFlDl+HI4eVc3PvDzVB7xwQb3HxkJYmMoBUq+e6n/Gx6v3li1h9eol9OvXj507d5YFONZUzKRJk3jnnXc4cuQIZ+vU4WfgOHAU1fzMQ/UBL1jfY4EwVA6Qeqj+Z7z1vSVaeG7AN8ToDkSE1q1bc+uttzJlyhSzzfFpiouLyxLWvPXWW2abo1FUvLjY37BYLPzhD39g9uzZnD171mxzfJoFCxZw7NgxxowZY7YpGjsCRowADz74IKWlpcyZM8dsU3yaqVOn0q9fP5o1a2a2KRo7AkqMcXFxDB8+nH/84x8ESOvb7Wzfvp01a9Ywfvx4s03RlCOgxAjw+OOPs3fvXr7++muzTfFJ/vGPf9CmTRu3JwXS1J6AE2Pbtm3p0aMHU6dONdsUnyM3N5d//etfjBs3zu2JZDW1J+DECDB+/HgWL17M/v37zTbFp/jwww8JCQlhhCdDomsumYAU45133kmTJk2YNm2a2ab4DKWlpUybNo3Ro0dTr149s83RuCAgxRgWFsaYMWP48MMPuXDhgtnm+ARLlixh//79PPLII2aboqmAgBQjwJgxY7h48SJz58412xSfYOrUqfzud7/Ts5N8mIAV45VXXsmQIUP0bBwgOzubr7/+Wg9n+DgBK0aAJ554gh9++IG1a9eabYqpTJ06lcTERG6//XazTdFUQkCLsVOnTtxwww1BPcxx7tw5Zs+ezfjx4wkNDTXbHE0lBLQYQQ1zfP755xw9etRsU0xh9uzZFBYWMnr0aLNN0VRBwItx6NChXH755UyfPt1sU0zh/fff57777uPyyy832xRNFQS8GCMiInjooYd4//33KSgoMNscr/L111+zfft2xo4da7YpmmoQ8GIEePTRRzl9+jSfB1nG16lTp5Kamsp1111ntimaahAUYmzSpAkDBgwIKkfOoUOHWLx4sR7O8COCQoygHDn//e9/2bRpk9mmeIX33nuPRo0acdddd5ltiqaaBI0Ye/bsSfv27fm///s/s03xOAUFBcyaNYuxY8cSHh5utjmaahI0YgTVd5w7dy6//PKL2aZ4lE8//ZTc3Fwefvhhs03R1ICgEuOIESOoW7cuH330kdmmeJRp06YxZMgQGjdubLYpmhoQVGKsU6cO999/P9OmTaO4otDjfs66devYtGmTdtz4IUElRlBN1SNHjvCf//ynVt+zdetW+vbtS1xcHLGxsfTu3Zt169a5ycpLZ+rUqVx33XV07dq1ynOXLl1KUlISYWFhXrBMUxVBJ8YWLVrQp0+fWg1zbNy4kW7duhEbG8uuXbvYv38/zZs3Jy0tjeXLl7vR2ppx/PhxFixYwOOPP17pefv27aN///48//zznDx50kvWaarEjNxXZrNs2TIB5Mcff6zx35aUlEjbtm0lPj5eLly4ULa/uLhYWrVqJYmJiZKfn+9Oc6vNyy+/LFdccYVcvHix0vOGDRsmb775phQVFUlCQoKEhoZ6yUJNJbgnP6O/UVpaKq1atZJHHnmkxn/7zTffCCCPPfaY07FXXnlFAJk/f747zKwRhYWFkpCQIC+88EKV59r/iGgx+gzpQddMBRV9fOzYsXzyySecOXOmRn+7atUqAK6//nqnY7Z9K1eurL2RNWT+/PmcOHGiWsMZ0dHRXrBIU1OCUoygoo+HhITw8ccf1+jvdu/eDcBVV13ldCwhIQGAPXv21Nq+mjJ16lQGDBhA06ZNvX5tjXsIWjHGxsYyfPhwpk6dSmlpabX/Ljc3F4C6LvLNxcTEAHD69Gm32Fhdtm7dyvr16/Vwhp8TtGIENV913759bvOAijWlgLcDBE+ZMoXk5GTS0tK8el2NewlqMSYnJ3PzzTfXaJgjLi4OgPPnzzsds+2zneMNTp8+zbx583jsscd0lHA/J6jFCKp2XLp0abX7ebZQh0eOHHE6ZgvtkeTFnOXTp08nIiKC++67z2vX1HiGoBdj//79ufrqq/nggw+qdf7NN98MwObNm52O2fZ5K6lMSUkJH3zwAaNHjy7rr2r8GLMHV3yBN954Q+Li4uTcuXNVnltSUiLJycnSpEkTh8H14uJiadOmjSQmJlY56O4uvvjiC7FYLPLTTz9d8nfocUafITjHGcvz8MMPk5+fz6efflrluSEhIcycOZOcnBxGjx7NiRMnOHXqFOPGjSM7O5sZM2YQFRXlBavVcMZtt93m1WaxxnNoMQJXXHEF99xzT7WTrHbt2pX169dz5swZWrVqRdOmTcnOzmb16tX8/ve/94LFsGvXLlatWnVJwxmLFy/GYrFgsVg4evQoJSUlZZ8//PBDD1irqQ4Wqc7TFwRs2bKF6667jtWrV9OzZ0+zzamS8ePH8+WXX7Jnzx5CQvRvagCQof8XrXTs2JEbb7zRL4JW5eXl8cknnzBu3DgtxABC/0/aMX78eBYuXOhy2MKXmDVrFsXFxYwaNcpsUzRuRIvRjiFDhtCwYUPef/99s02pEBFh2rRpjBw5kgYNGphtjsaNaDHaER4ezkMPPcT06dPJz8832xyXLF++nN27d+ukpwGIFmM5xo4dS25uLhkZGWab4pKpU6eSlpZG+/btzTZF42a0GMsRHx/PwIEDmTx5stmmOHHw4EGWLVumV2cEKFqMLhg/fjzff/893377rdmmODB16lQaN25M//79zTZF4wG0GF1w00030alTJ58a5rh48SKzZs3i0Ucf1VHCAxQtxgp45JFH+Oyzz3wmeto///lPzp07x4MPPmi2KRoPocVYAffeey+xsbE+Mz3s/fff55577qFRo0Zmm6LxEFqMFRAdHc0DDzzAe++9R1FRkam2ZGVl8f333zNu3DhT7dB4Fi3GShg3bhwnT55k0aJFptoxdepUunTpQufOnU21Q+NZtBgr4eqrr6Zv376mOnKOHTvGwoUL9XBGEKDFWAXjx48nMzOTH3/80ZTrv//++8TFxTF48GBTrq/xHlqMVdC7d29at27tkGT1yJEjvPjiiwwcONBt1zl69CidO3dm9uzZZVPxCgsLmTFjBmPHjvXagmWNiZgbacA/mDJlitSpU0f+/e9/y6BBgyQ0NFQAadGihduusX37dgHEYrFI/fr15fnnn5fJkydLWFiYHD582G3X0fgs6ToXWBUUFBQQHh5OdHQ0/fv3Jzw8nJKSEoAapwaoDFvgYxHhzJkz/O1vf6OoqIjmzZuzZcsWEhISdCjGAEc3Uyvg559/ZsKECTRq1Ihx48aVicV+mCMvL89t1ysfhbywsBAR4eDBg/Tv358WLVrw7rvvcu7cObddU+Nb6LAbLli5ciW33norFoulrBasiPz8fCIjI2t9zX/+85+MGjWqwlQDFosFEaFZs2b8+OOPOjRj4KHDbriiV69ePPvss9UKTmXLvVFbcnNzCQ0NrfSciIgIPvnkEy3EAEWLsQLefPNN7r///ioF4i4x5uTkVBrPxmKxMHfuXLp37+6W62l8Dy3GCrBYLEyfPp077rij0pz37so4lZubW2FNbLPFnUMpGt9Di7ESQkNDmTt3Lp07d65w2ZI7m6mu+qcWi4U33nhDr9YIArQYqyA6Opply5bRsmVLJ0FaLBa3ifH06dNOYgwJCWHs2LFMmDDBLdfQ+DZajNWgfv36rFixgoYNGzoIMiwszG3N1F9//dXhc1hYGIMHD/apBc4az6LFWE2aNGnC6tWriYmJKetDhoSEuNWBYyM8PJzu3bszZ84cHaQ4iND/0zXgmmuu4auvviI8PLxMJO7sM4ISYnJyMosXL3bL+KXGf9DT4WpI586d+fzzz+nXrx8FBQWc3r4d3nkHDhyAo0fh+HE4eRJyc6G0FPLyoLgY6tSByEiIioK4OIiPh4QEaNIEkpLIs4rxqquuYsWKFXossRocB7ZbXweAo9Z9J4FcoBTIA4qBOkAkEAXEAfFAAtAESAJSgGSgrvfMd0LPwKkuJSWwdStkZkJmJv9ctYqR584xCKhthNVS1K/iFcC3zZvT9JZboEcPSEuDxMRafntgUAJsBTKtr3XAKTdfw4ISZirQA0gDvFj6GVqMlVFYCCtXwhdfwKJF8MsvDof/DnwJLLftCAmBRo2gYUNo0ABCQyE2FsLC4MIFKCiAixchJ0fVomfPAupXvBmQhfqFduDaa2HgQLjrLkhxOhrQFAIrgS+ARcAvlZ9OCNAIaAg0AEKBWNQP3QWgALgI5KBq0bPVsOFaYCBwFy7+b9yLFqNL9u6F6dNh1iz47Tfn46GhSiSdOpFeVMTd990HrVpB48ZKeNXlwgU4dIjj69ax97//JTUvD9avh4oS73TsCGPHwr33QgA3Y/cC04FZgIvSJxQlkk5AO6At0ApoTM36XReAQ8BOYAequbseqCjtUUdgLHAv4IHS12J0YMMGmDQJli2D8sXSrBnceSf06gU33QT163vOjn37ICtL2bFsGZRfqVGvHowZA888o2rhAGEDMAlYBpR/KJsBdwK9gJsAD5Y++1CtlGXWV/l1MvWAMcAzqFrYTWToxcUiIps2idx2m4iSoPFq3FhkwgSR7783z7aLF0UWLRK5+26RiAhH++rWFXnmGZGcHPPscwObROQ2EaHcq7GITBARE0tfLorIIhG5W0QixNG+uiLyjIi4qfTTg1uMp0+LjBsnEhrq+JB37y6Sni5SWGi2hY6cOCHyxhsi8fGO9jZsKPLxxyKlpWZbWCNOi8g4EQkVx4e8u4iki4iPlb6cEJE3RCReHO1tKCIfi0gtSz+IxbhkiUijRo4PdY8eIitXmm1Z1Vy4IDJ5srMo09JE/CRExxIRaSSOD3UPEfGD0pcLIjJZnEWZJiK1KP0gFGNhocjLL4uEhBgPcZMmIrNnm21ZzTl/Xt1LZKRxL/Xrq1rdRykUkZdFJESMh7iJiPhh6ct5UfcSKca91BdVq18CQSbGnByR1FTjwbVYRJ54QuTcObMtqx07d4p06eJ4X6+9ZrZVTuSISKoYD65FRJ4QET8vfdkpIl3E8b4uofSDSIyHDom0bevYz1q61Gyr3EdhochzzznW+I88IlJcbLZlIiJySETaimM/K4BKXwpF5DlxrPEfEZEalH6QiPH4cZEWLYyHNCXFb/pWNeaLL0Sio417ffhh0x07x0WkhRgPaYrUqm/l03whItFi3OvDUm3HThCIMTdXpEMH4+Hs2VN5UQOZtWtFGjQw7nniRNNMyRWRDmI8nD1FeVEDmbUi0kCMe65m6QeBGAcMMB7Krl39v39YXTZsUOOQtnufN88UMwaI8VB2Ff/vH1aXDaLGIW33Xo3STw/sJVTTpqk5pQCtW8PixVDXzHn5XqRLF5g/35ieN2aMWlniRaah5pQCtAYWY+6qCG/SBZiPMT1vDGplSWUErhgPHICnn1bbUVGQng6XX26qSV7nttvgpZfU9pkz4MU4OgcAa+kTBaQDQVb63AZYS58zQFWlH7hinDhRrZAAeOutoFvxUMbEiWAL77hqlWodeOOyqBUSAG/h8RUPLlm6dClJSUmVRvfzNBMBW3DNVajWQUUE5kTxLVugUyfVW2rXTq1DrCL+qafo2rUrV1xxBYu9JAKXbNkC11+vFju3bQvbtoEH83ZsQa2oENSqiq2olRbeYt++ffzxj3/k4MGDHDhwgPPnz1NcXOxFCxzZAlyPWrfaFtiGWjtZjgCNKD5tmrHq4q23TBOiz9CxIwwbprZ37FALpD3INIxVF2/hXSECvPTSS3Tr1o3NmzcTGxvr5as70xGwlj47UIujXRF4Yrx4ETKsa++bN4fbbzfXHl/h8ceN7Y8+8thlLmJEPmgOmFH6M2fOZMKECaY2T8tjV/pUVPqBJ8YVK1T8GVAOC51GTXHDDUa/eeFCFUbEA6xARS4A5bAwo/Sjo6NNuGrl3IDRb16ICiNSnsAT49q1xrauFR2xlUdeHngoLbpd6ZtSK/oytvLIA1yVfuCJceNG9R4TE7we1Iro1s3Y3rDBI5ewlj4xmONB9WXsSh9Xpe87jWp3YYsfk5TkdcdNWFhYhfkcy2cdbtSoESdOnPCGWQbJycZ2RXF2aontW5PwvuPG17ErfZdxdgJPjLYAUlde6fVLu3Kf+8TQhg37SQ+uAm25Adu3er/0fR/7SQ+uSj/wmqm2gX4f7MSbjv1UwPPnPXIJ20C/Ln1n7KcCuir9wBPjZZepdzclpAkoTtmF/fXQ1EBr6aNL3xn7oMuuSj/wxGh7yE6eNNcOX8Q+CHODBh65hO0h06XvjH0QZlelH3hibNVKve/ZoyZHawy++87YbtPGI5ewlj57UJOjNQZ2pY+r0g88MdomRZeWGsMcGsX69cb2jTd65BK2SdGlGMMc3mbx4sVYLBYsFgtHjx6lpKSk7POHH35oklUqWrkNV6UfeBPFv/1WreUDNQPHxML3KfLzVdarnBwVHf3nnz1ymW9Ra/lAzcDRpa/IR2W9ykFFR3dR+gE4UbxzZzXGCPDZZ86h8YOVhQuVEAFGjPDYZTqjxhgBPsM5NH6wshAlRICKSj/wxGixwP33q+1z5+Ddd001xycoLYW331bbFguMGuWxS1mA+63b5wBd+qrJbi19LEBFpR94YgQVYsI2xPH2206p3IKOOXPUmkaAoUPVahYPMgZjiONtqk7lFujMQa1pBBiKWs3iisAUY4MG8MILavvsWXjkEXPtMZOTJ2HCBLUdEQGvvebxSzYArKXPWSCIS5+TgLX0iQAqK/3AFCPA+PEqCBXAggUwY4a59phBaSkMH26Muf7xj9CihVcuPR4VhApgARCEpU8pMBxjzPWPQKWl77lgdT7Ajz+KREWpUIURESJffWW2Rd7lySeNUI2dOokUFHj18j+KSJSoUIURIhJkpS9PihGqsZOIVFH6AR6qMSXFcFwUFsKQIbB5s7k2eYtXX4XJk9X2ZZfBvHmqmepFUjAcF4XAECBISp9XgcnW7cuAeahmaqV4/vfBB3j6aaOGiIkR+fJLsy3yHKWlKjOV7X6jo0Wyskw16WkxaogYEQng0pdSUZmpbPcbLSLVLP0giCguoh7QUaOMBzQy0j9TwFXF+fMiw4YZ9xkRobIem0ypiIwS4wGNFP9MAVcV50VkmBj3GSEq63E1CRIxijjXGCAyYoRIXp7ZlrmHnTtVQh/7FsCyZWZbVUb5GgMRGSEiAVL6slNUQh/7FkANSz+IxGjjnXdEwsKMh7ZlS5Hly8226tLJzxd5/XXHzFOJiSJbtphtmUveEZEwMR7aliLix6Uv+SLyujhmnkoUkS01/6ogFKOIyLp1Ildf7VhLDhkisn+/2ZbVjCVLRJKSHO9jwACRU6fMtqxS1onI1eJYSw4Rkf3mmXRJLBGRJHG8jwEicomlH6RiFFFZjEeOVFl+bQ9yeLjIQw+J/Pyz2dZVzrJlKqOWvQjj4kSmTTM9F2N1yRGRkaKy/Noe5HAReUhEfLz0ZZmojFr2IowTkWlS7VyMrghiMdrIynLsa4HK/tu7t0h6us9k/pUzZ0Q++MAx16R9rX7ihNkWXhJZ4tjXQlT2394iki41yvzrUc6IyAfimGvSvlZ3Q+lrMYqISFGRyMcfi1xzjfOD3rSpyLPPqnyH3q51zp0TycgQuecex1yLoGr0fv1ENm3yrk0eoEhEPhaRa8T5QW8qIs+Kynfo7Tr/nIhkiMg94phrEVE1ej8RcWPppwfeesbaUFwMc+fC1KlqXWR5EhLgllugRw9ITTWiCriLggK1Gj8zE9asgawsI8CWjchIGDQInnpKJfcJIIqBucBU1LrI8iQAtwA9gFSMqALuogC1Gj8TWANkYQTYshEJDAKeQiX3cSMZWowVsWULTJ+uEo5WFNawXj0VizQlRa2hjI+HxERo1Egdi4pSEdkiItRyrqIiNXH97Fk4fFjNGT14EHbuhO3bITtb/SAAe4Fr7K/Vrh2MHAmjR8MVV3j67k1nCzAdlXC0oqCS9VCxSFNQayjjgUSgkfVYFCoiWwRqOVcRauL6WeAwas7oQWAnsB3IRv0guKIdMBIYDXio9LUYq6SkRNVUCxbAV1/B3r0ev+R/Ub/861JS6HLvvTBwoLFgOsgoQdVUC4CvUD9SFVJY6LYpf+GohdL9gIEYC6Y9iBZjjTl2TIlz/XpVm23b5hgC8VIICYGmTVUN26EDpKbSa9IkcvPy+PbbbwkN9pR2dhxDiXM9qjbbhjUEYn4+NGwIn34K/frV6DtDgKaoGrYD6oewK15Pea7F6BZOnFAxZY4fh6NHVfMzL0/1AS9cUO+xsRAWpnKA1Kun+p/x8eq9ZUvHAMPAzp076dChA//4xz8YO3asSTfmH5wAPs/MZHxaGn/++WeKmzUjD9UHvGB9j0WFz49BNWETUM3aBKAlXheeKzICL7y/GTRurF5uJDk5mSeeeIIXXniBQYMGcaUJ6Qr8hcbAqcxMEhMT+UuzZmabc8kE9hIqP+fPf/4zderU4QVb1AJNhWRmZpKWlma2GbVCi9GHiY2N5a9//SsfffQRGzyUwi0QKCwsZMOGDfTs2dNsU2qF7jP6Ab169SI3N1c7cypg7dq1pKamkp2dzTXXXFP1H/gmARg3NQCZMmUK27ZtY0YwxvGpBpmZmcTHx/uzEAHdTPUL7J05v/76q9nm+ByZmZncfPPNZptRa7QY/QTtzHFNcXFxQPQXQYvRb9DOHNds2rSJvLw8LUaNd7nnnntIS0tj3LhxlJSUmG2OT7B69WoaN25MUgBMF9Ri9DO0M8eRzMxMevbsicViMduUWqPF6GdoZ45BcXEx69evD4gmKmgx+iXamaPYsmULZ8+e1WLUmId25ihWr17NlVdeSRsPpUT3NnoGjh8T7DNz7rjjDqKjo8nIyDDbFHegZ+D4M8HszCkpKWHdunUB00QF3Uz1a4LZmfPDDz+Qm5urxajxHYLVmZOZmUmDBg1o27at2aa4DS1GPydYnTmZmZn06NGDkJDAeYQD506CmGCbmVNaWsratWsDqokKWowBQzA5c7Zt28apU6f8fmV/ebQYA4RgcuZkZmZSv359UlJSzDbFrWgxBhDB4syx9RcDbWxVizGACAZnjogEZH8RtBgDDk84c7Zu3Urfvn2Ji4sjNjaW3r17s27dOrd8d03ZsWMHv/zyS5X9xaVLl5KUlERYmP9EI9ViDEDc6czZuHEj3bp1IzY2ll27drF//36aN29OWloay5cvd4O1NSMzM5N69erRoUMHl8f37dtH//79ef755zl58qR3jaslem5qgPLss88yc+ZMfvrpp0sOgFxaWkr79u3Jyclh3759REdHA2oqWtu2bblw4QLZ2dlERka60/RKufvuuzl//jxLlixxefzee++lffv2PPPMMzRt2pQTJ05QXFxROhufQs9NDVTc4czJyspix44dDB48uEyIAKGhoQwbNozDhw+zePFid5hbLUSErKysSvuLM2fOZMKECX7VPLWhxRiguMOZs2rVKgCuv/56p2O2fStXrrx0I2vI7t27OXnyZKVitP/R8De0GAOY2jpzdu/eDcBVV13ldCwhIQGAPXv21M7IGpCZmUlMTAzXXXed167pTbQYA5zaOHNyc3MBqFvXOUdTTEwMAKdPn66VfTUhMzOT7t27Ex4e7rVrehMtxgDHUzNzbH4/bwaCqqq/6O9oMQYBl+rMiYuLA+D8+fNOx2z7bOd4mj179nDs2DEtRo1/c6nOnNatWwNw5MgRp2NHjx4F8Fq80szMTOrUqePSmRQoaDEGCZfizLHlr9i8ebPTMdu+Xr16uc/ISsjMzKRbt25ERER45XpmoMUYRNicOdOnT6/W+T179iQ5OZn58+eTn59ftr+kpIR58+aRmJhI3759PWWuA2vWrAnoJipoMQYVycnJPPnkk0ycOLFazpyQkBBmzpxJTk4Oo0eP5sSJE5w6dYpx48aRnZ3NjBkziIqK8rjd+/bt49ChQ1qMmsCips6crl27sn79es6cOUOrVq1o2rQp2dnZrF69mt///vcetlaRmZlJVFQUnTt3rvLcxYsXY7FYsFgsHD16lJKSkrLPH374oResvXT03NQgZN68eQwfPpx169bRtWtXs82pklGjRnH48OGyGUEBip6bGoz4W8wcW3KbQEeLMUipqTPHLA4dOsTBgwe1GDWBS02dOWbxzTffEBkZSZcuXcw2xeNoMQYx/hAzJzMzky5duvj1aozqosUYxMTExPh8zJxg6S+C9qZq8N1sVkeOHCExMZGvv/7aazN9TER7UzW+68xZvXo1ERERfjH84g60GDU+68zJzMykc+fOLtdTBiJajBrAN505wdRfBC1GjRV7Z85///tfs83h+PHjZGdnB5UYtQNH44CvOHPmzp3LyJEjycnJITY21jQ7vIh24GgcmTp1qk84czIzM7n++uuDRYiAbqZqytGmTRufcOYEW38RtBg1LvCmM+f06dO89NJLfP3112VxdX755Rd++umnoBOj7jNqXGJbZrV27VpuvPFGp+MlJSVu6VMWFBQQHR2NiBAaGsp1111HQkIC//73vzl8+DBNmjSp9TX8hAwtRk2FuHLm7N+/nyeffJKRI0cyaNAgt1ynXr165OXllX0ODw+nqKiI0NBQ2rdvz+9+9zt69uxJampqIPchMxCNpgJ27twp4eHh8t5778nFixflL3/5i0RERAggzz33nNuu06JFCwEqfIWHhwsgf/vb39x2TR8kXdeMmkr505/+xCeffEKdOnU4ePBg2WLk1NRUsrKy3HKNnj17VvpdYWFhXHvttWzcuNGn5s66mQz/S9Wj8RpHjhzhwIEDnDhxgpCQEEpLS8uOff/995SWlhISUnsfYGJiotP322OxWJg9e3YgCxHQ3lSNCwoLC3n33Xdp2bIlCxcuBHASyvnz58nOznbL9Ro3blxh/ozQ0FBeeeUV2rZt65Zr+TK6ZtQ4cPDgQW6++WYOHDhAZT2YkJAQvvvuO1q1alXrazZq1MjltcLCwkhKSuLZZ5+t9TX8AV0zahy4+uqr+dOf/kRISEilzcKwsDC+++47t1yzcePGLrMLiwhz5swJ2KxT5dFi1DgxduxYvvnmG2JjYyvMAFxYWMi6devccr34+HinZnBoaCjPP/88nTp1css1/AHtTdVUyL59++jTpw8HDhygqKjI6XhERATnzp2rdc21fft2UlJSyj6HhYVx9dVXs337dq9ELPcR9ERxTcW0aNGCzZs306tXL5dN1sLCQnbs2FHr6zRu3Njhc0lJCbNnzw4mIQK6maqpgtjYWBYvXswzzzzjdMxd/cbLL7+8rDkcFhbGU089Rffu3Wv9vf6GFqOmSkJDQ3nrrbeYMWMGYWFhDrWkO8RosVho0KABAAkJCbz66qu1/k5/RA9taKrNQw89RJs2bejXrx95eXkUFxc7OHGOc5zt1n8HOMBRjnKc45zkJLnkUkopeeRRTDF1qEMkkUQRRRxxXGh0AX6F7rO7M6/OPFJIIZlk6hIc8W9AO3A0l8C+ffvoc3sfsvdkYwm1cHve7WyI3sApTl36l94BJALTjF0WLCSRRCqp9KAHaaSRSGJtzfdV9KoNTfUppJCVrOQLvmDhmYX8evevsBxYB3RzPj+EEBrRiIY0pAENCCWUWGIJI4wLXKCAAi5ykRxy+Pm1nyl4ogCqWJRxLdcykIHcxV2kkFL5yf6FFqOmavayl+lMZxaz+I3fjAMlwEQIaRJCh8c70IlOtKMdbWlLK1rRmMaEVbMnVFRURFF4EYc4xE52soMdbGc761nPEY64/JuOdGQsY7mXe4khxg13aipajJqK2cAGJjGJZSxDcHxMmtGMO7mTXvTi/x34f6Q09VwttY99ZJHFMuu/c5xzOF6PeoxhDM/wDA1p6DE7PIwWo8aZzWzmRV7kS7502N+YxtzP/dzN3XSkoym25ZPPcpbzKZ+ykIUUUlh2rC51eYRHeIEXuIzLTLGvFmgxagxyyeVFXuR93qcEI4lqd7rzBE9wJ3cSju/MEz3JST7iI6YwheMcL9vfkIa8zduMZCQWLCZaWCO0GDWKpSzlAR7gJCfL9vWgBy/zMrdwi4mWVc1FLjKd6fwP/+MgyjTS+IRPuIqrTLSu2ujpcMFOEUW8wiv0o1+ZEJvQhNnMJpNMnxciQDTRPMET7GUvL/MykUQCsJrVtKMdGWSYbGH10DVjEHOa0wxgAGtYA6hxvcd5nNd53a8H23exi9GMZiMbAXVfk5jERCaabFml6JoxWDnMYVJJLRNiQxqyhCVMZrJfCxGgDW1Ywxqe4zlCCEEQXuRFHuVRh76wr6FrxiDkBCe4iZvYxz4AUkhhKUv9pW9VIxaykHu5l4tcBOBhHuYDPvBFx46uGYONM5yhD33KhNiTnmSRFZBCBLiTO1nBChqgJqLPYAYv8ZLJVrlGizHIGMUotrIVgK50ZQlLiCPOVJs8TXe6s5SlZc3v13mdz/jMZKuc0WIMIqYxjUUsAqA1rVnMYr/vH1aXLnRhPvPLpueNYQwHOGCuUeXQYgwSDnCAp3kagCiiSCedy7ncZKu8y23cVtZEPcMZHuRBky1yRIsxSJjIxDInxlu8ZeqKhxYtWvDpp5+acu2JTKQ7KorAKlaxmMWm2OEKLcYgYAtbmMtcANrRjvGMN9WeqKgoIiMjTbl2KKFMYQoh1kd/AhOcJsGbhRZjEDCNaWUP3Fu8RSjeDZP/2Wefceutt/Ljjz8CEBkZSWRkJIWFhfz973/n5ptvprCwsIpvcR8d6cgwhgGwgx1kkum1a1eGFmOAc5GLZdPBmtOc27nd6zakpaWRmppKv379eOihh8jPz2fFihWkpKSwZs0aXnjhBa8HKn6cx8u2P+Ijr167QryW8EpjCotkkWD997q8bqot+fn5MnLkSAHkiiuukKysLFPtSZEUQZBYiZViKTbVFhFJ1zVjgLOWtWXbZtSKoNKCv/nmmyQnJxMWFkabNm0YNmwYDzzwAP3792f58uWV5vXwFLbyyCOPH/nR69cvjxZjgGObLB1DjGke1G+++YZVq1bxxRdfMHPmTKKiovjd737Hjh076NmzJ2+++aZX+4w2utkF7tnABq9fvzxajAGOLX5MEkled9zYGDp0KCtWrKB9+/YAFBQUUFBQQEREBE8//TTffPONKd7VZJLLtiuKs+NNtBgDHFsAqSu50mRLDAoKCsjPzzfbDIdJDw6BtkxCBzEOcGwD/dFEm2yJwd69e802AcBhKuB5zptoiULXjAGOLTDTaU6bbInvYR902RemBmoxBji2h8w+to1G8Qu/lG3blliZiRZjgNMKleZ7D3s4wxmTrfEtvsNI2tOGNiZaotBiDHBsk6JLKS0b5tAo1rO+bPtGbjTREoUWY4DTgx5l2+mkm2iJb5FPftnazmY084mEOlqMAU5nOpNEEgCf8ZlTaPxgZSELySEHgBGMMNkahRZjgGPBwv3cD8A5zvEu75prkA9QSilv8zagymcUo0y2SKHFGASMYUzZEMfbvO3gRQxG5jCHLWwBYChDaU5zky1SaDEGAQ1owAu8AMBZzvIIj5hskXmc5CQTmABABBG8xmsmW2SgxRgkjGc8rWkNwAIWMIMZJlvkfUopZTjDy8Zc/8gfaUELk60y0GIMEmxBqKKIApQ4l7PcZKu8y9M8zUpWAtCJTrzKqyZb5IgWYxCRQkqZ46KQQoYwhM1sNtkq7/AqrzKZyYCaIjiPeUQQYa5R5dBiDDIe47GykI1nOUsaaXzFVyZb5TkE4RVe4WVeBtSE+UUs4hquMdkyZ7QYg5D/5X/L3PnnOMcABjCHOSZb5X4ucIHhDOcv/AVQDpt5zCOVVJMtc40WYxBiwcIsZpXVFgUUMIpRjGRkwEwK2MUuutK1LERlDDEsYhH96W+yZRWjxRikWLDwCq/wDu+Uhbz/hE+4jutYwQqTrbt0CijgDd6gE53YxjYAEklkDWu4jdtMtq5ytBiDnCd5kkwyuZqrAcgmm1u5lbu52+dyUVTFUpbSnvYO0dMHMICtbKUDHcw1rhpoMWroRje2sIWRjCzLW5hBBkkk8TAPs5/9JltYOV/yJTdyI33pyx72ABBHHNOYxhd84RNrFauDTpaqcWANaxjHuLImHkAIIdzCLfyBPzCQgaYFtrLnLGeZxzymMa0sxZ2NIQxhClNoRCNzjLs0MrQYNU4UU8ynfMprvMZeHOPVNKUpQxjCIAZxAzd4NQPwec6zjGV8zuf8h/84xK2xYOEO7uBlXqYTnbxmkxvRYtRUTDHFzGUuU5nKt3zrdDyBBG7hFnrQg1RSy6IKuIsCCviO78gkkzWsIYussr6gjUgiGcQgnuIpfxWhDS1GTfXYwhamM535zK8wrGE96pFMMimkkEQS8cSTSCKNaEQ96hFFFHWpSwQRnOMcRRRx1vrvMIc5yUkOcpCd7GQ728kmm2KKXV6rHe0YyUhGM5oruMKTt+4ttBg1NaOEEjLJZAEL+IqvnJqxniKccDrTmX70YyADyxZMBxBajJracYxjZJLJetazne1sY5tDCEQndgOLgOcqPiWEEJrSlBRS6EAHUkmlK10DPeW5FqPG/ZzgBD/zM8c5zlGOcpKT5JFHAQXsSt/FmqFreEAeIIwwYoihHvVIIIF44kkggZa0DHThuSJDRxTXuJ3G1n+uSCedNaxhJjO9bJXvowf9NRofQYtRo/ERtBg1Gh9Bi1Gj8RG0GDUaH0GLUaPxEbQYNRofQYtRo/ERtBg1Gh9Bi1Gj8RG0GDUaH0GLUaPxEbQYNRofQYtRo/ERtBg1Gh9Bi1Gj8RG0GDUaH0GLUaPxEbQYNRofQYtRo/ERtBg1Gh9Bi1Gj8RG0GDU+ydatW+nbty9xcXHExsbSu3dv1q1bZ7ZZHkWLUeNzbNy4kW7duhEbG8uuXbvYv38/zZs3Jy0tjeXLl5ttnsfQEcU1XiU9PZ2hQ4dS0WNXWlpK+/btycnJYd++fURHRwNQUlJC27ZtuXDhAtnZ2URGRnrTbG+QoWtGjU+RlZXFjh07GDx4cJkQAUJDQxk2bBiHDx9m8eLFJlroObQYNT7FqlWrALj++uudjtn2rVy50qs2eQstRo1PsXv3bgCuuuoqp2MJCQkA7Nmzx6s2eQstRo1PkZubC0Ddus5ZqGJiYgA4ffq0N03yGlqMGr/B5vSxWCwmW+IZtBg1PkVcXBwA58+fdzpm22c7J9DQYtT4FK1btwbgyJEjTseOHj0KQFJSwKUQB7QYNT7GzTffDMDmzZudjtn29erVy6s2eQstRo1P0bNnT5KTk5k/fz75+fll+0tKSpg3bx6JiYn07dvXRAs9hxajxqcICQlh5syZ5OTkMHr0aE6cOMGpU6cYN24c2dnZzJgxg6ioKLPN9AhajBqfo2vXrqxfv54zZ87QqlUrmjZtSnZ2NqtXr+b3v/+92eZ5jDCzDdBoXNGxY0eWLl1qthleRdeMGo2PoMWo0fgIWowajY+gxajR+AhajBqNj6DFqNH4CFqMGo2PoMWo0fgIWowajY+gxajR+AhajBqNj6DFqNH4CFqMGo2PoMWo0fgIegmVxmMcPXqUlJQUioqKHPbXqVOH2NjYss8Wi4Ubb7yRr776ytsm+hRajBqPkZCQwDXXXMOmTZsqzK0BSox9+vTxomW+iW6majzKyJEjCQmp+jEbMmSIF6zxbbQYNR5l6NChlR4PCQmhR48eZaH7gxktRo1HufLKK0lLSyM0NNTlcYvFwogRI7xslW+ixajxOCNGjKiwz2ixWLjrrru8bJFvosWo8Th33XUXYWHOvsKwsDD69OlDgwYNTLDK99Bi1HiGUuA4sA3qZdejb5e+hIU6CrKkpIT7ut4Hm4Fs4KwJdvoQOo245tIpAHYC24DtwD6UAA8BJ4Fi49TP+ZwhDEEwHrdoovmN36hDHePEukAi0Bi4CmgDtANSgKZAYCagAsjQ44ya6nMAWA18A3wL7MVBcJXRl77UoQ7nUZmkwglnEIMchQhwHthtfZUnFiXMm4CeQA/rvgBB14yairkIfAn8GyXAg1WcHwo0AhKAeFQNdyUQBdSB++fez9zv5lJYXAjA0seW0qdZHygEzgFHUDXrEeAYUFVO1FCgE3ALMAhwzjzuT2RoMWocuQAsAeYDS1EiccXVQAdUTdXe+p5EpXO6li9fXhaev379+vz666+Eh4dX/AdngR2oZrCtKfw9Ffctm6JEORjogr81abUYNVaygZnADCDHxfF4VPOwN/A7oFnNL1FcXEyjRo3IycnhkUce4b333qv5l5SgmrDrgK+tL1c1aBLwAPAw4B/OWi3GoEaARcBUYJX1sz3XomqZQShHSkWUAPtRNdcB4Chwwu69ADgDlMLjZx5nSukUsqKySI1OVU3YaKA+0ATluEmwbicBbYGGlVy72Gr758AXwK/ljscA9wFPAK0r+R7z0WIMWv4D/BnYWm5/IvAH4B7gGhd/JygP6hpU7bQd2IUSXDVYz3ru4R4OcICQ6o6sXY4S5XVAKtAd1TctTwmQCcwBPgPy7Y6FAvei7tnVfZmPFmPQsQJ4EeUNtWEBegGPAv1RD649J1FOnCXAWuBUNa9lq+3qAPXU90qsMP3QdMY0HqMEfBElmt+s18mv8NscaQWkAQNQDpzIcsd/Az4C3kfV2jbCgJHAK6gfHt9BizFoOA48CaTb7bOgmqAvoxww9hwDPgUWAhtQg/iuSMQYC2yDago2QfUxo13/iYhgsVTgXcmx2noY5bzZhVH7VuS4iQVus97LnTgKsxTljPoLqka3EWPd9zi+spBQizHgEeAT4I84OmZ6A/+DavrZKEF5UD+0vpcfQwxB9SNTUc6cm1Ci8walKHFmoZrHWag+aXkuB0agHDfJ5f7+c+Al4Ce7/e2BD4Cu7je5hmgxBjTHgeGoMUIbbVFNt5vs9l1ECfCvqNkz9kShmrADUE1YV301s9iCckAtwrnvC6r5+gLKfhvFwBRU39E2bBOKarq/hHMT3XtoMQYs36CEeNz6ORr4E/A8RjPuPPAe8DdUf82eG1G1yxBUk87X2QfMsr6OlTt2IzAR6Gu37xgwAdVqsJEG/Avv1faOaDEGHAK8an3Z+nmdgblAC7tzPgOeRc12sVEHGA2MxbkP6S8Uo5rYU1BjkPb0Av6BY/M1A/Wjc8b6uTGqX53qWTNdoMUYUJSgPKLT7fb9AfVgRlg/bwfGo4YAbMRa/+4pKh/T8zc2Aq8DizHGUMNR9/8KysMLaprfUOv5oFoO/0SNsXqPDL2EKlDIRz08NiHGAAtQzokI1MP4N9T8TZsQw1CD4QeAtwgsIYKaEvdvYBNGH7kIeAc1le+/1n1Xo8pkjPVzATAMNTTiRbQYA4EC4A7UMATAFcBKwLaA/ihwK/AMxuD8zSgHyGT8ZbrYpXMdyvv6T9SwC6ixxx6o4Y1iVG34PjDJerwYeAg1O8lL6GaqvyPAKAxHRBPUSosU6+dNQD/UtDRQ6wXfQfWTgpGzqGaqvePmNlQ/0bYc62NU+RSjqqvP8EaTVTdT/Z5nMB6sBGA9hhC/RLn3bUK8HrWqPliFCKqfOAfluLG1CL4EumEM69yPaqJaUE6wEajpfx5Gi9GfmQH83bpdH+VFvNr6eRaq6Zpn/TwKNVjeypsG+jCDUQ6bJOvn7ag5r/usn0cAb1q381Eze8qPwboZLUZ/ZS/K+wnKQ5iBmk0Cqu/4B5R31YKa7jYLw6OqUVyDcuL0tH4+gupL20T3HGq6HKjZS8NRZeohtBj9kWLUsiDbDJK3UWsMAb5CuemLUUJ8D+XG96+Ftt6jAarMbNkFDlu3bZPh/4aaNABqkvxfPWeKFqM/8ibGmFgvjF/vw6hlQoXWz2+hBvA1lROJmrfaw/p5J6qZKqjhn08wnDt/RjVpPYAWo79xElUTAlyG8vyFoMbPhmBMBn8WNf1NUz2iUWOStplHyzBqwRbAu9btQtSUQg+gxehvvIrRPP0zKpwhqH6hrbbsgeF80FSf+qi+t20u7kSU9xnUNMFu1u3FqCh5bkaPM/oTP6PWDBaigi/tRjWx9qFWYxSg+kBbgP9njokBwTzUDBxQ/cV1qD53JmoyOagZPe4d7tDjjH7FBxj9wb9grL54GmNmzd/QQqwt96AmAoDytmZYt3tiOHrWon703IgWo79QglreA2oOqe2XeyNqPR+oaV8jvWxXoPI3jAgAL2CsgHnS7pzZ7r2kFqO/8BXGcqcRqLFFUPMpbbyNR/5HO3TogMViqfbrtddeIyYmxmn/X//qPC5w5MgRl9+xcOFCh/NefPFFp3N273YVdtxNJKP6iaC6ASus270xWh7/pNqBuKqFaPyD+0UE6+tH675cEalr3ddKREo9c+lrr71WMjIyHPaNGTNGAFm2bJnD/qFDh8qkSZNERGTLli0CyIABA6q8xty5cwWQ5557rtLzevbsKTNmzKjZDVwqP4hR5oPs9k+027/UbVdL1zWjv7DO+t4MY+7pArCmrlDzTfXAvntpj1qYDWrYw7YAub/dOevddznfiIulqZzfUNPfwHCvg1oWZKPybN21YuvWrdU+d968eZ4zxAyGAt+hxnHXoxw4HVHjkhcx1kS6AV0z+gP/xVip3sVuv8213gxjvFHjXuzDb9haJ+EYUfW+xW3zVbUY/YGf7bY7WN9PYaww6O5Va4KLjqg1oGBMqgDj/yEP+MU9l9Ji9AfsI3jbQmOcsNvnm+HqA4NwjGVp9mV+pd22q0RBl4AWoz9gL0bbgtjf7PZd4UVbghFb+VZU5tVNd1AFWoz+wBm77Tjre67dvsu8ZonHCA1V0YNLSirvgJWUlJSd6zVsP4D2NaB9mVeV1LWaaDH6A/aZti9Y3+va7TuP3xMTo2Znnz1bUUINRW5uLvXq1av0HLdjm5gf42IfOP5f1AItRn/gcrvtUy722Tef/JSkJBX/YseOHRWeU1BQwN69e2nZsqW3zFLYcj5W1DS1/7+oBVqM/oB9KEXbQ2D/YBzHbwkLC2P37t20aNGC1q1bs2HDBrKzs12em56ezpVXXkm7dl4Od24rXy1GjUPuB9tzehWGSDd41xxP8c477xASEkKfPn1YsGABOTk5lJSUcOzYMd577z3Gjx/P3//+d0JCvPjY7sMYukix22/7fwjD0bNaG9w2s07jOX4SYy7keLv9d1j3hYnIWe+YMmvWLEFNQXB45eXlOZxXt25dl+e5eu3atavs7zZv3iz33XefNG3aVCIjIyUiIkKuuuoqGTJkiKxbt847N2nPLDHK/hO7/Y2s+zq67UrpenGxPyCo8cXfgE6owMSg8itOsG7PRyUL1biXuzHWM+5HLereC9i6rY8C/+eWK+nFxX6BBSOZ548YDpu7MCaHf+hto4KA31ATxEEliW1q3V5pd44bk6xqMfoLt1vfizAWGSdhxPxcjkpgo3EfszDWK9pH2bMtKg7DCJHpBrQY/YV7USsFwHGFuS1UfykqKJXGPZzFiA4Xgyp/gD0YDrO+qHyObkKL0V+oj7GO7nvUagFQfZq21u1PUMt9NLXndQwv6pMYuRzfx1hBc797L6kdOP7E1xjNot4YoSAWozJNgerDrEGvVK0NO1BJgvJRNd8eVBDjI6iuwUVUtq8DGOFPao924PgVvVFZpUAJ05Ym+w4MkW4AXvSyXYHEeVRrI9/6+XWMaOKvoIQIKmat+4QI6JrR/9iAWu0vqDV136IeisOotXenUJ2PxRhhBTXV536MPnl/VBIhC/ADqrYsRg1r7MDdYtQ1o9/RFZWeDGArKsI4QCIq76Atp+AQjJXpmurxZwwhJmLkaMxHJRoqth57DbfXiqAdOP7JFIz5kG+gIl2DGv6wTQI4j/pl91CSloDj7xgpxOugMhnbyngCRjn2Qf3QeQAtRn8kAZhm3S5Fxfe0ef5eR+WiB7X+rheO4SI0zryJygANRq5L22D+QuAf1u1GqJrTQ1H4tBj9lSEY0cP3o8a8zqEelA9QIepBibQnKn+ExpFi1HS2F1B98BBUU982weI7VPNUUOX6Ie6bFO4CLUZ/ZhpGIs9NGElSbQ/VA9ZjBaisu69g9HuCnWPArRgtjGhU09T2I7YP5aW2Ldz+i/WzB9Fi9GfqoJpRLayfl6JmihSgmlszgcmo/+VS1APVHSMGa7CyEBWg+Bvr58tR0wltE+13o4aQbE3/h4GXPG+WFqO/0xCVh6OR9XMGyslgi17xBKqJWt/6+VvUyo//I/hqyZOooYu7MBYHX49qjt5k/bwJld/ykPXz7ahU7F5AizEQaIGaANDE+vkblOPGFlpwCGq1hy1N9llgPGqx7FfeM9M0ilCZh1thDF1YUOnX16GCQIPKVnwzRpiNAailaV6azaTFGCi0Q4Wfb2X9vAnVFFtu/fz/gFUob6stwNVuVB7CfgSmx7UItfKiLWp+qS3K3jWoZVDvAhEoB83/oMrBFmhqJEqI0XgNLcZA4mpU/g1bspZfUc2sV1F9xlCU53AXjmNli1Gu/N+hBOvvc7LOo5qWLVFOLFuIjBjUMMZ2VA0IqvXQGzWWWIKqMZ8HPsbr83v1dLhApAB4Cse+TnfUigP7WE7foPLWl0/ekoQaqxyFEcHcH9iEGn6Yi9FnBlX7jUQ5sGxNeUHlV3wGw1FTHzXrZqA3jHUiQ4sxkPkCVTPkWj+HocbVXscxBuhK1BSv1eX+PgLl/r8TNZvHg2Nsl8wPqMzNn6P6xfZEAQ8Cf8IxtfpeVDmssNvXETW0YV6qBC3GgGcf8AdU89NGU1QtMRzVdLWxETVhIB3nwMihqAnqt6IyM92AV/tTZZwA1qKa44tREx7K0wz1I/QQjot/f0X1DadirOAPB55GjcFGesTi6qLFGDRkAI+h3Ps2mqH6Sg/iKMqzqNAes1FDIaU4E4EaFrge5SBpi0q97a5UAwIcBHaiVkhsRzmoKhojjUENyj+A8iTbe0NyUFPa3sGx+XoTatDfy2FYK0CLMag4jVrrOAPlabSRjBrquA9j7Z6N46igTAtRfcyqctg3RMV5TUCNfV6FCn9fFyVg2/t5oBDlvSxC/Uj8Ahy1bh+i6rQFDVHN5wEoJ0xUuePZKLF9hGO+kiaoSeGj8aVsz1qMQcl+lIf1nzgO/NcDRqCCL7mqLS6gQn6sQzUV1+G2pC/VIh5Vm3W3vnfEeTygGNV8nYbqE9o/3Q1RLYGxmNPErhwtxqDmJ5QzJx3nGi8FGGx9JVfw96UoYe/EaE7+jKpNj2Gslq8Jl6FqriaoMdO2QBvUj0NFYfSLUH3i+SinVfkUbU1QA/zjcVuSGg+gxahBNQ9nopw3B10cT0aNV/ZEOW/quzjHFadRwryI0Sy1vcegnCe29waomq98U7Mifkat4/wGWIJzwlILkIbymt6JP8QE0mLU2FGCmmw+F9XUy3NxTiiqeZiKCvvRDlV7edITeQo1hLEdNZa4GhVmxBXNUbX5/aga1X/QYtRUQD7wJarptwTH5KzlCUPNdmmNCldha2ZehRqbrIPqo0VZtyMxHDdnUT8Ctlr0mPV1BCW4bVSdZaslasXFYNQkeP9Ei1FTDUpQ81jXoSakr8RteewvCZsjpzdqCl+zyk/3E7QYNZdACSqW6LZyr4PWY+6iDqq2TUE1h9tbt+Mr+yO/RYtR40ZKUGOER1BNy8Oo5u05lLf2AsqZk48aRgkF4lDDE3GoccnGqOZtExyTxAY+Gb7vY9L4D6EY/UVNjdFLqDQaH0GLUaPxEcIw8rJqNBrz2PD/AbetQncRG8WFAAAAAElFTkSuQmCC", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from PIL import Image\n", + "file = Image.open(circuit.draw())\n", + "file.show()\n", + "file.close()" + ] + }, + { + "cell_type": "markdown", + "id": "46753da7", + "metadata": {}, + "source": [ + "### Finally, let's make homomorphic inference" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "c0b246f7", + "metadata": {}, + "outputs": [], + "source": [ + "homomorphic_predictions = []\n", + "for x_i in (int(x_i[0]) for x_i in x_q):\n", + " inference = QuantizedArray(circuit.run(x_i), y_q.parameters)\n", + " homomorphic_predictions.append(inference.dequantize())\n", + "homomorphic_predictions = np.array(homomorphic_predictions, dtype=np.float32)" + ] + }, + { + "cell_type": "markdown", + "id": "68f67b3f", + "metadata": {}, + "source": [ + "### And visualize it" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "92c7f2f5", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnZElEQVR4nO3deXxU1d3H8c9J2GRL2IyssmVhUVGoS91QniqghfKAW62ipaKiGIMCIgJBFAUrMVZBqWhBRUQEccNSQVBbpICPhSIiIBBIIHsYshGSnOePe5EkBgiQcCeZ7/v1mtfcOffO5Me8hm9Ozj1zrrHWIiIiNUuQ1wWIiEjlU7iLiNRACncRkRpI4S4iUgMp3EVEaqBaXhcA0Lx5c9u+fXuvyxARqVY2bNiQZq1tUd4+vwj39u3bs379eq/LEBGpVowxu4+1T8MyIiI1kMJdRKQGUriLiNRACncRkRpI4S4iUgMp3EVEaiCFu4hIDaRwFxHxQJYvn1Z39eKj1Wur5PUV7iIiZ9iqLwtoMTyKfR02MPmdF6vkZ/jFN1RFRAJBdjaMeayAWUld4ILd9My8jvWvvF0lP0vhLiJShbLzsnn8zcf5fnsWa76B3LBVcMEeehf34YsX/l5lP1fhLiJSRXLzc+k8LoLkJvugAdDHab+Ga1g5+fMq/dkKdxGRKpBfkE/bmAgyztkHfx/A3RdPYNgwaB7aiMi2kVX+8xXuIiKVbOfufLpNiCCvUyKNvxnA6plL6dHjzNagcBcRqSTWwmtzCrj371HY7nuITOjPfz9aSi0PklbhLiJSCXbtgj/dU8CKRs5MmMvzr+PrOZ94Vo/CXUTkFBUXF7Ns3d9ZsrSAt94qpqDXaLjgJ641fVjxTNXNhKkIhbuIyCnIzc+lw5gIUpolQl1gmNPem96smFi1M2EqQuEuInKSDubk0yY6El/bRIL/eQX/c14vunQxtG9xLtEDo70uD1C4i4iclH+uyeeav0RwOHIvrTYOZP3cxbRs6X8ruSjcRUQqIC8PJkwq4PkdUXD+Hnqm38D69z/wuqxj8r9fNyIifubLL+G8Cwp4fnsUnL+bPrYv61/82NlprbfFHYPCXUTkGHw+GDECru5dwO4LusIFO/nNj235fNKnzgHWQkwMxMZ6Wmd5FO4iImXk5ufSckQnQp41zAo1MK4uhd13cO22Niyfv8cJ9CPBHh8PWVl+14PXmLuISAkJiblEjY8gr0MitTd3omPLEBo0gJ7NezJ74qvQ3A30+HjnCdHREBcHxnhbeBnG+sFvm169etn169d7XYaIBDBr4a35+dz1YQTFXfcQuXMg/3n1A+rWLefAoBKDHsXFngW7MWaDtbZXefs0LCMiAS8pCQYMLODOJVEUd93DVfk38MPfjhHsMTGl244M0fgZhbuIBCxrYc4ciOpawMe1o+C83Vwf3JfVz3xc/sFHxtijo50ee3S089gPA15j7iISkH76Ce65B1Z+UUi9W7pC1E5+E/QbPntiWflPMAZCQ0uPscfFOftCQzXmXh6NuYvIabG2dLiWfVzCwZxcukZfxd5DuwCoE5JHQYtcrjXXsmLiikr9WVXttMfcjTG7jDGbjDHfGWPWu21NjTH/MMZsc++buO3GGPOiMWa7MWajMeaiyvuniIiUERtbeljkOHPP132bS7N7I9jbdgPBDfOo3zSf2rUNA+oOqFiwwy+D3M967EeczLDMNdbatBKPHwNWWGufNcY85j4eC/QDwt3bJcAs915EpHJZ68wxPzItMS6u9Li426suKIApT+fz1PdR0D2RnqkDWTfzA3/N5UpxOmPuA4He7vZcYBVOuA8E5llnvOcbY0yoMaaltXbf6RQqIvILJce9y8w99z01mQGTr2FX+n727YeChvuh+wF+Y29g+UsfeFbymVLR2TIWWG6M2WCMGe62hZUI7P1AmLvdGthT4rl73bZSjDHDjTHrjTHrU1NTT6F0ERFKB7wre+oUwp+IZLVZze662yhot42gZtkMrDuQ5bHlzISpgSrac7/CWptojDkb+Icx5oeSO6211hhzUmdmrbWzgdngnFA9meeKiPyszNzz7GA4d0QbMjr44ONbubfnO0ybBiEhHtbogQr13K21ie59CrAEuBhINsa0BHDvU9zDE4G2JZ7exm0TEalcZeae79uXTditzcjo4KPB8gF8MX0+r7wSeMEOFQh3Y0wDY0yjI9vAdcB/gQ+Boe5hQ4Gl7vaHwJ3urJlLgQMabxeRKlFi7vmiq56mTUwkueHphK+5gpTeF9P7mhp8xvQEKjIsEwYsMc5p5VrAfGvtZ8aYdcBCY8wwYDdws3v8p0B/YDuQC9xd6VWLiLhSH4jlgZF5vPdOJHRP5Oq837Fq2WK/naJ4ppww3K21PwEXlNOeDvQpp90CD1RKdSIi5fDl+Jjw9gS++97H2rVwqN1y6J5E/1o38smzS7wuzy9o+QERqVZ8OT46PR5OWtMUaAL0ddr71urLJ+M/8rQ2f6JwF5Fqw5eTTdtRkfhapRD098GM/M0jDBkCzUJC6dKui9fl+RWFu4hUCxs35/KraREUdNrP2etuYs1bC+nY0euq/JfCXUT8WmEhTP9zLuP/LwK67qNn8mDWfbQw0M+XnpDWcxcRv7VpE1xyWT7jN0RB10SuswNZP3ORgr0C1HMXEb9SXFzMZ/9ewZtvH2bhQuDqEdBtDzfUvoGPH//A6/KqDYW7iPgNX46PDmMjyGiRDM2BEU5731p9+fjxwFgTprIo3EXELySnZdNxbCS57ZKps+Zqrr/ofNq3h/Bzwhk5YKTX5VU7CncR8dwny3IZ+FYERRH76bT1Jr5duJDGjb2uqnpTuIuIZ7Ky4OFRuczNjoBu+7g6ZzCr5i/0uqwaQbNlRMQTH3wAUV3zmXswCrol8tvav2PV9EVel1VjKNxF5IxKToabb4ZBg/PJ6B0J3fdwY50b+fBxrQlTmTQsIyJnxIFsH50eOZ/0JgnQERhrOVwX+tXqx0fjtCZMZVO4i0iV+35rNhc+E0lBh/3U3dKJDi0bUrce/Drs18y8b6bX5dVICncRqTLFxfDiS7nErImAqP1cmHQL6+cvIEgDwlVO4S4ilSrDl8GvnvwVewuTOHwYbO1CiCqkrx3MslcXeF1ewFC4i0ilycrOImJiBOmh6bDtbExxMCEhMKTZDbz24F+9Li+gKNxFpFL4cnx0HBdBZvN0WHoXg9q/wcsvQ8uWXlcWmBTuInLaUtJ9dHgsnNw2qdRdfidvj3+DwYO9riqwKdxF5LR8/kU2fV+PpKhzCh03/551H82laVOvqxKFu4ickuxsGP1YLq+kRUCX/VydfQurFr7tdVniUriLSIVl+DIYNGMQO1NT2ZcEhaFJ0OUAA+sM5oPnNBPGnyjcRaRCsrKz6Dwhgswm6VA/CDo765f8b/0hvDf6Pa/LkzIU7iJyQr4cH+eOicAX5syEebzfG0yYAPXqeV2ZHIvCXUSO68effJw/JZxD7VNp8tWdrHz9DXr08LoqORGFu4iUZi0Yg7Xw6mvZjFgVgY1I4YI9t7Fu2Vxq1/a6QKkIhbuIHBUbC1lZ7IqO44/D8/iiWQR0Sabf1gv5dP58r6uTk6BwFxEAsg5mMmnbh6zbfIh/fzmMoqhlELmfwZ/Cosuu+rlHL9WDwl1EyMrOosPj4WRFpEMEwPdg4XefwaLLoiEuTsFezSjcRQJcamYW7R8LJ7dlOrWX38bo393LdVN70/wwdMsB/qVgr460qrJIAFv9Tx8toyPJbZXGuRuHsuf9t3k6cQlXZ7nBDhAT4wzJSLWicBcJQHl5MGp0Nr1fjqCoUwpX+25n1/tvEPZsDMTHQ3S0c6WN6GjnsQK+2tGwjEiA+eoruHtYLjsucmbCDKpzC4uff8vZGRrqBPqRMfa4uKPtGpqpVoz1g9/GvXr1suvXr/e6DJEaq7CokGXffMnsvxbx8cfFBF83jKLIRIaUt3RA2VkxmiXjt4wxG6y1vcrbp567SHVwGoGblZ3FuWPD8Z2dBh2AkVAEDDprUPlrwpR9XQV7tVThcDfGBAPrgURr7Y3GmA7AAqAZsAG4w1pbYIypC8wDegLpwC3W2l2VXrlIoHC/WPTzUIm1zhh4aKiz7zh+SvDRdXIEh9qlUX/tNfS9JIqwMOjauisP/vbBM1C8eOVkeu7RwBagsft4GhBnrV1gjHkFGAbMcu8zrbWdjTG3usfdUok1iwQOa51gj493HsfFOcF+5KRnmR58cXExvlwf1sK7i3IZseJCbHgq5++6k38vmUvdut78M+TMq1C4G2PaADcATwOjjDEGuBb4vXvIXCAWJ9wHutsAi4CXjDHG+sPgvkh1U/KkZnz80ZCP/uUXi1IyU+g6uSvpTdKPPj8c+hX9nk/fmHsGixZ/UNGpkC8AY4Bi93EzIMtaW+g+3gu0drdbA3sA3P0H3ONLMcYMN8asN8asT01NPbXqRQJByYA/okywpx1II3JyJOkh6QSt+RVBK68nIuF6JodP4dMndXWkQHTCnrsx5kYgxVq7wRjTu7J+sLV2NjAbnNkylfW6IjXOkTH2kmJifg74DF8GnSdEcqBpFiy5jyubzOK1OdC5syfVip+oSM/9cmCAMWYXzgnUa4F4INQYc+SXQxsg0d1OBNoCuPtDcE6sisjJOhLsx/hiUXpWJm3HRHCgaQa1P7mHVx+YxcqVCnapQM/dWjsOGAfg9twftdbebox5DxiCE/hDgaXuUz50H69x96/UeLvIKTLmmF8sWpPbmqsejqCwfTptv72bfy2ZTZs23pYr/uN05rmPBRYYY54C/g+Y47bPAd40xmwHMoBbT69EkQAXG1tqVkzBYUNso8k8kxoOEWlcmXknq5e+runoUspJhbu1dhWwyt3+Cbi4nGPygZsqoTYRcaVkpXLJU5ewrzCZgsNg6xyGiEKG1L2d917QTBj5JX1DVcTPpR1II2JyJAdCsmD72QSZIEIaw21hg5h530yvyxM/pXAX8WMZvgw6jo/kYPMsWHw/wy+byfTpEBLidWXi7xTuIn5qd2IWkbERHGqdQaOV9/DhizPp3dvrqqS6ULiL+KF33svi9o/CsR3T6b7jbtZ+Npv69b2uSqoThbuIH0lNhftH+ng/OAIi0rihcCgfv/m612VJNaRwF/FYSmYKQ164iZ3JGexLgqLmCXCuj9sa3s78R/7mdXlSTSncRTyUdiCN8NhIfE2yoLGBxhCE4baQ23nr4be8Lk+qMYW7iEfSsjI4d1wkuWFZBH94H8/dMYuHHoLgYK8rk5pA4S7igQ0bM7n0hQgK22XQ8t/D+XrRLDp29LoqqUkU7iJnUGEhTJ2exaQtEdApncvT/8hXn7yqpQOk0lV0PXcROU2bNsGvLvMxaXMEdE7j5np38fVf5ijYpUqo5y5ShTJ8GUx+5yn+uTaPb78FznsfOqfy+8a383bMG16XJzWYwl2kiqQdSKPjE+HO0gHn4tws3NrwVt6O0UwYqVoKd5EqsGd/BuETIzjUMov6nw9l/B/u4JKLISw0jO4duntdngQAhbtIJfDl+Ji3ch5FxUX88INl9pYpFLfPpOuP97Lmk1do3NjrCiXQKNxFTlNSehJRU6I42OTg0cb20L9wGJ/Mf8WzuiSwKdxFTsP+jP10mdKFgyEHqff5bRxK6sU1veHBu6IYdGV/r8uTAKZwFzlFKZkpRE6Owhfqg0UxRNaewZyF0LOn15WJaJ67yClJzUqjw/hIfKEHMEse4unbZ7BunYJd/Id67iInaeP3GfSaEcHh1lmErbmfVQvjiYryuiqR0hTuIidQXFxMfkE+xcXwl1k+Hv9PN+iYya9T7uWrz2YSpL9/xQ8p3EWOIyk9ie5TupPZJPNoY0e4ue4w3p2pmTDivxTuIsdwZCaML8SH+dfFBB9uTEQk3N77Wh6/ZRxYS6mFYco+FvGQwl2kHCmZKYRPiiK7qTMTZlDUDF5+Gc45xz0gNhaysiAuzgl0ayEmBkJDnX0iHtNooUgZe5PTaDcukuymBzhr2UMsenIG779fItitdYI9Pt4J9CPBHh/vtFvrYfUiDvXcRUr4ZHkaAxZEUNw2i8jv7+dfn8XTtGmZg4xxeuzgBHp8vLMdHX20Jy/iMWP9oJfRq1cvu379eq/LkACWnQ0xozN4LSccOmbQv+BePpl6ghOm1lJqqkxxsYJdzihjzAZrba/y9qnnLgErKT2Jy6ZeRvLhNAoKwNYrgLBC7gwZxtyYCgR7TEzptpgY9dzFb2jMXQLS/oz9RD4ZRULDBA5lNCAotyFNCpvyUOuHmBvz2vGfXHKMPTra6bFHR5cegxfxmHruEnBSMlPoNCGK3OYHMe/HMG7gDCZMgHr1KvgCxjizYkqOsR8Zgw8NVc9d/ILG3CWg/HdrGhf9OZzDrbJo8eVDLI+Lp0ePU3wxzXMXj2nMXQKetfCXV9J4eG0E9twsLkm6n6+Wx1O79mm8aNkgV7CLH9GYu9R4u3bBtddnEL0mEts+k1vq3ss3f515esEu4ufUc5caKSk9iVtfvI0diVns2we23U5oc5C7mw3j9ZFaE0ZqvhOGuzGmHvAlUNc9fpG1dpIxpgOwAGgGbADusNYWGGPqAvOAnkA6cIu1dlcV1S/yC/sz9hM5uYuzdEBzA80hqNgwtNndvD7yBDNhRGqIigzLHAKutdZeAPQA+hpjLgWmAXHW2s5AJjDMPX4YkOm2x7nHiZwRiakptB8fRXYTH/U+imFet2KKnymmaHoRr4983evyRM6YE/bcrTOdJtt9WNu9WeBa4Pdu+1wgFpgFDHS3ARYBLxljjPWHaTlS/ZWZkZKQvJtH5j1K7uFcDhyANelfU9zaR+eND/H1pzMIC/OwVhEPVWjM3RgTjDP00hl4GdgBZFlrC91D9gKt3e3WwB4Aa22hMeYAztBNWpnXHA4MB2jXrt3p/SskMJRZiXH3/l10ezKcnDD3Y1gfqA398kfw6ZJ4DwsV8V6Fwt1aWwT0MMaEAkuA076omLV2NjAbnHnup/t6UsOVXIkR2Pv4I3R7MoKc5oU0WTySzC1PcNvvYfoz9WnTsqG3tYr4gZOaLWOtzTLGfAFcBoQaY2q5vfc2QKJ7WCLQFthrjKkFhOCcWBU5dSW+BZo0K54u2fHktAIWjiO04GneW2bo08fbEkX8yQlPqBpjWrg9dowxZwG/AbYAXwBD3MOGAkvd7Q/dx7j7V2q8XSqFMSQ9MZrOfwgmuzWw6FEe7vs0mzYp2EXKqshsmZbAF8aYjcA64B/W2o+BscAoY8x2nDH1Oe7xc4Bmbvso4LHKL1sC0ffb9tH+sUjy2hTRdNFQ1mz5ijhiaFBffQeRsioyW2YjcGE57T8BF5fTng/cVCnVSUDLys7indXvUFxs2bChiL/tfgLbLoeLvx7IlxveoO5jMUcvlKGldkVK0TdUxS/tTt5Nt2e7kROa4zQYoB3c+t/LeOfzJVqJUeQEFO7id/am7nWCvVEOtT+7neL087n+erj3tu4MiO13NMiPBLyCXeQXFO7iV5LSk4h6qgs5ITmwcBy/Pmcqr30KnTsf4wkKdpFyaVVI8Rt7UpLoOCGKnNBsai8dzaujprJy5XGCXUSOST13OTNOcGGL1Wv20+f1LhS1OkiHDaP48pPptGnjQZ0iNYR67lL1YmNLX1vUWoofjqZw0kRy8woZPX4/vWdHUdTKx/XZD7Hjo+cV7CKnST13qVpllg0gLo6EB4ZxXv4b+JoC06dAHaAd3B06gtcna00YkcqgcJeqVXLKYnw8e1+Jp+sdkNMG+Ppi6pgGREbCHb37MXrwaE9LFalJdIFsOTOsJaleEJ3vCCavVREsHMe9vacybRqEhHhdnEj1pAtki7es5Yd7HuT8P9TjcKt8Qt67hyXdWnHNLKupjCJVRCdUpWpZy5u//TNdi9/kcJt8eu4aRdK1IVyzeGTpk6wiUqnUc5cqk5oK9z6YypIWT0Pbg9xaO5p35j3vBHrtw1o2QKQKKdylUu1O3s2V064k5XAGhw4BLQ9BSCH3hz3AzBEvOAdp2QCRKqdwl0qTkJJA16ndyA3JgZ3NqBVsaFS3Pn9qfxfT755e+mAFu0iVUrhLpUhI3kvEk9041CyHWovHMX3YVB56CIKDva5MJDAp3OWUbNu7jQEvDuBg4UEKCyHFpmGbH6Ld2tF88eFUOnb0ukKRwKZwl5O2I2kHFzx/AXkN8wg+VJeiIqAoiL6+MXy6bJpGXET8gMJdTsrOfTs577nzyGuYR6svp5D05RMMGACzZkGrVl5XJyJHKNylwhJSEug+vTt5DfMwC2M5nPkECxbAzTfr/KiIv9GXmKRC9qbuJeqpruQ2yoWFT/D7X03i++/hllsU7CL+SD13OaHtCUl0faYLh1vk0GjZOBbETaF/f6+rEpHjUbjLL+zct5Oxb4/lUOEhUlPhG99KbKtsemwbzeoVU2nc2OsKReREFO5Syo6kHc4J09A8p6ExcBbcHDSKd+dPP+5zRcR/KNzlZyVnwjRcOomcLX/i3nth4viGtGwR6nV5InISFO4COGvCdJvWnbxGefBuLJ3qT2LO19Czp9eVicip0GyZmqrsUrrHWVp3d3ICkVO6kdc4l6BFE3hq6CTWrVOwi1Rn6rnXRLGxznVLj6y8aK2zdnpoqLOvhLXf7eXyWd0oCsuh1T/H8fniJ+nSxYOaRaRSKdxrmnIuSE1MjPM4Opqsg5ks+uf7FBUVs2q1ZUHGaGiVTZ/MMfx9+VQt9CVSQyjca5oyF6T+OeSjo9kx+kHOm9j66EyYBsBZcE/oo8x+cpon5YpI1dAFsmsqayHo6CmVnYk76Pacs3RA0GdDqZ0Xzg394Y+DL+SGS/SNJJHqSBfIDjRHxthdu+tB16eiyG92GBZMZuB5E3n5ZWjZ0sMaRaRKabZMTXMk2N0x9m27dhF+Zz3ymx+mwZIxLHpmAosXK9hFajqFe01jjDMrJjqaJb95hMinu3M4LJ/uK4aScENTBg/RKl8igUDDMjVQ9qOxPPRoIm8s7gKts7mJMSz8+lkt3ygSQBTuNcSOpB30mN6D7CbZTkNLoBhGthzFi/dpJoxIoDlhuBtj2gLzgDDAArOttfHGmKbAu0B7YBdws7U20xhjgHigP5AL3GWt/bZqyhdw1oTp/tx55DfKg68v5qw69YiMgDuuGsioQaO8Lk9EPFCRnnsh8Ii19ltjTCNggzHmH8BdwApr7bPGmMeAx4CxQD8g3L1dAsxy76UK7E7eTdTU7hSE5mEWTuaxwROZOBHq1fO6MhHx0gnD3Vq7D9jnbh80xmwBWgMDgd7uYXOBVTjhPhCYZ50J9N8YY0KNMS3d15HTtHnXZq544QoO1j6IBYqDi6GJJWzVBD57dyI9enhdoYj4g5MaczfGtAcuBNYCYSUCez/OsA04wb+nxNP2um2lwt0YMxwYDtCuXbuTrTsgbUnYQs8Xe3Ko4SHOTu1MWoqBYkO/tn9i6YrR1K7tdYUi4i8qHO7GmIbA+8DD1lqfKTHzwlprjTEn9VVXa+1sYDY431A9mecGoq17tnLRCxdxqP4hwtf9mW3LH+GKK+C11yAy0uvqRMTfVGieuzGmNk6wv22tXew2JxtjWrr7WwIpbnsi0LbE09u4bXKKtu3dRo8ZPcivn0/tRdPY969HeOklWL1awS4i5TthuLuzX+YAW6y1M0rs+hAY6m4PBZaWaL/TOC4FDmi8/dTt3LeT8/58AfkN82HBU/TpMIbNm+GBB0otHSMiUkpFhmUuB+4ANhljvnPbHgeeBRYaY4YBu4Gb3X2f4kyD3I4zFfLuyiy4xrK29JeMrGV7YgJdn+3O4SZ51PtgCrMnjecPf9B3kUTkxCoyW+Zr4Fhx0qec4y3wwGnWFVjci2tsfvhPDJl5EzmFORSk+0hplIdtWkDXjZP4YsUTnH2214WKSHWhb6h6zb24xpa/xtOz+CUOhRQRlFeL4kZBUBTETcUTWPhBrNdVikg1o3D3mjFsHXUfFxW9xKHGRTR/ewxpO6fxp25reO6rSwltojEYETl5CnePJaQk0CPuQvJDiuCdaTTaOYQF9KHPps81uC4ip0zzLTx2Wez/kN84HxZM4eGfarOJ8+jDSmdNdj+4SpaIVE8Kd4+kp0OPIU+TFLaNemsvYs2Oj4mL3k2D4myIjnYutqGAF5FTpGGZM8xaWLQI7oveS8atkwjOqMOuHtcTFpXrXNi65AWuQ0M1NCMip0ThfgYlJcGIEbB0KdQbcj00LmLmJTMJ6ze89Dz3IwGvYBeRU6RwPwOshWujR7DK/BU6WMwoyG9cRM9DPRneb7hzUNkgV7CLyGlQuFexn36Cqx4YQeIlswhKrk+bOq2oEwzNC5vz0diPvC5PRGoohXsVKSqCv/wFHp07kqIBs6ifEsrOqds4u0lzr0sTkQCgcK8CmzfDsGGwNjsGBr9Eo6wQtj+9VcEuImeMpkJWooICePJJuPBC+M/hR2DwCzQ+0JgfJ/3A2U20MIyInDnquVeSdeuc3vqmTRA1aCw/dJ9BowON2DJhC+c0Pcfr8kQkwCjcT1NuLlxz/zj+nTOf4C7Q7PJifmixl4a+hnw//ntaNWvldYkiEoAU7qdh1SoYOOF+fH1egZwggoqD8QHnHDyHtY+vpU2LNl6XKCIBSuF+Cg4cgLFj4dV/PQCDXqFhWig7n95G8xCdMBUR/6BwPwnj3xzPwm+Ws3s3HCYPBm0m5EAIP07ZqmAXEb+icK+gW6YNZWH+PGgKhDptLXwt2Dhxo2bCiIjfUbifgLVwTcwfWR06D7Y3Z2z7bTw5KZQ6dbyuTETk2BTux7F3L1w+YjgJF71Brd3N+DJ6G5f9KtTrskRETkhfYipHcTG8+ip0/N39JFz0V+rvb0rSjB8V7CJSbajnXsb27XDPPbAq3ZkJ0zijCTumbaV5SFOvSxMRqTCFu2vu8rf425JNfPU1mCY/wKAPCT0QytbJP2gmjIhUOwp3YEDsUD4y8+AcYIjT1jizMVsmbtFMGBGplgI63A8dgkvu+yP/OXceZkdzRnZ5iV//2hAcFMSNl9xIvTr1vC5RROSUBFy478jYQWZ+Jps2Qcyf4zlw+VvU2duM76dso9O5oaUvdyciUk0FVLi/9O+XGLls5NGGK6BhQn12z9hK05BQJ9hjYpwLU8fGelSliMjpC5ipkLPWzWLkspHUTxgI8x+B+Ybun7QgYW4uTSdNORrs8fGQleU8FhGppgIi3J9ZHs+IT0fAD9fTcNnvCNoeR+9WV/PNyp9o8mC0E+hBQc59dDTExWloRkSqNWP9oIfaq1cvu379+ip57WsfHcoXjebBj8C7QBFceeWVLFu2jAYNGjg99KASv+OKixXsIlItGGM2WGt7lbevxo65JyfDpff9kV0XzCP4p2bEtJtEp5dqc9ZZZzFkyJCjwR4TU/qJMTHquYtItVftwz0lJ4WzGxydi24tvPUW/Omleyjo9wb19zVj1wvbadEktPQTS46xHxmKOfIYFPAiUq1V6zH3GWtm0H1mdzanbAYgIQH694c7/3w/Bf1eIyS9KXue+/GXwQ5OcIeGlh5jj4tzHoeGKthFpFqr1mPuP6b/SO+/9abIFnFvnVXEje9CfvgDFP52JqEHmrBt8o8nXjqg7Lx2zXMXkWrieGPuJ+y5G2NeN8akGGP+W6KtqTHmH8aYbe59E7fdGGNeNMZsN8ZsNMZcVHn/jF+KaBbBnKtWkplhmJJwLQ2u+4Mb7KEVC3an6OM/FhGphioyLPM3oG+ZtseAFdbacGCF+xigHxDu3oYDsyqnzPINHTqb/hf3p3BOLUxwCsnnv03jA43ZOkmXvRORwHbCE6rW2i+NMe3LNA8Eervbc4FVwFi3fZ51xnq+McaEGmNaWmv3VVrFJXTr1pp27a7g4ouhIO0AyWHJLJ6wWIt9iUjAO9XZMmElAns/EOZutwb2lDhur9v2i3A3xgzH6d3Trl27UypizJgbGDPmhlN6rohITXbas2XcXvpJn5W11s621vay1vZq0aLF6ZYhIiIlnGq4JxtjWgK49ylueyLQtsRxbdw2ERE5g0413D8EhrrbQ4GlJdrvdGfNXAocqKrxdhERObYTjrkbY97BOXna3BizF5gEPAssNMYMA3YDN7uHfwr0B7YDucDdVVCziIicQEVmy9x2jF19yjnWAg+cblEiInJ6qvXyAyIiUj6Fu4hIDaRwFxGpgfxi4TBjTCrOiVkvNQfSPK7hZKnmqlfd6gXVfKb4Q83nWmvL/aKQX4S7PzDGrD/W6mr+SjVXvepWL6jmM8Xfa9awjIhIDaRwFxGpgRTuR832uoBToJqrXnWrF1TzmeLXNWvMXUSkBlLPXUSkBlK4i4jUQAEb7saYXcaYTcaY74wx6922cq8N6zVjTKRb55GbzxjzsDEm1hiTWKK9v8d1+u31dk+y5ueMMT+4dS0xxoS67e2NMXkl3u9X/KjmY34WjDHj3Pd5qzHmej+q+d0S9e4yxnzntnv+Phtj2hpjvjDGfG+M2WyMiXbb/frzXIq1NiBvwC6geZm26cBj7vZjwDSv6yyn7mCcq1+dC8QCj3pdU4nargIuAv57ovcUZ/XQZYABLgXW+lHN1wG13O1pJWpuX/I4P3ufy/0sAF2B/wB1gQ7ADiDYH2ous/95YKK/vM9AS+Aid7sR8KP7Xvr157nkLWB77scwEOeasLj3v/OulGPqA+yw1nr9jd5fsNZ+CWSUaT7We/rz9Xattd8AoUcuAHMmlVeztXa5tbbQffgNzkVn/MYx3udjGQgssNYestbuxFmO++IqK+4YjlezMcbgLBv+zhkt6jistfustd+62weBLTiXDPXrz3NJgRzuFlhujNngXs8Vjn1tWH9yK6X/Ezzo/hn4ur8MI5Vxstfb9Td/xOmRHdHBGPN/xpjVxpgrvSrqGMr7LFSH9/lKINlau61Em9+8z8aY9sCFwFqq0ec5kMP9CmvtRUA/4AFjzFUld1rnby2/midqjKkDDADec5tmAZ2AHjgXIX/em8oqxh/f0+MxxowHCoG33aZ9QDtr7YXAKGC+MaaxV/WVUa0+C2XcRukOi9+8z8aYhsD7wMPWWl/Jff7+eQ7YcLfWJrr3KcASnD9Vj3VtWH/RD/jWWpsMYK1NttYWWWuLgb/iwZ/bFVAtr7drjLkLuBG43f1PjDu0ke5ub8AZv47wrMgSjvNZ8Pf3uRbwv8C7R9r85X02xtTGCfa3rbWL3eZq83kOyHA3xjQwxjQ6so1zAu2/HPvasP6iVA+nzJjeIJx/g7+pdtfbNcb0BcYAA6y1uSXaWxhjgt3tjkA48JM3VZZ2nM/Ch8Ctxpi6xpgOODX/+0zXdxz/A/xgrd17pMEf3mf3PMAcYIu1dkaJXdXn8+z1GV0vbkBHnBkE/wE2A+Pd9mbACmAb8DnQ1OtaS9TcAEgHQkq0vQlsAjbifLhaelzjOzh/Uh/GGXMcdqz3FGdWwcs4vbJNQC8/qnk7zvjpd+7tFffYwe7n5TvgW+C3flTzMT8LwHj3fd4K9POXmt32vwH3lTnW8/cZuAJnyGVjic9Bf3//PJe8afkBEZEaKCCHZUREajqFu4hIDaRwFxGpgRTuIiI1kMJdRKQGUriLiNRACncRkRro/wG5agbec7nuLAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ax.plot(inputs, homomorphic_predictions, color=\"green\")\n", + "display(fig)" + ] + }, + { + "cell_type": "markdown", + "id": "c18dbdd1", + "metadata": {}, + "source": [ + "### Enjoy!" + ] + } + ], + "metadata": { + "execution": { + "timeout": 10800 + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/user/explanation/FHE_AND_FRAMEWORK_LIMITS.md b/docs/user/explanation/FHE_AND_FRAMEWORK_LIMITS.md index 8a42d1a13..09fbab7c6 100644 --- a/docs/user/explanation/FHE_AND_FRAMEWORK_LIMITS.md +++ b/docs/user/explanation/FHE_AND_FRAMEWORK_LIMITS.md @@ -20,12 +20,12 @@ For most FHE scheme but TFHE, the application of a non-linear function is compli Since this is an early version of the product, not everything is done, to say the least. What we wanted to tackle first was the cryptographic complexities. This is why we concentrated on the cryptographic part, and let some engineering problems for later. -### Limited to scalars - -Today, the **Concrete Framework** is mostly limited to scalars. Notably, in our numpy frontend, we can not use [tensors](https://numpy.org/doc/stable/user/theory.broadcasting.html?highlight=vector). As explained in [this section](FUTURE_FEATURES.md), this limit will be removed in the next version. - ### Currently executing locally +```{warning} +FIXME(Benoit): we'll see later if this is still a valid limit +``` + As of today, the execution of the FHE program is done locally. Notably, in the current version, there is no client (on which we encrypt the private data, or decrypt the returned result) or server (on which the computation is done completely over encrypted data), but a single host. As explained in [this section](FUTURE_FEATURES.md), this limit will be removed in the next version, such that the **Concrete Framework** can be used in production. ### Currently slow diff --git a/docs/user/explanation/FUTURE_FEATURES.md b/docs/user/explanation/FUTURE_FEATURES.md index 8e608766e..610678ceb 100644 --- a/docs/user/explanation/FUTURE_FEATURES.md +++ b/docs/user/explanation/FUTURE_FEATURES.md @@ -5,21 +5,12 @@ is currently in a preliminary version, and quite constrained in term of function news is that we are going to release new versions regularly, where a lot of functionalities will be added progressively. In this page, we briefly list what the plans for next versions of the **Concrete Framework** are: -- **management of tensors**: today, we are mostly limited to scalars, but in the next version, the functions we compile -will possibly contain tensors, which is one of the basic features of `numpy` - **better performance**: further versions will contain improved versions of the **Concrete Library**, with faster execution; also, the **Concrete Compiler** will be improved, to have faster local execution (with multi-threading for example) and faster production execution (with distribution over a set of machines or use of hardware accelerations) -- **more user-friendly API's**: we would like to make our API easier for a user. Notably, we would like to allow direct -compilations of classic Machine Learning framework models (e.g., tensorflow or pytorch) -- **more complete benchmarks**: we will have an extended benchmark, containing lots of functions that one day one would -want to compile; then, we will measure the framework progress by tracking the number of successfully compiled functions -over time. Also, this public benchmark will be a way for other competing frameworks or technologies to compare fairly -with us, in terms of functionality or performance -- **easier installation**: we plan to have pip installation of our framework very soon -- **Machine Learning helpers**: our midterm direction is to provide our users a set of tools to help her turn her use case -in an homomorphic equivalent. This set of tools will help her reduce the needed variable precision and/or optimize the -operations required to make the fastest possible compiled model. +- **more support for torch, and support for other ML frameworks**: we will continue to extend our support for torch models, and have conversions from Keras, tensorflow +- **more complete benchmarks**: we will have an extended benchmark, containing lots of functions that one day one would want to compile; then, we will measure the framework progress by tracking the number of successfully compiled functions over time. Also, this public benchmark will be a way for other competing frameworks or technologies to compare fairly with us, in terms of functionality or performance +- **Machine Learning helpers**: our midterm direction is to provide our users a set of tools to help her turn her use case in an homomorphic equivalent. This set of tools will help her reduce the needed variable precision and/or optimize the operations required to make the fastest possible compiled model. Also, if you are especially looking for some new feature, you can drop a message to . diff --git a/docs/user/explanation/QUANTIZATION.md b/docs/user/explanation/QUANTIZATION.md index 585867392..2c2c8f0ea 100644 --- a/docs/user/explanation/QUANTIZATION.md +++ b/docs/user/explanation/QUANTIZATION.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Jordan/Andrei): see if this is still appropriate, update etc; make link with USE_QUANTIZATION.md +``` + # Quantization ```{note} diff --git a/docs/user/explanation/WHAT_IS_FHE.md b/docs/user/explanation/WHAT_IS_FHE.md index 9acdb3bd2..9526d90d2 100644 --- a/docs/user/explanation/WHAT_IS_FHE.md +++ b/docs/user/explanation/WHAT_IS_FHE.md @@ -10,3 +10,7 @@ You can learn more about FHE using the following links: - [monthly technical FHE.org meetup](https://www.meetup.com/fhe-org/) - [videos and resources](http://fhe.org/) +```{warning} +FIXME(Alex/Jeremy): should we link to Zama blogposts or not? +FIXME(Benoit): if yes, I'll do it +``` diff --git a/docs/user/howto/COMPILING_AND_EXECUTING.md b/docs/user/howto/COMPILING_AND_EXECUTING.md index 932ff707b..f0482fe30 100644 --- a/docs/user/howto/COMPILING_AND_EXECUTING.md +++ b/docs/user/howto/COMPILING_AND_EXECUTING.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(all): I've moved this section (from how-to) to the basic section, because it must be read before the tutorials +``` + # Compiling and Executing ## Importing necessary components @@ -20,6 +24,12 @@ def f(x, y): ## Compiling the function +```{warning} +FIXME(arthur): move to the new API +FIXME(all): do you think guys we should just explain the new API or should we explain the two API's? I would say: just the new one, and maybe the old one in the dev section (but not ever sure). + +``` + To compile the function, you need to provide what are the inputs that it's expecting. In the example function above, `x` and `y` could be scalars or tensors (though, for now, only dot between tensors are supported), they can be encrypted or clear, they can be signed or unsigned, they can have different bit-widths. So, we need to know what they are beforehand. We can do that like so: @@ -68,8 +78,14 @@ Be careful about the inputs, though. If you were to run with values outside the range of the inputset, the result might not be correct. ``` +```{warning} +FIXME(benoit): explain the API to encrypt, run_inference, decrypt, keygen etc when they are available + +``` + ## Further reading - [Arithmetic Operations Tutorial](../tutorial/ARITHMETIC_OPERATIONS.md) - [Working With Floating Points Tutorial](../tutorial/WORKING_WITH_FLOATING_POINTS.md) - [Table Lookup Tutorial](../tutorial/TABLE_LOOKUP.md) +- [Compiling a torch model](../tutorial/COMPILING_TORCH_MODEL.md) diff --git a/docs/user/howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md b/docs/user/howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md index 96a8e8ba4..b23cf6700 100644 --- a/docs/user/howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md +++ b/docs/user/howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md @@ -1,6 +1,6 @@ # Debugging / Support / Submitting Issues -First, let's not forget that this version of **Concrete** is a beta product, meaning that it is not completely polished, contains several bugs (would they be known or unknown at this time). Also, let's not forget that FHE is a highly hot topic, and notably, that it cannot be considered as a solved problem. +First, let's not forget that this version of **Concrete** is a first version of the product, meaning that it is not completely finished, contains several bugs (would they be known or unknown at this time), and will improve over time with the feedback from early users. Also, let's not forget that FHE is a highly hot topic, and notably, that it cannot be considered as a solved problem. Anyway, let's list some ways to debug your problems here. If nothing seems conclusive, you can still report the issue, as explained in a later section of this page. @@ -31,16 +31,22 @@ Once you're sure it is a bug, it would be nice to try to: ## Asking the community +```{warning} +FIXME(Alex): add the different links for the community +``` + We have created a Slack channel (TODO: LINK TO BE ADDED), such that you can directly ask the developers and community about your issue. Hopefully, it is just a misunderstanding or a small mistake on your side, that one can help you fix easily. And, the good point with your feedback is that, once we have heard the problem or misunderstanding, we can make the documentation even clearer (such as, completing the FAQ). ## Having a look to the compilation artifacts -When things are more complicated, or if you want to have a look by yourself, you may want to have a look to the compilation reports, which are called artifacts. This is as simple as described in [TODO: add the link to the tutorial about having artifacts]. +When things are more complicated, or if you want to have a look by yourself, you may want to have a look to the compilation reports, which are called artifacts. This is as simple as described in [here](../tutorial/COMPILATION_ARTIFACTS.md) This function will create a directory, containing notably: -[TODO: Umut to fix / complete the following information] +```{warning} +FIXME(Umut): check it is still accurate +``` - bounds.txt: a file describing the expected ranges of data in the different steps of the computation - cryptographic_parameters.txt: a file describing the different keys - ir_nodes.txt: a file describing the different nodes in the intermediate representation (IR) @@ -50,7 +56,10 @@ This function will create a directory, containing notably: Attaching the artifact with your issue or Slack message may help people to have a look at the core of the problem. The more precise your bug, the more likely we can reproduce and fix -[TODO: Umut, is it still needed or do we already have some of those information in artifacts?] +```{warning} +FIXME(Umut): is it still needed or do we already have some of those information in artifacts? +``` + In order to simplify our work and let us reproduce your bug easily, any information is useful. Notably, in addition to the python script, some information like: - the OS version - the python version @@ -62,4 +71,8 @@ may be useful to us. Don't remember, **Concrete** is a project where we are open ## Submitting an issue +```{warning} +FIXME(Alex): add the link +``` + In case you have a bug, which is reproducible, that you have reduced to a small piece of code, we have our issue tracker (TODO: LINK TO BE ADDED). Remember that a well-described short issue is an issue which is more likely to be studied and fixed. The more issues we receive, the better the product will be. diff --git a/docs/user/howto/INSTALLING.md b/docs/user/howto/INSTALLING.md index af1e45adc..4934f3f73 100644 --- a/docs/user/howto/INSTALLING.md +++ b/docs/user/howto/INSTALLING.md @@ -3,7 +3,11 @@ ## Docker image ```{note} -Currently the project is only available as a docker image. To get the image you need to login to ghcr.io with docker. +The easiest way to install the framework is as a docker image. To get the image you need to login to ghcr.io with docker. +``` + +```{warning} +FIXME(Arthur): to check this is still valid ``` ```shell @@ -37,3 +41,11 @@ Alternatively you can just open a shell in the docker: ```shell docker run --rm -it ghcr.io/zama-ai/concretefhe:v0.1.0 /bin/bash ``` + +## python package + +```{warning} +FIXME(Arthur): explain how to install from pypi, when it is ready +``` + + diff --git a/docs/user/howto/PRINTING_AND_DRAWING.md b/docs/user/howto/PRINTING_AND_DRAWING.md index c660d9c47..8bb477350 100644 --- a/docs/user/howto/PRINTING_AND_DRAWING.md +++ b/docs/user/howto/PRINTING_AND_DRAWING.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(all): should we update to the new API or have it for both? +``` + # Printing and Drawing Sometimes, it can be useful to print or draw fhe circuits, we provide methods to just do that. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile your function into an fhe circuit. diff --git a/docs/user/howto/REDUCE_NEEDED_PRECISION.md b/docs/user/howto/REDUCE_NEEDED_PRECISION.md index 3c2d505fe..eb4f413e6 100644 --- a/docs/user/howto/REDUCE_NEEDED_PRECISION.md +++ b/docs/user/howto/REDUCE_NEEDED_PRECISION.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Umut): put somewhere an example of an error dump if the user asks for too much precision and explain +``` + # Having a Function Which Requires Less Precision ## Why can some computation work with less precision? @@ -23,3 +27,7 @@ If for some reason you have a tolerance on the result's precision, then you can This is illustrated in both advanced examples [Quantized Linear Regression](../advanced_examples/QuantizedLinearRegression.ipynb) and [Quantized Logistic Regression](../advanced_examples/QuantizedLogisticRegression.ipynb). The end result has a granularity/imprecision linked to the data types used and for the Quantized Logistic Regression to the lattice used to evaluate the logistic model. + +```{warning} +FIXME(jordan/andrei): update with your insights / knowledge on ML +``` diff --git a/docs/user/howto/USE_QUANTIZATION.md b/docs/user/howto/USE_QUANTIZATION.md new file mode 100644 index 000000000..91627bced --- /dev/null +++ b/docs/user/howto/USE_QUANTIZATION.md @@ -0,0 +1,9 @@ +```{warning} +FIXME(Jordan): do this +``` + +# Using Quantization in Concrete Framework + +(as a user) + +explain issues the user will still fail and that we plan to research on it \ No newline at end of file diff --git a/docs/user/index.rst b/docs/user/index.rst index 79a673578..33b56017d 100644 --- a/docs/user/index.rst +++ b/docs/user/index.rst @@ -8,22 +8,26 @@ Getting Started README.md howto/INSTALLING.md benchmarks.md + howto/COMPILING_AND_EXECUTING.md .. toctree:: :maxdepth: 1 :caption: Tutorial + tutorial/COMPILING_TORCH_MODEL.md tutorial/ARITHMETIC_OPERATIONS.md tutorial/TABLE_LOOKUP.md tutorial/WORKING_WITH_FLOATING_POINTS.md + tutorial/INDEXING.md + tutorial/MACHINE_LEARNING_TOOLS.md tutorial/COMPILATION_ARTIFACTS.md .. toctree:: :maxdepth: 1 :caption: How to - howto/COMPILING_AND_EXECUTING.md howto/PRINTING_AND_DRAWING.md + howto/USE_QUANTIZATION.md howto/REDUCE_NEEDED_PRECISION.md howto/DEBUG_SUPPORT_SUBMIT_ISSUES.md howto/FAQ.md @@ -34,6 +38,7 @@ Getting Started advanced_examples/QuantizedLinearRegression.ipynb advanced_examples/QuantizedLogisticRegression.ipynb + advanced_examples/QuantizedGeneralizedLinearModel.ipynb .. toctree:: :maxdepth: 1 diff --git a/docs/user/tutorial/ARITHMETIC_OPERATIONS.md b/docs/user/tutorial/ARITHMETIC_OPERATIONS.md index d4446deb3..4264b224d 100644 --- a/docs/user/tutorial/ARITHMETIC_OPERATIONS.md +++ b/docs/user/tutorial/ARITHMETIC_OPERATIONS.md @@ -1,3 +1,10 @@ +```{warning} +FIXME(Umut): update a bit, with the new API +FIXME(Umut/Arthur): update a bit to explain things with the tensors and new operations. At the same time, I think we can exhaustively give examples of every supported functions, since we start to have a lot, so maybe, we would just explain a bit? +FIXME(all): actually, I am not even sure we should keep this .md, it can't be exhaustive enough, and looks pretty trivial. What do you think + +``` + # Arithmetic Operations In this tutorial, we are going to go over all arithmetic operations available in **Concrete**. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile the functions below. diff --git a/docs/user/tutorial/COMPILATION_ARTIFACTS.md b/docs/user/tutorial/COMPILATION_ARTIFACTS.md index 207f08c17..4ba939dd4 100644 --- a/docs/user/tutorial/COMPILATION_ARTIFACTS.md +++ b/docs/user/tutorial/COMPILATION_ARTIFACTS.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Umut): check it is still valid. I guess yes, but some files may have gone or renamed. +``` + # Compilation Artifacts In this tutorial, we are going to go over the artifact system, which is designed to inspect/debug the compilation process easily. diff --git a/docs/user/tutorial/COMPILING_TORCH_MODEL.md b/docs/user/tutorial/COMPILING_TORCH_MODEL.md new file mode 100644 index 000000000..20b7498b5 --- /dev/null +++ b/docs/user/tutorial/COMPILING_TORCH_MODEL.md @@ -0,0 +1,4 @@ +```{warning} +FIXME(jordan): do this section, maybe from one .ipynb that you would do +``` +# Compiling a Torch Model diff --git a/docs/user/tutorial/INDEXING.md b/docs/user/tutorial/INDEXING.md new file mode 100644 index 000000000..4a9e077bf --- /dev/null +++ b/docs/user/tutorial/INDEXING.md @@ -0,0 +1,9 @@ +```{warning} +FIXME(Umut): to be done +``` + +# Indexing + +# Slicing + + diff --git a/docs/user/tutorial/MACHINE_LEARNING_TOOLS.md b/docs/user/tutorial/MACHINE_LEARNING_TOOLS.md new file mode 100644 index 000000000..a982af56c --- /dev/null +++ b/docs/user/tutorial/MACHINE_LEARNING_TOOLS.md @@ -0,0 +1,14 @@ +```{warning} +FIXME(Andrei/Jordan): to be done +``` + +# Machine Learning Tools + +Give examples with matrix multiplications, reshape, relu, sigmoid, flatten, clip, transpose, dot etc + +Do we need to speak about batch norm? + +Do we need to speak about sum? + +Do we need to speak about pools? + diff --git a/docs/user/tutorial/TABLE_LOOKUP.md b/docs/user/tutorial/TABLE_LOOKUP.md index e96b60cc4..30074e43a 100644 --- a/docs/user/tutorial/TABLE_LOOKUP.md +++ b/docs/user/tutorial/TABLE_LOOKUP.md @@ -1,3 +1,8 @@ +```{warning} +FIXME(Umut): update a bit, with the new API +FIXME(Umut): add the multiTLU case +``` + # Table Lookup In this tutorial, we are going to go over the ways to perform table lookups in **Concrete**. Please read [Compiling and Executing](../howto/COMPILING_AND_EXECUTING.md) before reading further to see how you can compile the functions below. diff --git a/docs/user/tutorial/WORKING_WITH_FLOATING_POINTS.md b/docs/user/tutorial/WORKING_WITH_FLOATING_POINTS.md index 32b54d454..c0cf62dd5 100644 --- a/docs/user/tutorial/WORKING_WITH_FLOATING_POINTS.md +++ b/docs/user/tutorial/WORKING_WITH_FLOATING_POINTS.md @@ -1,3 +1,7 @@ +```{warning} +FIXME(Arthur): update a bit, with the new API +``` + # Working With Floating Points ## An example @@ -110,6 +114,10 @@ List of supported binary functions if one of the two operators is a constant sca - true_divide +```{warning} +FIXME(Benoit): see what kind of other supported operations we could list here +``` + ## Limitations Floating point support in **Concrete** is very limited for the time being. They can't appear on inputs, or they can't be outputs. However, they can be used in intermediate results. Unfortunately, there are limitations on that front as well.