mirror of
https://github.com/tlsnotary/tlsn.git
synced 2026-01-09 14:48:13 -05:00
Tee dev cleanup (#759)
* build: added scripts for local tee/sgx development * Improved documentation: move all explanation to one README file
This commit is contained in:
Hendrik Eeckhaut
22
.github/workflows/ci.yml
vendored
22
.github/workflows/ci.yml
vendored
@@ -280,7 +280,7 @@ jobs:
|
||||
crates/notary/server/tee/notary-server.manifest
|
||||
crates/notary/server/tee/notary-server.manifest.sgx
|
||||
crates/notary/server/tee/config
|
||||
crates/notary/server/tee/notary-server-sgx.md
|
||||
crates/notary/server/tee/README.md
|
||||
if-no-files-found: error
|
||||
|
||||
- name: Attest Build Provenance
|
||||
@@ -304,24 +304,17 @@ jobs:
|
||||
CONTAINER_REGISTRY: ghcr.io
|
||||
if: github.ref == 'refs/heads/dev' || (startsWith(github.ref, 'refs/tags/v') && contains(github.ref, '.'))
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
sparse-checkout: './crates/notary/server/tee/notary-server-sgx.Dockerfile'
|
||||
|
||||
- name: Download notary-server-sgx.zip from gramine-sgx job
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: notary-server-sgx.zip
|
||||
path: ./notary-server-sgx
|
||||
|
||||
- name: Create Dockerfile
|
||||
run: |
|
||||
cat <<EOF > ./Dockerfile
|
||||
FROM gramineproject/gramine:latest
|
||||
WORKDIR /work
|
||||
COPY ./notary-server-sgx /work
|
||||
RUN chmod +x /work/notary-server
|
||||
LABEL org.opencontainers.image.source=https://github.com/tlsnotary/tlsn
|
||||
LABEL org.opencontainers.image.description="TLSNotary notary server in SGX/Gramine."
|
||||
ENTRYPOINT ["gramine-sgx", "notary-server"]
|
||||
EOF
|
||||
|
||||
- name: Log in to the Container registry
|
||||
uses: docker/login-action@v2
|
||||
with:
|
||||
@@ -342,7 +335,8 @@ jobs:
|
||||
push: true
|
||||
tags: ${{ steps.meta-notary-server-sgx.outputs.tags }}
|
||||
labels: ${{ steps.meta-notary-server-sgx.outputs.labels }}
|
||||
file: ./Dockerfile
|
||||
file: ./crates/notary/server/tee/notary-server-sgx.Dockerfile
|
||||
|
||||
build_and_publish_notary_server_image:
|
||||
name: Build and publish notary server's image
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -1,3 +1,114 @@
|
||||
This folder contains the necessary files to build a Docker image for running the Notary Server on Intel SGX-enabled hardware.
|
||||
|
||||
The container is built as part of the CI pipeline. For details on the build process, refer to the [CI workflow configuration](../../../../.github/workflows/ci.yml).
|
||||
## Compile the Notary Server for Intel SGX
|
||||
|
||||
We use [Gramine](https://github.com/gramineproject/gramine) to run the Notary Server on Intel SGX. Gramine allows the Notary Server to run in an isolated environment with minimal host requirements.
|
||||
|
||||
The isolated environment is defined via the manifest template (`notary-server.manifest.template`).
|
||||
|
||||
The Notary Server for SGX is compiled with the Rust feature flag `tee_quote`. This enables the server to use an ephemeral private notary key for signing attestations (`private_key_pem_path: "/ephemeral/notary.key"`) and also adds the SGX *quote* to the server's `/info` endpoint.
|
||||
|
||||
### CI
|
||||
|
||||
The [notary-server-sgx Docker container](https://github.com/tlsnotary/tlsn/pkgs/container/tlsn%2Fnotary-server-sgx) is built as part of the CI pipeline. For details on the build process, refer to the [CI workflow configuration](../../../../.github/workflows/ci.yml).
|
||||
|
||||
CI builds a zip file named `notary-server-sgx.zip`, which contains the compiled binary and the signed manifest. This zip file is available for all releases and `dev` builds in the build artifacts. We also publish a Docker image `notary-server-sgx` at <https://github.com/tlsnotary/tlsn/pkgs/container/tlsn%2Fnotary-server-sgx>. Check the section below for details on running this container.
|
||||
|
||||
### Development
|
||||
|
||||
You can also build everything locally using the `run-gramine-local.sh` script.
|
||||
|
||||
This script creates and signs the Gramine manifest for the Notary Server in a local development environment. It requires the Gramine SDK, so the most convenient way to use it is within a Docker container that includes the necessary dependencies and tools.
|
||||
|
||||
> ⚠️ This script assumes that the `notary-server` binary is already built (for `linux/amd64`) and available in the current directory. Make sure it is built with the `tee_quote` feature:
|
||||
> `cargo build --bin notary-server --release --features tee_quote`
|
||||
|
||||
#### Build the Docker Image
|
||||
|
||||
To build the Docker image for local development, run:
|
||||
```sh
|
||||
docker build -f gramine-local.Dockerfile -t gramine-local .
|
||||
```
|
||||
#### Run the Gramine Script
|
||||
|
||||
Once the image is built, you can run the `run-gramine-local.sh` script inside the container:
|
||||
```
|
||||
docker run --rm -it \
|
||||
--platform=linux/amd64 \
|
||||
-v "${PWD}:/app" \
|
||||
-w /app/ \
|
||||
gramine-local \
|
||||
"bash -c ./run-gramine-local.sh"
|
||||
```
|
||||
|
||||
If successful, the script will generate the following files:
|
||||
* `notary-server.sig`
|
||||
* `notary-server-sigstruct.json`
|
||||
* `notary-server.manifest`
|
||||
* `notary-server.manifest.sgx`
|
||||
|
||||
|
||||
You can verify that the provided **enclave signature (`notary-server.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values in `notary-server-sigstruct.json`, by running the following command inside a **Gramine Docker container** to inspect the enclave's signature:
|
||||
|
||||
```sh
|
||||
docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \
|
||||
"gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig"
|
||||
```
|
||||
|
||||
The output should be the same as `notary-server-sigstruct.json`
|
||||
|
||||
## How to Run TLSNotary on Intel SGX?
|
||||
|
||||
Before running the Notary Server on Intel SGX hardware, ensure your system has the required Intel SGX components installed:
|
||||
```sh
|
||||
wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
|
||||
cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
|
||||
|
||||
# Add the repository to your sources:
|
||||
echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
|
||||
|
||||
sudo apt-get update
|
||||
sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y
|
||||
```
|
||||
|
||||
For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).**
|
||||
|
||||
### Docker Compose
|
||||
|
||||
To run the Notary Server using Docker Compose, create a docker-compose.yml file like the following:
|
||||
```yaml
|
||||
services:
|
||||
dev:
|
||||
container_name: dev
|
||||
image: ghcr.io/tlsnotary/tlsn/notary-server-sgx:dev
|
||||
restart: unless-stopped
|
||||
devices:
|
||||
- /dev/sgx_enclave
|
||||
- /dev/sgx_provision
|
||||
volumes:
|
||||
- /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket
|
||||
ports:
|
||||
- "7047:7047"
|
||||
entrypoint: [ "gramine-sgx", "notary-server" ]
|
||||
```
|
||||
|
||||
To retrieve the SGX attestation quote, query the `/info` endpoint:
|
||||
```sh
|
||||
curl localhost:7047/info | jq
|
||||
```
|
||||
|
||||
### Run local build directly with Gramine
|
||||
|
||||
To run a locally built Notary Server inside a Gramine-protected SGX enclave, execute:
|
||||
```sh
|
||||
docker run --detach \
|
||||
--restart=unless-stopped \
|
||||
--device=/dev/sgx_enclave \
|
||||
--device=/dev/sgx_provision \
|
||||
--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
|
||||
--publish=7047:7047 \
|
||||
--volume="$(pwd):/work" \
|
||||
--workdir=/work \
|
||||
gramineproject/gramine:latest \
|
||||
"gramine-sgx notary-server"
|
||||
```
|
||||
6
crates/notary/server/tee/gramine-local.Dockerfile
Normal file
6
crates/notary/server/tee/gramine-local.Dockerfile
Normal file
@@ -0,0 +1,6 @@
|
||||
FROM --platform=linux/amd64 gramineproject/gramine:latest
|
||||
|
||||
RUN apt update && \
|
||||
apt install -y jq openssl zip && \
|
||||
apt clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
11
crates/notary/server/tee/notary-server-sgx.Dockerfile
Normal file
11
crates/notary/server/tee/notary-server-sgx.Dockerfile
Normal file
@@ -0,0 +1,11 @@
|
||||
FROM gramineproject/gramine:latest
|
||||
WORKDIR /work
|
||||
|
||||
# Copies `notary-server-sgx.zip` from the CI build or created locally via `run-gramine-local.sh`.
|
||||
COPY ./notary-server-sgx /work
|
||||
RUN chmod +x /work/notary-server
|
||||
|
||||
LABEL org.opencontainers.image.source=https://github.com/tlsnotary/tlsn
|
||||
LABEL org.opencontainers.image.description="TLSNotary notary server in SGX/Gramine."
|
||||
|
||||
ENTRYPOINT ["gramine-sgx", "notary-server"]
|
||||
@@ -1,68 +0,0 @@
|
||||
# TLSNotary Notary Server for Intel SGX
|
||||
|
||||
This package contains the **SGX-enabled version of the TLSNotary Notary Server**, built with **Gramine** to run securely on Intel SGX hardware. Follow the instructions below to install dependencies, verify the integrity of the enclave, and launch the server.
|
||||
|
||||
## Install Intel SGX Drivers & Runtime Dependencies
|
||||
|
||||
Ensure your system has the required Intel SGX components installed.
|
||||
|
||||
```sh
|
||||
wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
|
||||
cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
|
||||
|
||||
# Add the repository to your sources:
|
||||
echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
|
||||
|
||||
sudo apt-get update
|
||||
sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y
|
||||
```
|
||||
|
||||
For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).**
|
||||
|
||||
## Verify the Enclave Signature (`MR_ENCLAVE`, `MR_SIGNER`)
|
||||
|
||||
Before running the Notary Server, verify that the provided **enclave signature (`.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values.
|
||||
|
||||
Run the following command inside a **Gramine Docker container** to inspect the enclave's signature:
|
||||
|
||||
```sh
|
||||
docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \
|
||||
"gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig"
|
||||
```
|
||||
|
||||
The output should be the same as `notary-server-sigstruct.json`
|
||||
|
||||
## Run the Notary Server in SGX
|
||||
|
||||
Once verification is complete, you can start the Notary Server inside a **Gramine-protected SGX enclave**.
|
||||
|
||||
```sh
|
||||
docker run --detach \
|
||||
--restart=unless-stopped \
|
||||
--device=/dev/sgx_enclave \
|
||||
--device=/dev/sgx_provision \
|
||||
--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
|
||||
--publish=7047:7047 \
|
||||
--volume="$(pwd):/work" \
|
||||
--workdir=/work \
|
||||
gramineproject/gramine:latest \
|
||||
"gramine-sgx notary-server"
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `--device /dev/sgx_enclave --device /dev/sgx_provision` → Exposes SGX devices.
|
||||
- `--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket` → Enables access to **Intel's AESM daemon** (required for remote attestation).
|
||||
- `-p 7047:7047` → Exposes the notary server on port **7047**.
|
||||
|
||||
## Attestation & Verification
|
||||
|
||||
The Notary Server runs inside an **Intel SGX enclave**, which supports **remote attestation**. When connecting to it, clients should request an **SGX quote** to verify:
|
||||
|
||||
- **MR_ENCLAVE** (ensures the correct enclave binary is running).
|
||||
- **MR_SIGNER** (ensures the enclave was signed by the expected key).
|
||||
- **Quote Freshness** (prevents replay attacks).
|
||||
|
||||
To retrieve the SGX attestation quote, navigate to `<your notary server>:7047/info`:
|
||||
```
|
||||
curl localhost:7047/info | jq
|
||||
```
|
||||
49
crates/notary/server/tee/run-gramine-local.sh
Executable file
49
crates/notary/server/tee/run-gramine-local.sh
Executable file
@@ -0,0 +1,49 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
echo "[*] Generating SGX signing key..."
|
||||
gramine-sgx-gen-private-key
|
||||
|
||||
if [ ! -f notary-server ]; then
|
||||
echo "[!] notary-server binary not found. Please copy it from ci, or build it first."
|
||||
echo "Note that notary-server must be built for linux/amd64 with tee_quote feature enabled"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
chmod +x notary-server
|
||||
|
||||
echo "[*] Creating Gramine manifest..."
|
||||
gramine-manifest \
|
||||
-Dlog_level=debug \
|
||||
-Darch_libdir=/lib/x86_64-linux-gnu \
|
||||
-Dself_exe=notary-server \
|
||||
notary-server.manifest.template \
|
||||
notary-server.manifest
|
||||
|
||||
echo "[*] Signing manifest..."
|
||||
gramine-sgx-sign \
|
||||
--manifest notary-server.manifest \
|
||||
--output notary-server.manifest.sgx
|
||||
|
||||
echo "[*] Viewing SIGSTRUCT..."
|
||||
gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig >notary-server-sigstruct.json
|
||||
|
||||
cat notary-server-sigstruct.json | jq .
|
||||
|
||||
mr_enclave=$(jq -r ".mr_enclave" notary-server-sigstruct.json)
|
||||
mr_signer=$(jq -r ".mr_signer" notary-server-sigstruct.json)
|
||||
|
||||
echo "=============================="
|
||||
echo "MRENCLAVE: $mr_enclave"
|
||||
echo "MRSIGNER: $mr_signer"
|
||||
echo "=============================="
|
||||
|
||||
zip -r notary-server-sgx.zip \
|
||||
notary-server \
|
||||
notary-server-sigstruct.json \
|
||||
notary-server.sig \
|
||||
notary-server.manifest \
|
||||
notary-server.manifest.sgx \
|
||||
config \
|
||||
README.md
|
||||
Reference in New Issue
Block a user