ci: calculate SGX mr_enclave for notary server in gramine docker (#701)

* calculate SGX mr_enclave for notary server in gramine docker * remove old tee github workflow * attest build result for dev branch builds and releases
2026-01-09 14:48:13 -05:00 · 2025-03-03 13:29:47 +01:00
parent 63bd6abc5d
commit be0be19018
4 changed files with 193 additions and 42 deletions
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -8,6 +8,11 @@ on:
      - "[v]?[0-9]+.[0-9]+.[0-9]+*"
  pull_request:

+permissions:
+  id-token: write
+  contents: read
+  attestations: write
+
 env:
  CARGO_TERM_COLOR: always
  CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
@@ -17,7 +22,7 @@ env:
  #   - https://github.com/privacy-scaling-explorations/mpz/issues/178
  # 32 seems to be big enough for the foreseeable future
  RAYON_NUM_THREADS: 32
-
+  GIT_COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
 jobs:
  fmt:
    name: Check formatting
@@ -38,6 +43,7 @@ jobs:

      - name: Check formatting
        run: cargo +nightly fmt --check --all
+
  build-and-test:
    name: Build and test
    runs-on: ubuntu-latest
@@ -62,6 +68,7 @@ jobs:

      - name: Test
        run: cargo test
+
  wasm:
    name: Build and Test wasm
    runs-on: ubuntu-latest
@@ -136,6 +143,7 @@ jobs:

      - name: Run integration tests
        run: cargo test --profile tests-integration --workspace --exclude tlsn-tls-client --exclude tlsn-tls-core -- --include-ignored
+
  coverage:
    runs-on: ubuntu-latest
    env:
@@ -155,4 +163,118 @@ jobs:
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          files: lcov.info
-          fail_ci_if_error: true
+          fail_ci_if_error: true
+
+  build-sgx:
+    runs-on: ubuntu-latest
+    needs: build-and-test
+    container:
+      image: rust:latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Install Clang
+        run: |
+          apt update
+          apt install -y clang
+
+      - name: Use caching
+        uses: Swatinem/rust-cache@v2.7.3
+
+      - name: Build Rust Binary
+        run: |
+          cargo build --bin notary-server --release --features tee_quote
+          cp --verbose target/release/notary-server $GITHUB_WORKSPACE
+
+      - name: Upload Binary for use in the Gramine Job
+        uses: actions/upload-artifact@v4
+        with:
+          name: notary-server
+          path: notary-server
+          if-no-files-found: error
+
+  gramine-sgx:
+    runs-on: ubuntu-latest
+    needs: build-sgx
+    environment: tee
+    container:
+      image: gramineproject/gramine:latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Restore SGX signing key from secrets
+        run: |
+          mkdir -p "${HOME}/.config/gramine/"
+          echo "${{ secrets.SGX_SIGNING_KEY }}" > "${HOME}/.config/gramine/enclave-key.pem"
+          # verify key
+          openssl rsa -in "${HOME}/.config/gramine/enclave-key.pem" -check -noout
+
+      - name: Download notary-server binary from build job
+        uses: actions/download-artifact@v4
+        with:
+          name: notary-server
+          path: crates/notary/server/tee
+
+      - name: Install jq
+        run: |
+          apt update
+          apt install -y jq
+
+      - name: Use Gramine to calculate measurements
+        run: |
+          cd crates/notary/server/tee
+
+          chmod +x notary-server
+
+          gramine-manifest \
+            -Dlog_level=debug \
+            -Darch_libdir=/lib/x86_64-linux-gnu \
+            -Dself_exe=notary-server \
+            notary-server.manifest.template \
+            notary-server.manifest
+
+          gramine-sgx-sign \
+            --manifest notary-server.manifest \
+            --output notary-server.manifest.sgx
+
+          gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig | tee >> notary-server-sigstruct.json
+
+          cat notary-server-sigstruct.json
+
+          mr_enclave=$(jq -r '.mr_enclave' notary-server-sigstruct.json)
+          mr_signer=$(jq -r '.mr_signer' notary-server-sigstruct.json)
+
+          echo "mrenclave=$mr_enclave" >>"$GITHUB_OUTPUT"
+          echo "#### sgx mrenclave" | tee >>$GITHUB_STEP_SUMMARY
+          echo "\`\`\`mr_enclave: ${mr_enclave}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY
+          echo "\`\`\`mr_signer: ${mr_signer}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY
+
+      - name: Upload notary-server and signatures
+        id: upload-notary-server-sgx
+        uses: actions/upload-artifact@v4
+        with:
+          name: notary-server-sgx.zip
+          path: |
+            crates/notary/server/tee/notary-server
+            crates/notary/server/tee/notary-server-sigstruct.json
+            crates/notary/server/tee/notary-server.sig
+            crates/notary/server/tee/notary-server.manifest
+            crates/notary/server/tee/notary-server.manifest.sgx
+            crates/notary/server/tee/config
+            crates/notary/server/tee/notary-server-sgx.md
+          if-no-files-found: error
+
+      - name: Attest Build Provenance
+        if: startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/dev'
+        uses: actions/attest-build-provenance@v2
+        with:
+            subject-name: notary-server-sgx.zip
+            subject-digest: sha256:${{ steps.upload-notary-server-sgx.outputs.artifact-digest }}
+
+      - uses: geekyeggo/delete-artifact@v5 # Delete notary-server from the build job, It is part of the zipfile with the signature
+        with:
+          name: notary-server
--- a/.github/workflows/tee-ci.yml
+++ b/.github/workflows/tee-ci.yml
@@ -1,39 +0,0 @@
-name: tee-build
-
-on:
-  push:
-    branches: [ "dev" ]
-
-concurrency:
-  group: ${{ github.head_ref || github.run_id }}
-  cancel-in-progress: true
-
-jobs:
-   build-measure-emulated:
-      environment: tee
-      runs-on: [self-hosted, linux]
-      container:
-        image: notaryserverbuilds.azurecr.io/prod/gramine
-        credentials:
-          username: notaryserverbuilds
-          password: ${{ secrets.AZURE_CR_BUILDS_PW }}
-        env:
-          GIT_COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
-      steps:
-        - name: get code
-          uses: actions/checkout@v4
-        - name: sccache
-          if: github.event_name != 'release'
-          # && github.event_name != 'workflow_dispatch'
-          uses: mozilla-actions/sccache-action@v0.0.6
-        - name: set rust env for scc
-          if: github.event_name != 'release'
-          # && github.event_name != 'workflow_dispatch'
-          run: |
-            echo "SCCACHE_GHA_ENABLED=true" >> $GITHUB_ENV
-            echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
-    
-        - name: get emulated measurement (call gramine.sh without the sgx arg)
-          working-directory: ${{ github.workspace }}/crates/notary/server/tee
-          run: |
-            chmod +x  ../../../../.github/scripts/gramine.sh && ../../../../.github/scripts/gramine.sh
--- a/crates/notary/server/tee/notary-server-sgx.md
+++ b/crates/notary/server/tee/notary-server-sgx.md
@@ -0,0 +1,68 @@
+# TLSNotary Notary Server for Intel SGX
+
+This package contains the **SGX-enabled version of the TLSNotary Notary Server**, built with **Gramine** to run securely on Intel SGX hardware. Follow the instructions below to install dependencies, verify the integrity of the enclave, and launch the server.
+
+## Install Intel SGX Drivers & Runtime Dependencies
+
+Ensure your system has the required Intel SGX components installed.
+
+```sh
+wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
+cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
+
+# Add the repository to your sources:
+echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
+
+sudo apt-get update
+sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y
+```
+
+For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).**
+
+## Verify the Enclave Signature (`MR_ENCLAVE`, `MR_SIGNER`)
+
+Before running the Notary Server, verify that the provided **enclave signature (`.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values.
+
+Run the following command inside a **Gramine Docker container** to inspect the enclave's signature:
+
+```sh
+docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \
+  "gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig"
+```
+
+The output should be the same as `notary-server-sigstruct.json`
+
+## Run the Notary Server in SGX
+
+Once verification is complete, you can start the Notary Server inside a **Gramine-protected SGX enclave**.
+
+```sh
+docker run --detach \
+  --restart=unless-stopped \
+  --device=/dev/sgx_enclave \
+  --device=/dev/sgx_provision \
+  --volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
+  --publish=7047:7047 \
+  --volume="$(pwd):/work" \
+  --workdir=/work \
+  gramineproject/gramine:latest \
+  "gramine-sgx notary-server"
+```
+
+Notes:
+- `--device /dev/sgx_enclave --device /dev/sgx_provision` → Exposes SGX devices.
+- `--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket` → Enables access to **Intel's AESM daemon** (required for remote attestation).
+- `-p 7047:7047` → Exposes the notary server on port **7047**.
+
+## Attestation & Verification
+
+The Notary Server runs inside an **Intel SGX enclave**, which supports **remote attestation**. When connecting to it, clients should request an **SGX quote** to verify:
+
+- **MR_ENCLAVE** (ensures the correct enclave binary is running).
+- **MR_SIGNER** (ensures the enclave was signed by the expected key).
+- **Quote Freshness** (prevents replay attacks).
+
+To retrieve the SGX attestation quote, navigate to `<your notary server>:7047/info`:
+```
+curl localhost:7047/info | jq
+```
--- a/crates/notary/server/tee/notary-server.manifest.template
+++ b/crates/notary/server/tee/notary-server.manifest.template
@@ -42,4 +42,4 @@ sys.disallow_subprocesses = true
 #### tlsn rev
 sgx.isvprodid = 7
 #### F
-sgx.isvsvn = 46
+sgx.isvsvn = 1