mirror of
https://github.com/zama-ai/concrete.git
synced 2026-02-08 19:44:57 -05:00
3c2a75186f
This CI "feature" is meant to circumvent the 6 hours hard-limit
for a job in GitHub Action.
The benchmark is done using a matrix which is handled by Slab.
Here's the workflow:
1. ML benchmarks are started in a fire and forget fashion via
start_ml_benchmarks.yml
2. Slab will read ci/slab.toml to get the AWS EC2 configuration
and the matrix parameters
3. Slab will launch at most max_parallel_jobs EC2 instances in
parallel
4. Each job will trigger ml_benchmark_subset.yml which will run
only one of the generated YAML file via make generate-mlbench,
based on the value of the matrix item they were given.
5. As soon as a job is completed, the next one in the matrix
will start promptly.
This is done until all the matrix items are exhausted.
31 lines
1.1 KiB
YAML
31 lines
1.1 KiB
YAML
# Start application benchmarks job on Slab CI bot.
|
|
name: Start ML benchmarks
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
|
|
# Have a weekly benchmark run on main branch to be available on Monday morning (Paris time)
|
|
# TODO: uncomment this section once MLBenchmarks are implemented
|
|
# schedule:
|
|
# # * is a special character in YAML so you have to quote this string
|
|
# # At 1:00 every Thursday
|
|
# # Timezone is UTC, so Paris time is +2 during the summer and +1 during winter
|
|
# - cron: '0 1 * * THU'
|
|
|
|
jobs:
|
|
start-ml-benchmarks:
|
|
steps:
|
|
- name: Start AWS job in Slab
|
|
shell: bash
|
|
# TODO: step result must be correlated to HTTP return code.
|
|
run: |
|
|
PAYLOAD='{"command": "ml-bench", "git_ref": ${{ github.ref }}, "sha": ${{ github.sha }}}'
|
|
SIGNATURE="$(slab/scripts/hmac_calculator.sh ${PAYLOAD} '${{ secrets.JOB_SECRET }}')"
|
|
curl -v -k \
|
|
-H "Content-Type: application/json" \
|
|
-H "X-Slab-Repository: ${{ github.repository }}" \
|
|
-H "X-Slab-Command: start_data" \
|
|
-H "X-Hub-Signature-256: sha256=${SIGNATURE}" \
|
|
-d @${{ PAYLOAD }} \
|
|
${{ secrets.SLAB_URL }}
|