After running performances regression benchmarks, a performance
changes checking is executed. It will fetch results data with an
external tool then it will look for anomaly in changes.
Finally it will produce a report as an issue comment with any
anomaly display in a Markdown array. A folded section of the
report message contains all the results from the benchmark.
Note that a fully custom benchmark triggered from an issue comment
would not generate a report. In addition HPU performance
regression benchmark is not supported yet.
Switch n3-H100-SXM5x8 to US-1 as CANADA is out of stock on this
instance.
Also L40 instances fallback on n3-RTX-A6000x1 to mitigate
resource shortages issues.
Several network errors occurred while trying to install git-lfs
from within backward compatibility tests workflow. Having git-lfs
installed directly in the Amazon Machine Image fix this issue.
Some libs were missing to be able to run Firefox out of the box.
Besides, action runner is now installed as ubuntu user since
Firefox is not able to run as root.
Swtich from Jest and Puppeteer to Python with Selenium. It relies
on browser and webdriver binaries from browser fabricators.
For now the Python script only supports Chrome browser.
Only GPUs with peer access to GPU 0 can be used for this at the moment.
Peer to peer copy is used if different GPUs are passed to memcpy_gpu_to_gpu
A gpu offset is passed as new parameter to pbs and keyswitch to adjust the input/output index user per gpu.
bsk and ksk are copied to all GPUs.
The CI now tests & runs benchmarks on p3.8xlarge aws instances
This is done to mitigate resource shortages in our base AWS region
(eu-west-3) due to the high number of instances that are launched
in parallel in our Pull Requests.
This is done to get quick feedback to developpers in a Pull Request.
It tests shorint level with only three sets of parameters. Integer
level is tested with only the default operations with two sets of
parameters.
This profile will be automatically triggered on each push in a
pull request. Conversely the full suite of test will also be
triggered automatically but once the review is approved.
