Compare commits

..

37 Commits

Author SHA1 Message Date
Pratik Basyal
1c31716923 Footnote update (#4966) 2025-06-26 15:33:29 -04:00
Pratik Basyal
c7e46fff48 6.1.5 column added to docs/6.2.1 (#4633)
* 6.1.5 column added
2025-04-17 11:47:47 -04:00
alexxu-amd
720cfafb20 Update index.md
(cherry picked from commit fe69fc1bb4)
2024-12-23 18:10:31 -05:00
Peter Park
06f3caf882 fix RDC version str 2024-11-05 17:13:54 -05:00
Peter Park
4c627a64a2 add Omniperf changelog 2024-11-05 17:13:54 -05:00
Peter Park
748fae0d77 Merge pull request #3976 from peterjunpark/docs/6.2.1
Update links to vllm perf validation doc (#3971)
2024-10-30 18:44:31 -04:00
Peter Park
06bc24dea4 Update links to vllm perf validation doc (#3971)
* update links to vllm perf validation doc

* add PagedAttention to wordlist

(cherry picked from commit 0fe08d93d7)

fix link
2024-10-30 18:33:36 -04:00
Peter Park
a511d9c086 Merge pull request #3967 from peterjunpark/docs/6.2.1
add vllm performance validation doc (#3964)
2024-10-30 14:32:59 -04:00
Peter Park
6a55e2a2f2 add suggestions to vllm perf validation doc (#3968)
(cherry picked from commit f1fb476f6f)
2024-10-30 14:27:25 -04:00
Peter Park
e1fa36d2b8 add vllm performance validation doc (#3964)
(cherry picked from commit 35b16c1293)
2024-10-30 13:27:29 -04:00
Sam Wu
4d5144d252 Merge pull request #3922 from peterjunpark/docs/6.2.1
docs: Remove invalid amd_iommu=on parameter
2024-10-18 09:40:58 -06:00
Kent Russell
9418f6ab11 docs: Remove invalid amd_iommu=on parameter
Per kernel-parameters.txt, there is no "on" option for amd_iommu. While
intel_iommu has it, amd_iommu is automatically on unless specified
otherwise. For more info, see these 2 links:

https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt
75aa74d52f/drivers/iommu/amd/init.c (L3481)

Signed-off-by: Kent Russell <kent.russell@amd.com>
(cherry picked from commit 74333b667d)
2024-10-18 11:22:02 -04:00
Peter Park
3a48fc142e Merge pull request #3881 from peterjunpark/docs/6.2.1
docs/6.2.1: RN - update documentation changes intro
2024-10-10 13:03:32 -04:00
Peter Park
6cf822649f fix documentation updates section intro 2024-10-09 15:49:33 -04:00
Peter Park
7dfeb73f0a Merge pull request #3804 from peterjunpark/docs/6.2.1
Docs/6.2.1: Add Ubuntu 24.04.1 (#3801)
2024-09-24 14:04:08 -04:00
Peter Park
2b73c2f752 Docs: Add Ubuntu 24.04.1 (#3801)
* add ubuntu 24.04.1

* add 24.04.1 to bottom os section

* fix heading and template

* Update compatibility-matrix.rst for OpenMP version

* Update compatibility-matrix-historical-6.0.csv for OpenMP version

* rm ubuntu 24.04.1 from 6.2.0

* Update docs/compatibility/compatibility-matrix.rst

Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>

* rm duplicate ubuntu in historical

---------

Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>
2024-09-24 13:35:19 -04:00
spolifroni-amd
d8980bf1ac Merge pull request #3795 from ROCm/develop
Merge branch 'develop' into docs/6.2.1
2024-09-23 16:46:50 -04:00
Peter Park
34ab07c082 Merge pull request #3783 from ROCm/develop
Merge branch 'develop' into docs/6.2.1
2024-09-21 10:38:28 -04:00
Young Hui - AMD
7b25fe66fe update llvm version for 6.2.1 (#3779) 2024-09-20 22:25:48 -04:00
Sam Wu
fe22f92e10 Merge pull request #3776 from peterjunpark/docs621
docs/6.2.1: fix wording in RCCL known issue
2024-09-20 16:08:23 -08:00
Peter Park
a986c67ae7 Revert "add MAD page"
This reverts commit 8055f6cfa7.
2024-09-20 20:06:28 -04:00
Peter Park
f468a71498 fix wording in RCCL known issue 2024-09-20 20:05:10 -04:00
Peter Park
8055f6cfa7 add MAD page 2024-09-20 20:05:10 -04:00
Sam Wu
8670cb5da6 Merge pull request #3774 from samjwu/621doc
Update 6.2.1 docs
2024-09-20 15:50:38 -08:00
Sam Wu
770dc8e353 Merge pull request #3765 from samjwu/update621doc
Update 6.2.1 docs
2024-09-20 13:46:17 -08:00
Sam Wu
792c3d952f Merge remote-tracking branch 'upstream/docs/6.2.1' into update621doc 2024-09-20 15:41:24 -06:00
Sam Wu
e0e3b3d129 Merge pull request #3759 from amd-jnovotny/fix-radeon-link-docs621
Cherry-pick to docs/6.2.1: Fix Radeon link and point at R6.1.3 as absolute link (#3757)
2024-09-19 12:29:25 -08:00
Jeffrey Novotny
c81c048ea3 Fix Radeon link and point at R6.1.3 as absolute link (#3757) 2024-09-19 16:23:30 -04:00
Jeffrey Novotny
d25c00f854 Merge pull request #3737 from amd-jnovotny/align-rocrand-api-change-docs621
Cherry-pick to docs/6.2.1: Update link to rocRAND data type support
2024-09-17 14:56:46 -04:00
Jeffrey Novotny
aa9154d51c Update link to rocRAND data type support 2024-09-17 14:27:48 -04:00
Sam Wu
74cf5a9e7e Merge pull request #3705 from ROCm/amd/pbhandar/docs/6.2.1
Sync develop into docs/6.2.1
2024-09-11 12:32:41 -08:00
Parag Bhandari
34946b9094 Merge branch 'develop' into docs/6.2.1 2024-09-11 16:26:32 -04:00
Sam Wu
3b0aaa8593 Merge pull request #3693 from amd-jnovotny/vllm-docker-guide-links-docs621
Cherry pick to docs/6.2.1: Add introduction and links to the new guide to the vLLM optimized Doc…
2024-09-09 11:18:02 -06:00
Jeffrey Novotny
411ae05233 Add introduction and links to the new guide to the vLLM optimized Doc… (#3637)
* Add introduction and links to the new guide to the vLLM optimized Docker image on AMD Infinity Hub

* Update target link for the Docker vLLM guide

* Change target URL

* Change link target URL again
2024-09-09 13:05:19 -04:00
Sam Wu
22eba2352f Merge pull request #3691 from amd-jnovotny/fbgemm-init-docs621
Cherry pick to docs/6.2.1: Add FBGEMM/FBGEMM_GPU to the Model acceleration libraries page (#3659)
2024-09-09 10:02:41 -06:00
Jeffrey Novotny
51ce1076ac Add FBGEMM/FBGEMM_GPU to the Model acceleration libraries page (#3659)
* Add FBGEMM/FBGEMM_GPU to the Model acceleration libraries page

* Add words to wordlist and fix a typo

* Add new sections for Docker and testing

* Incorporate comments from the external review

* Some minor edits and clarifications

* Incorporate further review coments and fix test section

* Add comment to test section

* Change git clone command for FBGEMM repo

* Change Docker command

* Changes from internal review

* Fix linting issue
2024-09-09 11:47:49 -04:00
Peter Park
893001fae2 Merge pull request #3669 from ROCm/develop
Merge develop into docs/6.2.1
2024-09-03 12:35:48 -04:00
429 changed files with 24655 additions and 35914 deletions

View File

@@ -23,40 +23,8 @@ ROCm-CI Azure DevOps Pipelines contains markup language files that orchestrate b
- [YAML schema](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines&viewFallbackFrom=azure-devops) - [YAML schema](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines&viewFallbackFrom=azure-devops)
- [Azure Pipelines Task Index](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/?view=azure-pipelines) - [Azure Pipelines Task Index](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/?view=azure-pipelines)
## VMSS Setup
The Azure VMSS used for build jobs have docker installed during provisioning through the Azure Portal's settings.
Select the VMSS, then go to Settings, Operating system.
In the Custom Data section, select to Modify Custom Data. Enter the code block below and Apply.
```bash
#cloud-config
bootcmd:
- mkdir -p /etc/systemd/system/walinuxagent.service.d
- echo "[Unit]\nAfter=cloud-final.service" > /etc/systemd/system/walinuxagent.service.d/override.conf
- sed "s/After=multi-user.target//g" /lib/systemd/system/cloud-final.service > /etc/systemd/system/cloud-final.service
- systemctl daemon-reload
apt:
sources:
docker.list:
source: deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
packages:
- docker-ce
- docker-ce-cli
groups:
- docker
runcmd:
- usermod -aG docker $USER
- systemctl restart docker
- systemctl enable docker
```
## Disclaimer ## Disclaimer
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard versionchanges, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated.AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.THIS INFORMATION IS PROVIDED AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard versionchanges, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated.AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.THIS INFORMATION IS PROVIDED AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.
© 2024 Advanced Micro Devices, Inc. All Rights Reserved.

View File

@@ -8,17 +8,17 @@ resources:
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp name: ROCm/aomp
ref: aomp-dev ref: amd-staging
- repository: aomp-extras_repo - repository: aomp-extras_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp-extras name: ROCm/aomp-extras
ref: aomp-dev ref: amd-staging
- repository: flang_repo - repository: flang_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/flang name: ROCm/flang
ref: aomp-dev ref: amd-staging
- repository: llvm-project_repo - repository: llvm-project_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
@@ -27,10 +27,7 @@ resources:
pipelines: pipelines:
- pipeline: rocr-runtime_pipeline - pipeline: rocr-runtime_pipeline
source: \ROCR-Runtime source: \ROCR-Runtime
trigger: trigger: true
branches:
include:
- amd-staging
# this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime # this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime
trigger: none trigger: none

View File

@@ -1,33 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml@pipelines_repo
parameters:
- name: pipelinesRepoRef
type: string
default: refs/heads/develop
- name: librariesRepoRef
type: string
default: refs/heads/develop
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
ref: ${{ parameters.pipelinesRepoRef }}
- repository: libraries_repo
type: github
endpoint: ROCm
name: ROCm/rocm-libraries
ref: ${{ parameters.librariesRepoRef }}
trigger: none
pr: none
jobs:
- template: /.azuredevops/ci-builds/mathlibs.yml@pipelines_repo
parameters:
checkoutRepo: libraries_repo
buildDependsOn: false

View File

@@ -1,38 +0,0 @@
# entrypoint for kicking off a unified build of the mathlibs
# this template is designed to be called by another pipeline (llvm, clr, etc.)
# `buildDependsOn` will need to be set when calling this template
# passes a `unifiedBuild` param to downstream pipelines, which will prevent duplicate jobs
# logic needs to be added in individual mathlib pipelines for handling `unifiedBuild`
parameters:
- name: checkoutRepo
type: string
default: monorepo
- name: buildDependsOn
type: object
default: false
- name: downstreamComponentMatrix
type: object
default:
- rocRAND:
name: rocRAND
sparseCheckoutDir: projects/rocrand
- rocPRIM:
name: rocPRIM
sparseCheckoutDir: projects/rocprim
- hipBLAS-common:
name: hipBLAS-common
sparseCheckoutDir: projects/hipblas-common
# - composable_kernel:
# name: composable_kernel
# sparseCheckoutDir: projects/composablekernel
jobs:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ parameters.buildDependsOn }}
triggerDownstreamJobs: true
unifiedBuild: true

View File

@@ -1,62 +1,37 @@
parameters: parameters:
- name: componentName
type: string
default: AMDMIGraphX
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
# - name: sparseCheckoutDir
# type: string
# default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git - git
- libdnnl-dev - cmake
- libdrm-dev
- libmsgpack-dev
- libnuma-dev
- libprotobuf-dev
- libsqlite3-dev
- libtbb-dev
- ninja-build - ninja-build
- nlohmann-json3-dev - libdrm-dev
- protobuf-compiler - libnuma-dev
- python3-pip - python3-pip
- python3-pybind11
- python3-venv - python3-venv
- libtbb-dev
- nlohmann-json3-dev
- libmsgpack-dev
- libsqlite3-dev
- libprotobuf-dev
- protobuf-compiler
- python3-pybind11
- name: pipModules - name: pipModules
type: object type: object
default: default:
- https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz - https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
- "onnx>=1.14.1" - onnx==1.14.1
- "numpy>=1.21.6" - numpy==1.21.6
- "typing>=3.7.4" - typing==3.7.4
- "pytest>=6.0.1" - pytest==6.0.1
- "packaging>=23.0" - packaging==23.0
- "protobuf>=3.20.2" - protobuf==3.20.2
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -75,54 +50,21 @@ parameters:
- rocminfo - rocminfo
- rocMLIR - rocMLIR
- rocprofiler-register - rocprofiler-register
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- aomp-extras
- clr
- composable_kernel
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocm-cmake
- ROCR-Runtime
- rocBLAS
- rocminfo
- rocMLIR
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: AMDMIGraphX
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -133,108 +75,37 @@ jobs:
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
# half version should be fixed to 5.6.0 # half version should be fixed to 5.6.0
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
buildType: specific
definitionId: ${{ variables.HALF560_PIPELINE_ID }}
buildId: ${{ variables.HALF560_BUILD_ID }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencySource: fixed
fixedComponentName: half
fixedPipelineIdentifier: ${{ variables.HALF560_PIPELINE_ID }}
skipLibraryLinking: true
skipLlvmSymlink: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }} -DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include -DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DMIGRAPHX_USE_COMPOSABLEKERNEL=OFF
-DBUILD_TESTING=ON -DBUILD_TESTING=ON
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml # REFERENCE: https://github.com/ROCm/composable_kernel/issues/782
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: MIGRAPHX_TRACE_BENCHMARKING
value: 1
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
# half version should be fixed to 5.6.0
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
buildType: specific
definitionId: ${{ variables.HALF560_PIPELINE_ID }}
buildId: ${{ variables.HALF560_BUILD_ID }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: CMake@1
displayName: MIGraphXTest CMake Flags
inputs:
cmakeArgs: >-
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-DAMDGPU_TARGETS=${{ job.target }}
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DBUILD_TESTING=ON
-DMIGRAPHX_ENABLE_C_API_TEST=ON
..
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testExecutable: make
testParameters: -j$(nproc) check
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
extraEnvVars:
- MIGRAPHX_TRACE_BENCHMARKING:::1

View File

@@ -1,45 +1,18 @@
parameters: parameters:
- name: componentName
type: string
default: hip_clr_combined
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- libnuma-dev - libnuma-dev
- mesa-common-dev - mesa-common-dev
- ninja-build
- ocl-icd-libopencl1 - ocl-icd-libopencl1
- ocl-icd-opencl-dev - ocl-icd-opencl-dev
- opencl-headers - opencl-headers
- zlib1g-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
@@ -54,33 +27,20 @@ parameters:
type: object type: object
default: default:
- llvm-project - llvm-project
- ROCR-Runtime
- name: jobMatrix # hip and clr are tightly-coupled
type: object # run this same template for both repos
default: # any changes for clr should just trigger HIP pipeline
buildJobs: # similarly for hipother repo, for Nvidia backend
- { os: ubuntu2204, packageManager: apt, platform: amd }
- { os: ubuntu2204, packageManager: apt, platform: nvidia }
- { os: almalinux8, packageManager: dnf, platform: amd }
- { os: almalinux8, packageManager: dnf, platform: nvidia }
# HIP with AMD backend
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hip_clr_combined_amd
- job: ${{ parameters.componentName }}_${{ job.os }}_${{ job.platform }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
@@ -88,47 +48,94 @@ jobs:
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }} # checkout triggering repo (either HIP or clr)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
# full checkout of rocm-systems superrepo, we need clr, hip, and hipother
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
# sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} # if this is triggered by HIP repo, matching repo is clr
# if this is triggered by clr repo, matching repo is HIP
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: matching_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: hipother_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
os: ${{ job.os }}
${{ if eq(job.platform, 'amd') }}:
dependencyList: ${{ parameters.rocmDependenciesAMD }} dependencyList: ${{ parameters.rocmDependenciesAMD }}
${{ elseif eq(job.platform, 'nvidia') }}: # CI case: download latest default branch build
dependencyList: ${{ parameters.rocmDependenciesNvidia }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# compile clr
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: clr componentName: clr
cmakeBuildDir: $(Agent.BuildDirectory)/s/projects/clr/build cmakeBuildDir: 'clr/build'
cmakeSourceDir: $(Agent.BuildDirectory)/s/projects/clr
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DHIP_COMMON_DIR=$(Build.SourcesDirectory)/HIP
-DHIP_PLATFORM=amd
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin -DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin
-DHIP_COMMON_DIR=$(Agent.BuildDirectory)/s/projects/hip
-DHIPNV_DIR=$(Agent.BuildDirectory)/s/projects/hipother/hipnv
-DHIP_PLATFORM=${{ job.platform }}
-DCLR_BUILD_HIP=ON -DCLR_BUILD_HIP=ON
-DCLR_BUILD_OCL=ON -DCLR_BUILD_OCL=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
artifactName: ${{ job.platform }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
artifactName: ${{ job.platform }} artifactName: amd
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml # HIP with Nvidia backend
- job: hip_clr_combined_nvidia
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
# checkout triggering repo (either HIP or clr)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# if this is triggered by HIP repo, matching repo is clr
# if this is triggered by clr repo, matching repo is HIP
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: matching_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: hipother_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependenciesNvidia }}
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- script: 'ls -1R $(Agent.BuildDirectory)/rocm'
displayName: 'Artifact listing'
# compile clr
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: clr
cmakeBuildDir: 'clr/build'
extraBuildFlags: >-
-DHIP_COMMON_DIR=$(Build.SourcesDirectory)/HIP
-DHIP_PLATFORM=nvidia
-DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin
-DCLR_BUILD_HIP=ON
-DCLR_BUILD_OCL=OFF
-DHIPNV_DIR=$(Build.SourcesDirectory)/hipother/hipnv
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
artifactName: nvidia

View File

@@ -1,63 +1,34 @@
parameters: parameters:
- name: componentName
type: string
default: HIPIFY
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cuda-toolkit-12-9 - cmake
- libcudnn9-dev-cuda-12
- libnuma-dev
- mesa-common-dev
- ninja-build - ninja-build
- python-is-python3 - libnuma-dev
- python3-dev - python3-dev
- python3-pip - python3-pip
- name: pipModules - python-is-python3
type: object - mesa-common-dev
default: - ccache
- lit - cuda-toolkit
- name: rocmDependencies - cudnn
type: object
default:
- llvm-project
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: HIPIFY
- job: ${{ parameters.componentName }}_build_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: - name: UPSTREAM_LLVM_GIT_URL
${{ if eq(job.os, 'ubuntu2404') }}: value: https://github.com/llvm/llvm-project.git
name: rocm-ci_medium_build_pool_2404 - name: UPSTREAM_LLVM_TAG
${{ else }}: value: llvmorg-18.1.2
name: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
steps: steps:
@@ -65,91 +36,72 @@ jobs:
displayName: 'Register CUDA packages' displayName: 'Register CUDA packages'
inputs: inputs:
targetType: inline targetType: inline
${{ if eq(job.os, 'ubuntu2204') }}:
script: | script: |
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb sudo mkdir --parents --mode=0755 /etc/apt/keyrings
sudo dpkg -i cuda-keyring_1.1-1_all.deb wget -q -O- https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-archive-keyring.gpg | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/cuda-archive-keyring.gpg > /dev/null
sudo rm -f cuda-keyring_1.1-1_all.deb echo "deb [signed-by=/etc/apt/trusted.gpg.d/cuda-archive-keyring.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /" | sudo tee /etc/apt/sources.list.d/cuda-ubuntu2204-x86_64.list
sudo apt update sudo apt update
${{ if eq(job.os, 'almalinux8') }}:
script: |
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- task: Bash@3 - task: Bash@3
displayName: Add lit to PATH displayName: git clone upstream llvm-project
inputs: inputs:
targetType: inline targetType: inline
script: | script: git clone $(UPSTREAM_LLVM_GIT_URL) --depth=1 --branch $(UPSTREAM_LLVM_TAG) --recurse-submodules
site_packages=$(python3 -m site --user-base)/bin workingDirectory: $(Pipeline.Workspace)
sudo ln -sf $site_packages/bin/lit $(Pipeline.Workspace)/llvm-lit
echo "##vso[task.prependpath]$site_packages"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - script: |
parameters: mkdir -p $(CCACHE_DIR)
checkoutRef: ${{ parameters.checkoutRef }} echo "##vso[task.prependpath]/usr/lib/ccache:/usr/local/cuda/bin"
dependencyList: ${{ parameters.rocmDependencies }} displayName: Update path for cuda and ccache
os: ${{ job.os }} - task: Cache@2
aggregatePipeline: ${{ parameters.aggregatePipeline }} displayName: Ccache caching
# cutensor is not available from apt or dnf
- task: Bash@3
displayName: 'Download and install cutensor'
inputs: inputs:
targetType: inline key: HIPIFY | $(Agent.OS) | "$(UPSTREAM_LLVM_TAG)"
script: | path: $(CCACHE_DIR)
wget -q --show-progress https://developer.download.nvidia.com/compute/cutensor/redist/libcutensor/linux-x86_64/libcutensor-linux-x86_64-2.2.0.0-archive.tar.xz restoreKeys: HIPIFY | $(Agent.OS)
tar -xvJf libcutensor-linux-x86_64-*.tar.xz
mkdir -p $(Pipeline.Workspace)/cutensor
cp -r libcutensor-linux-x86_64-*/* $(Pipeline.Workspace)/cutensor/
- task: Bash@3
displayName: 'List downloaded CUDA files'
inputs:
targetType: inline
script: ls -la1R /usr/local/cuda-12.9
# script: cp $(Pipeline.Workspace)/llvm-project/llvm/build/bin/FileCheck $(Pipeline.Workspace)/llvm/bin
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: upstream-llvm
os: ${{ job.os }} cmakeBuildDir: $(Pipeline.Workspace)/llvm-project/llvm/build
consolidateBuildAndInstall: true installDir: $(Pipeline.Workspace)/llvm
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DLLVM_ENABLE_PROJECTS=clang
-DLLVM_INCLUDE_TESTS=OFF
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-GNinja
- task: Bash@3
displayName: python install lit
inputs:
targetType: inline
script: sudo python3 $(Pipeline.Workspace)/llvm-project/llvm/utils/lit/setup.py install
- task: Bash@3
displayName: install FileCheck
inputs:
targetType: inline
script: cp $(Pipeline.Workspace)/llvm-project/llvm/build/bin/FileCheck $(Pipeline.Workspace)/llvm/bin
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: HIPIFY
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;/usr/local/cuda/targets/x86_64-linux/lib
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DHIPIFY_CLANG_TESTS=ON -DHIPIFY_CLANG_TESTS=ON
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.9 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda/targets/x86_64-linux
-DCUDA_DNN_ROOT_DIR=/usr/local/cuda-12.9 -DCUDA_DNN_ROOT_DIR=/usr/local/cuda/targets/x86_64-linux
-DCUDA_CUB_ROOT_DIR=/usr/local/cuda-12.9/targets/x86_64-linux/include/cub -DCMAKE_PREFIX_PATH=$(Pipeline.Workspace)/llvm;/usr/local/cuda/targets/x86_64-linux/lib
-DCUDA_TENSOR_ROOT_DIR=$(Pipeline.Workspace)/cutensor/ -DLLVM_EXTERNAL_LIT=$(Pipeline.Workspace)/llvm-project/llvm/build/bin/llvm-lit
multithreadFlag: -- -j32 multithreadFlag: -- -j32
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
os: ${{ job.os }} componentName: HIPIFY
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml testDir: $(Build.SourcesDirectory)/build
# parameters: testExecutable: make
# componentName: HIPIFY testParameters: test-hipify
# testDir: $(Build.SourcesDirectory)/build testPublishResults: false
# testExecutable: make
# testParameters: -j 32 test-hipify
# testPublishResults: false
# os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
registerCUDAPackages: true
extraCopyDirectories:
- llvm-project

View File

@@ -1,61 +1,33 @@
parameters: parameters:
- name: componentName
type: string
default: MIOpen
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- jq
- libdrm-dev
- libmsgpack-dev
- libsqlite3-dev
- libstdc++-12-dev
- ninja-build
- python3-pip
- python3-venv
- software-properties-common - software-properties-common
- zip - python3-pip
- name: pipModules - cmake
type: object - ninja-build
default: - libsqlite3-dev
- cget - libbz2-dev
- nlohmann-json3-dev
- libgtest-dev
- libdrm-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- half - rocMLIR
- rocRAND - rocRAND
- rocBLAS - rocBLAS
- hipBLAS - hipBLAS
- hipBLASLt - hipBLASLt
- hipBLAS-common - hipBLAS-common
- half
- composable_kernel
- rocm-cmake - rocm-cmake
- llvm-project - llvm-project
- ROCR-Runtime - ROCR-Runtime
@@ -63,228 +35,62 @@ parameters:
- clr - clr
- rocminfo - rocminfo
- roctracer - roctracer
- name: rocmTestDependencies
type: object
default:
- clr
- half
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- rocRAND
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
- name: downstreamComponentMatrix
type: object
default:
- MIVisionX:
name: MIVisionX
checkoutRepo: mivisionx_repo
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- MIOpen_build
- AMDMIGraphX:
name: AMDMIGraphX
checkoutRepo: amdmigraphx_repo
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- MIOpen_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: MIOpen
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} # The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml # Upgrade boost from apt by following https://launchpad.net/~mhier/+archive/ubuntu/libboost-latest
- task: Bash@3
displayName: 'update boost version'
inputs:
targetType: inline
script: sudo add-apt-repository ppa:mhier/libboost-latest -y
- task: Bash@3
displayName: 'install boost'
inputs:
targetType: inline
script: sudo apt-get --yes install libboost1.83-dev libboost-system1.83-dev libboost-filesystem1.83-dev
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/miopen-get-ck-build.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- task: Bash@3 # manual build case: triggered by ROCm/ROCm repo
displayName: Build and install other dependencies ${{ elseif ne(parameters.checkoutRef, '') }}:
retryCountOnTaskFailure: 3 dependencySource: tag-builds
inputs:
targetType: inline
workingDirectory: $(Agent.BuildDirectory)/s
script: |
sed -i '/composable_kernel/d' requirements.txt
mkdir -p $(Agent.BuildDirectory)/miopen-deps
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
cmake -P install_deps.cmake --prefix $(Agent.BuildDirectory)/miopen-deps --generator Ninja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DMIOPEN_BACKEND=HIP -DMIOPEN_BACKEND=HIP
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/miopen-deps -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DGPU_TARGETS=${{ job.target }}
-DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF -DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF
-DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF -DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DBUILD_TESTING=ON -DBUILD_TESTING=ON
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- miopen-deps
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
timeoutInMinutes: 180
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/miopen-get-ck-build.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Build and install other dependencies
retryCountOnTaskFailure: 3
inputs:
targetType: inline
workingDirectory: $(Agent.BuildDirectory)/s
script: |
sed -i '/composable_kernel/d' requirements.txt
mkdir -p $(Agent.BuildDirectory)/miopen-deps
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
cmake -P install_deps.cmake --prefix $(Agent.BuildDirectory)/miopen-deps
- task: CMake@1
displayName: 'MIOpen Test CMake Flags'
inputs:
cmakeArgs: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/s/bin;$(Agent.BuildDirectory)/miopen-deps
-DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DMIOPEN_BACKEND=HIP
-DMIOPEN_TEST_FLAGS=" --disable-verification-cache"
-DCMAKE_BUILD_TYPE=release
-DBUILD_DEV=OFF
-DMIOPEN_USE_MLIR=ON
-DMIOPEN_GPU_SYNC=OFF
$(Agent.BuildDirectory)/s
- task: Bash@3
displayName: 'MIOpen Test Build'
inputs:
targetType: inline
workingDirectory: build
script: |
cmake --build . --target tests -- -j$(nproc)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml --exclude-regex "test_rnn_seq_api|GPU_Conv2dTuningAsm_FP32|GPU_Conv2dTuningAsmBwdWrw_FP32"'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- miopen-deps
# - ${{ if parameters.triggerDownstreamJobs }}:
# - ${{ each component in parameters.downstreamComponentMatrix }}:
# - ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
# - template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
# parameters:
# checkoutRepo: ${{ component.checkoutRepo }}
# # sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
# buildDependsOn: ${{ component.buildDependsOn }}
# downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
# triggerDownstreamJobs: true
# unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,85 +1,66 @@
parameters: parameters:
- name: componentName
type: string
default: MIVisionX
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
# - name: sparseCheckoutDir
# type: string
# default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build - ninja-build
- git
- wget - wget
- unzip - unzip
- libstdc++-12-dev
- pkg-config - pkg-config
- protobuf-compiler
- libprotoc-dev
- ffmpeg - ffmpeg
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libswscale-dev - libswscale-dev
- build-essential
- libgtk2.0-dev
- libavcodec-dev
- libavformat-dev
- libswscale-dev
- libtbb2
- libtbb-dev
- libjpeg-dev
- libpng-dev
- libtiff-dev
- libdc1394-dev
- libgmp-dev - libgmp-dev
- libomp-dev
- libopencv-dev - libopencv-dev
- protobuf-compiler
- libprotoc-dev
- python3-pip
- name: pipModules - name: pipModules
type: object type: object
default: default:
- future==1.0.0 - future==0.18.2
- pytz==2022.1 - pytz==2022.1
- numpy==1.23 - numpy==1.21
- google==3.0.0 - google==3.0.0
- protobuf==3.12.4 - protobuf==3.12.4
- onnx==1.12.0 - onnx==1.12.0
- nnef==1.0.7
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- AMDMIGraphX
- clr
- half
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocBLAS
- rocDecode
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - half
- rocBLAS
- MIOpen
- AMDMIGraphX
- rpp - rpp
- rocDecode
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
- aomp
- clr - clr
- half - half
- hipBLAS-common - hipBLAS-common
@@ -92,33 +73,21 @@ parameters:
- roctracer - roctracer
- rpp - rpp
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: MIVisionX
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -130,12 +99,14 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -143,36 +114,24 @@ jobs:
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_DEP_ROCMCORE=ON -DROCM_DEP_ROCMCORE=ON
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: MIVisionX_testing
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }} dependsOn: MIVisionX
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: LD_LIBRARY_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm/lib:$(Agent.BuildDirectory)/rocm/include/mivisionx/VX
pool: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -180,34 +139,30 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- task: Bash@3 ${{ elseif ne(parameters.checkoutRef, '') }}:
displayName: Build MIVisionX tests dependencySource: tag-builds
inputs: # anything in /opt may be persistent across runs
targetType: inline # so we need to remove the symlink if it already exists
script: | - script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral sudo rm -f /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
mkdir mivisionx-tests mkdir mivisionx-tests
cd mivisionx-tests cd mivisionx-tests
cmake $(Agent.BuildDirectory)/rocm/share/mivisionx/test cmake /opt/rocm/share/mivisionx/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: MIVisionX
testDir: 'mivisionx-tests' testDir: 'mivisionx-tests'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -1,220 +1,52 @@
parameters: parameters:
- name: componentName
type: string
default: ROCR-Runtime
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - libelf-dev
- g++ - g++
- libdrm-dev - libdrm-dev
- libelf-dev
- libnuma-dev - libnuma-dev
- ninja-build
- pkg-config
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- llvm-project - llvm-project
- rocprofiler-register - rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- amdsmi
- llvm-project
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCR_Runtime
- job: ROCR_Runtime_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_SHARED_LIBS=ON -DBUILD_SHARED_LIBS=ON
-DCMAKE_BUILD_TYPE=Release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ROCR_Runtime_test_${{ job.os }}_${{ job.target }}
dependsOn: ROCR_Runtime_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- task: Bash@3
displayName: Build kfdtest
inputs:
targetType: 'inline'
workingDirectory: $(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest
script: |
if [ -e /opt/rh/gcc-toolset-14/enable ]; then
source /opt/rh/gcc-toolset-14/enable
fi
mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm ..
make
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: kfdtest
testExecutable: BIN_DIR=$(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest/build ./run_kfdtest.sh
testParameters: '-p core --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest/scripts
os: ${{ job.os }}
- task: Bash@3
displayName: Build rocrtst
inputs:
targetType: 'inline'
workingDirectory: $(Agent.BuildDirectory)/s/rocrtst/suites/test_common
script: |
echo $(Agent.BuildDirectory)/s/rocrtst/thirdparty/lib | sudo tee -a /etc/ld.so.conf.d/rocm-ci.conf
sudo cat /etc/ld.so.conf.d/rocm-ci.conf
sudo ldconfig -v
ldconfig -p
if [ -e /opt/rh/gcc-toolset-14/enable ]; then
source /opt/rh/gcc-toolset-14/enable
fi
BASE_CLANG_DIR=$(Agent.BuildDirectory)/rocm/llvm/lib/clang
export NEWEST_CLANG_VER=$(ls -1 $BASE_CLANG_DIR | sort -V | tail -n 1)
mkdir build && cd build
cmake .. \
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm \
-DTARGET_DEVICES=${{ job.target }} \
-DROCM_DIR=$(Agent.BuildDirectory)/rocm \
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin \
-DOPENCL_INC_DIR=$BASE_CLANG_DIR/$NEWEST_CLANG_VER/include
make
make rocrtst_kernels
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocrtst
testExecutable: ./rocrtst64
testParameters: '--gtest_filter="-rocrtstNeg.Memory_Negative_Tests:rocrtstFunc.Memory_Max_Mem" --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Agent.BuildDirectory)/s//rocrtst/suites/test_common/build/${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
# docker image will be missing libhwloc5

View File

@@ -16,7 +16,8 @@ jobs:
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.LOW_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
@@ -28,5 +29,4 @@ jobs:
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -23,57 +18,36 @@ parameters:
- rocminfo - rocminfo
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCdbgapi
- job: ROCdbgapi_build_${{ job.os }}
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -1,24 +1,15 @@
parameters: parameters:
- name: componentName
type: string
default: ROCgdb
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
# reference: https://github.com/ROCm/ROCgdb/blob/amd-staging/README-ROCM.md # reference: https://github.com/ROCm/ROCgdb/blob/amd-staging/README-ROCM.md
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- bison - bison
- cmake
- dejagnu - dejagnu
- flex - flex
- libbabeltrace-dev - libbabeltrace-dev
@@ -27,9 +18,6 @@ parameters:
- liblzma-dev - liblzma-dev
- libmpfr-dev - libmpfr-dev
- ncurses-dev - ncurses-dev
- pkg-config
- python3-dev
- python3-pip
- texinfo - texinfo
- zlib1g-dev - zlib1g-dev
- name: rocmDependencies - name: rocmDependencies
@@ -42,56 +30,46 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCgdb
- job: ${{ parameters.componentName }}_build_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: PKG_CONFIG_PATH - name: PKG_CONFIG_PATH
value: $(Agent.BuildDirectory)/rocm/share/pkgconfig value: $(Agent.BuildDirectory)/rocm/share/pkgconfig
pool: pool: $(JOB_TEST_POOL)
${{ if eq(job.os, 'ubuntu2404') }}:
name: rocm-ci_medium_build_pool_2404
${{ else }}:
name: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
parameters: parameters:
os: ${{ job.os }}
configureFlags: >- configureFlags: >-
--program-prefix=roc --program-prefix=roc
--enable-64-bit-bfd --enable-64-bit-bfd
@@ -113,92 +91,21 @@ jobs:
--with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm --with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm
LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib" LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib"
makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib' makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PKG_CONFIG_PATH
value: $(Agent.BuildDirectory)/rocm/share/pkgconfig
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
parameters:
os: ${{ job.os }}
configureFlags: >-
--program-prefix=roc
--enable-64-bit-bfd
--enable-targets="x86_64-linux-gnu,amdgcn-amd-amdhsa"
--disable-ld
--disable-gas
--disable-gdbserver
--disable-sim
--enable-tui
--disable-gdbtk
--disable-shared
--disable-gprofng
--with-expat
--with-system-zlib
--without-guile
--with-babeltrace
--with-lzma
--with-python=python3
--with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm
LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib"
makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib'
- task: Bash@3 - task: Bash@3
displayName: Setup test environment displayName: Setup test environment
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
echo "##vso[task.prependpath]/opt/rocm/bin" echo "##vso[task.prependpath]/opt/rocm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3 - task: Bash@3
displayName: check-gdb displayName: check-gdb
continueOnError: true continueOnError: true
inputs: inputs:
targetType: inline targetType: inline
script: | script: make check-gdb TESTS=gdb.rocm/simple.exp
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
make check-gdb TESTS=gdb.rocm/simple.exp
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3 - task: Bash@3
displayName: print gdb log displayName: print gdb log
@@ -206,10 +113,3 @@ jobs:
targetType: inline targetType: inline
script: find -name gdb.log -exec cat {} \; script: find -name gdb.log -exec cat {} \;
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
gpuTarget: ${{ job.target }}
extraEnvVars:
- PKG_CONFIG_PATH:::/home/user/workspace/rocm/share/pkgconfig

View File

@@ -5,29 +5,19 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build - ninja-build
- libdrm-dev
- libyaml-cpp-dev - libyaml-cpp-dev
- libpci-dev - libpci-dev
- libpci3 - libpci3
- libgst-dev
- libgtest-dev - libgtest-dev
- git - git
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- amdsmi
- aomp
- clr - clr
- hipBLAS-common - hipBLAS-common
- hipBLASLt - hipBLASLt
@@ -43,8 +33,6 @@ parameters:
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
- amdsmi
- aomp
- clr - clr
- hipBLAS-common - hipBLAS-common
- hipBLASLt - hipBLASLt
@@ -52,31 +40,12 @@ parameters:
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocm_smi_lib - rocm_smi_lib
- rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
confDir: MI300X
- gfx90a:
target: gfx90a
confDir: MI210
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCmValidationSuite
- job: ROCmValidationSuite_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -86,9 +55,16 @@ jobs:
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm
- name: HIP_INC_DIR - name: HIP_INC_DIR
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -99,50 +75,40 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/llvm/include
-DCPACK_PACKAGING_INSTALL_PREFIX=$(Build.BinariesDirectory) -DCPACK_PACKAGING_INSTALL_PREFIX=$(Build.BinariesDirectory)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
# - ROCM_PATH:::/home/user/workspace/rocm
# - HIP_INC_DIR:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: ROCmValidationSuite_testing
- job: ROCmValidationSuite_test_${{ job.target }} dependsOn: ROCmValidationSuite
dependsOn: ROCmValidationSuite_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -150,23 +116,25 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ROCmValidationSuite componentName: ROCmValidationSuite
testExecutable: $(Agent.BuildDirectory)/rocm/bin/rvs -c $(Agent.BuildDirectory)/rocm/share/rocm-validation-suite/conf/${{ job.confDir }}/gst_single.conf testExecutable: $(Agent.BuildDirectory)/rocm/bin/rvs -c $(Agent.BuildDirectory)/rocm/share/rocm-validation-suite/conf/MI300X/gst_single.conf
testParameters: '' testParameters: ''
testDir: $(Agent.BuildDirectory) testDir: $(Agent.BuildDirectory)
testPublishResults: false testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,221 +0,0 @@
parameters:
- name: componentName
type: string
default: Tensile
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libboost-filesystem-dev
- libboost-program-options-dev
- libmsgpack-dev
- name: pipModules
type: object
default:
- tox
- pytest
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn: ${{ parameters.buildDependsOn[job.target] }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: python3 setup.py bdist_wheel
workingDirectory: $(Agent.BuildDirectory)/s
- task: Bash@3
displayName: Rename wheel file with job OS
inputs:
targetType: inline
workingDirectory: $(Agent.BuildDirectory)/s
script: |
wheelFile=$(find "$(Agent.BuildDirectory)/s/dist" -type f -name "*.whl" | head -n 1)
newWheelFile="$(basename "$wheelFile" .whl)-${{ job.os }}.whl"
mv "$wheelFile" "$(dirname "$wheelFile")/$newWheelFile"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Agent.BuildDirectory)/s/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- task: Bash@3
displayName: Save pipeline artifact file names
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.ArtifactStagingDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: Tensile_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 180
dependsOn: Tensile_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*${{ job.os }}*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- task: Bash@3
displayName: pip install
inputs:
targetType: inline
script: find -name *.whl -exec pip install {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: tox test
inputs:
targetType: inline
script: tox run -v -e ci -- -m pre_checkin
workingDirectory: $(Agent.BuildDirectory)/s
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
pythonEnvVars: true
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
# docker image will not have python site-packages in path, but the env vars will make it easier

View File

@@ -1,142 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libnuma-dev
- ninja-build
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: TransferBench_build_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: TransferBench_test_${{ job.target }}
dependsOn: TransferBench_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: TransferBench All-to-all
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/TransferBench'
testParameters: 'a2a'
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: TransferBench Peer-to-peer
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/TransferBench'
testParameters: 'p2p'
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,110 +5,57 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- libdrm-dev - libdrm-dev
- ninja-build
- pkg-config
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: amdsmi
- job: amdsmi_build_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
vmImage: 'ubuntu-24.04'
${{ else }}:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DBUILD_TESTS=ON -DBUILD_TESTS=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: amdsmi_testing
- job: amdsmi_test_${{ job.os }}_${{ job.target }} dependsOn: amdsmi
dependsOn: amdsmi_build_${{ job.os }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: amdsmi componentName: amdsmi
testDir: '$(Agent.BuildDirectory)' testDir: '$(Agent.BuildDirectory)'
testExecutable: 'sudo ./rocm/share/amd_smi/tests/amdsmitst' testExecutable: './rocm/share/amd_smi/tests/amdsmitst'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -42,9 +37,13 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: aomp-extras componentName: aomp-extras
@@ -55,9 +54,4 @@ jobs:
-DAOMP_VERSION_STRING=$(LATEST_RELEASE_TAG) -DAOMP_VERSION_STRING=$(LATEST_RELEASE_TAG)
-GNinja -GNinja
installDir: $(Build.BinariesDirectory)/llvm installDir: $(Build.BinariesDirectory)/llvm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -1,102 +1,86 @@
parameters: parameters:
- name: componentName
type: string
default: aomp
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
# reference: # reference:
# https://github.com/ROCm/aomp/blob/aomp-dev/docs/SOURCEINSTALL_PREREQUISITE.md # https://github.com/ROCm/aomp/blob/aomp-dev/docs/SOURCEINSTALL_PREREQUISITE.md
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- bison
- ccache
- cmake - cmake
- flex - python3-pip
- gawk
- git
- mesa-common-dev
- ninja-build - ninja-build
- libbabeltrace-dev - pkg-config
- libbison-dev - libpci-dev
- libnuma-dev
- libffi-dev
- git
- libopenmpi-dev
- gawk
- mesa-common-dev
- libtool
- libdrm-amdgpu1 - libdrm-amdgpu1
- libdrm-dev - libdrm-dev
- libdw-dev - libdw-dev
- libffi-dev - libgtest-dev
- libgmp-dev - libsystemd-dev
- liblzma-dev
- libmpfr-dev
- libncurses5-dev
- libnuma-dev
- libopenmpi-dev
- libpci-dev
- libssl-dev - libssl-dev
- libstdc++-12-dev - libstdc++-12-dev
- libsystemd-dev - ccache
- libtool - libgmp-dev
- libmpfr-dev
- texinfo
- libbison-dev
- bison
- flex
- libbabeltrace-dev
- libncurses5-dev
- liblzma-dev
- python3-setuptools
- python3-dev
- libudev-dev - libudev-dev
- parallel - parallel
- pkg-config # Referencing comment snippet below but excluding rocprofiler and roctracer.
- python3-dev # This is to remove need for separate build per gpu target.
- python3-pip # With our selected build flags, compilation and installation work fine without these two.
- python3-setuptools #
- texinfo # snippet from https://github.com/ROCm/aomp/blob/aomp-dev/bin/build_aomp.sh#L131-L134
#
# For ROCM build (AOMP_STANDALONE_BUILD=0) the components roct, rocr,
# libdevice, project, comgr, rocminfo, hipamd, rocdbgapi, rocgdb,
# roctracer, rocprofiler, rocm_smi_lib, and amdsmi should be found
# in ROCM in /opt/rocm. The ROCM build only needs these components:
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- llvm-project - amdsmi
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr - clr
- llvm-project - llvm-project
- ROCdbgapi
- ROCgdb
- rocm-cmake
- rocm-core - rocm-core
- rocminfo - rocminfo
- ROCR-Runtime - rocm_smi_lib
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime
- name: jobMatrix - roctracer
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: aomp
- job: ${{ parameters.componentName }}_build_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'ubuntu2404') }}:
name: rocm-ci_medium_build_pool_2404
${{ else }}:
name: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
# checkout the repos tied to openmp-extras, plus llvm-project # checkout the repos tied to openmp-extras, plus llvm-project
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
@@ -111,125 +95,356 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: llvm-project_repo checkoutRepo: llvm-project_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml dependencySource: staging
parameters: # manual build case: triggered by ROCm/ROCm repo
os: ${{ job.os }} ${{ elseif ne(parameters.checkoutRef, '') }}:
useAmdclang: false dependencySource: tag-builds
componentName: extras # Because clang is not being rebuilt and to separate downloaded ROCm
cmakeBuildDir: '$(Build.SourcesDirectory)/aomp-extras/build' # dependencies from the new artifacts, we use temporary symbolic links
cmakeSourceDir: '$(Build.SourcesDirectory)/aomp-extras' # for the compilation and installation to go through.
installDir: '$(Build.BinariesDirectory)/llvm' - script: |
extraBuildFlags: >- sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm mkdir -p $(Build.BinariesDirectory)/bin
-DCMAKE_BUILD_TYPE=Release ln -s $(Agent.BuildDirectory)/rocm/llvm/bin/clang $(Build.BinariesDirectory)/bin/clang
-DAOMP_STANDALONE_BUILD=0 ln -s $(Agent.BuildDirectory)/rocm/llvm/bin/clang++ $(Build.BinariesDirectory)/bin/clang++
-DAOMP_VERSION_STRING=9.99.99 ln -s $(Agent.BuildDirectory)/rocm/llvm/bin/llvm-config $(Build.BinariesDirectory)/bin/llvm-config
-GNinja ln -s $(Agent.BuildDirectory)/rocm/llvm $(Build.BinariesDirectory)/llvm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml displayName: Extra build environment setup
parameters: # We follow the sequence described in the aomp repo instructions
os: ${{ job.os }} # https://github.com/ROCm/aomp/blob/aomp-dev/docs/SOURCEINSTALL.md
componentName: openmp # But instead of calling build_aomp.sh directly, we have to split up the calls to build each component
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm-project/openmp/build' # in order for the omp.h header file to be both be published as an artifact,
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm-project/openmp' # and be found when building subsequent components.
installDir: '$(Build.BinariesDirectory)/llvm' # Environment variables specified per bash script were determined from looking through the
extraBuildFlags: >- # individual scripts and iteratively testing compilation.
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Build.BinariesDirectory)" # Splitting up the build_aomp.sh call is also easier for debugging within Azure, as the former
-DCMAKE_BUILD_TYPE=Release # method leads to a giant build log compared to separate logs per script call.
-DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ #
-DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang # Components compiled and the order for non-standalone build found at
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ # https://github.com/ROCm/aomp/blob/aomp-dev/bin/build_aomp.sh#L135-L143
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DOPENMP_ENABLE_LIBOMPTARGET=1
-DLIBOMP_COPY_EXPORTS=OFF
-DLIBOMP_OMPD_SUPPORT=ON
-DCMAKE_SKIP_INSTALL_RPATH=TRUE
-DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DLIBOMP_FORTRAN_MODULES_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/flang
-DLIBOMP_MODULES_INSTALL_PATH=$(Build.BinariesDirectory)/llvm/include/flang/
multithreadFlag: -- -j32
- task: Bash@3 - task: Bash@3
displayName: 'ROCm symbolic link' displayName: Build Prereq
inputs: inputs:
targetType: inline filePath: $(Build.SourcesDirectory)/aomp/bin/build_prereq.sh
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm env:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml AOMP_REPOS: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Build extras
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_extras.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_EXTRAS: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install extras
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_extras.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_EXTRAS: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Build openmp
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_openmp.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_LEGACY_OPENMP: 1
AOMP_USE_NINJA: 1
ALTAOMP: $(Agent.BuildDirectory)/rocm/llvm
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_OPENMP: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install openmp
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_openmp.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_LEGACY_OPENMP: 1
AOMP_USE_NINJA: 1
ALTAOMP: $(Agent.BuildDirectory)/rocm/llvm
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_OPENMP: $(Build.BinariesDirectory)
- script: ln -s $(Build.BinariesDirectory)/include/omp.h $(Build.SourcesDirectory)/llvm-project/llvm/include/omp.h
displayName: Link omp header
- task: Bash@3
displayName: Build offload
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_offload.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_LEGACY_OPENMP: 1
AOMP_USE_NINJA: 1
ALTAOMP: $(Agent.BuildDirectory)/rocm/llvm
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_OPENMP: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install offload
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_offload.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_LEGACY_OPENMP: 1
AOMP_USE_NINJA: 1
ALTAOMP: $(Agent.BuildDirectory)/rocm/llvm
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_OPENMP: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Build flang-legacy
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang-legacy.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install flang-legacy
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang-legacy.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Build pgmath
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_pgmath.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install pgmath
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_pgmath.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Build flang
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install flang
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Build flang_runtime
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang_runtime.sh
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Install flang_runtime
inputs:
filePath: $(Build.SourcesDirectory)/aomp/bin/build_flang_runtime.sh
arguments: install
env:
AOMP_REPOS: $(Build.SourcesDirectory)
AOMP_PROJECT_REPO_NAME: llvm-project
AOMP_STANDALONE_BUILD: 0
AOMP_BUILD_SANITIZER: 0
AOMP_BUILD_DEBUG: 0
AOMP_USE_NINJA: 1
INSTALL_PREFIX: $(Build.BinariesDirectory)
CMAKE_INSTALL_PREFIX: $(Build.BinariesDirectory)
LLVM_PROJECT_ROOT: $(Build.SourcesDirectory)/llvm-project
AOMP: $(Build.BinariesDirectory)
AOMP_INSTALL_DIR: $(Build.BinariesDirectory)
INSTALL_FLANG: $(Build.BinariesDirectory)
# Clean up build environment before publish artifact
- script: |
rm $(Build.BinariesDirectory)/bin/clang
rm $(Build.BinariesDirectory)/bin/clang++
rm $(Build.BinariesDirectory)/bin/llvm-config
rm $(Build.BinariesDirectory)/llvm
displayName: Remove temporary symbolic links
# Copy the files to artifact staging temporarily to clean up binaries directory
# and then copy files back to llvm subdirectory in the cleaned up binaries directory
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters: parameters:
os: ${{ job.os }} sourceDir: $(Build.BinariesDirectory)
componentName: offload targetDir: $(Build.ArtifactStagingDirectory)
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm-project/offload/build' - task: DeleteFiles@1
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm-project/offload' displayName: 'Cleanup Binaries Directory'
installDir: '$(Build.BinariesDirectory)/llvm' inputs:
extraBuildFlags: >- SourceFolder: '$(Build.BinariesDirectory)'
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Build.BinariesDirectory)" Contents: '/**/*'
-DCMAKE_BUILD_TYPE=Release RemoveDotFiles: true
-DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
-DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_SKIP_INSTALL_RPATH=TRUE
-DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DLIBOMPTARGET_LLVM_INCLUDE_DIRS=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DCMAKE_EXE_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/llvm/lib"
-DCMAKE_SHARED_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/llvm/lib"
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} sourceDir: $(Build.ArtifactStagingDirectory)
os: ${{ job.os }} targetDir: $(Build.BinariesDirectory)/llvm
- task: DeleteFiles@1
displayName: 'Cleanup Staging Directory'
inputs:
SourceFolder: $(Build.ArtifactStagingDirectory)
Contents: '/**/*'
RemoveDotFiles: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: aomp_testing
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} dependsOn: aomp
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} ${{ if eq(parameters.checkoutRef, '') }}:
os: ${{ job.os }} dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: ROCm symbolic link displayName: ROCm symbolic link
inputs: inputs:
targetType: inline targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
@@ -257,7 +472,6 @@ jobs:
Contents: FileCheck Contents: FileCheck
TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests/bin TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests/bin
retryCount: 3 retryCount: 3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3 - task: Bash@3
displayName: Test AOMP displayName: Test AOMP
continueOnError: true continueOnError: true
@@ -272,9 +486,3 @@ jobs:
SKIP_TEST_PACKAGE: 1 SKIP_TEST_PACKAGE: 1
MAINLINE_BUILD: 1 MAINLINE_BUILD: 1
SUITE_LIST: smoke SUITE_LIST: smoke
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -1,174 +0,0 @@
parameters:
- name: componentName
type: string
default: aqlprofile
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/aqlprofile/cmake_modules
-DAQLPROFILE_BUILD_TESTS=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/rocm/share/hsa-amd-aqlprofile/
testExecutable: ./run_tests.sh
testParameters: ''
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -32,28 +27,14 @@ parameters:
type: object type: object
default: default:
- clr - clr
- composable_kernel
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: composable_kernel
- job: composable_kernel_build_${{ job.target }}
timeoutInMinutes: 240 timeoutInMinutes: 240
variables: variables:
- group: common - group: common
@@ -63,6 +44,12 @@ jobs:
pool: ${{ variables.ULTRA_BUILD_POOL }} pool: ${{ variables.ULTRA_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -73,10 +60,14 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- script: | - script: |
mkdir -p $(CCACHE_DIR) mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache" echo "##vso[task.prependpath]/usr/lib/ccache"
@@ -84,11 +75,11 @@ jobs:
- task: Cache@2 - task: Cache@2
displayName: Ccache caching displayName: Ccache caching
inputs: inputs:
key: composable_kernel | $(Agent.OS) | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ key: composable_kernel | $(Agent.OS) | $(JOB_GPU_TARGET) | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR) path: $(CCACHE_DIR)
restoreKeys: | restoreKeys: |
composable_kernel | $(Agent.OS) | ${{ job.target }} | $(DAY_STRING) composable_kernel | $(Agent.OS) | $(JOB_GPU_TARGET) | $(DAY_STRING)
composable_kernel | $(Agent.OS) | ${{ job.target }} composable_kernel | $(Agent.OS) | $(JOB_GPU_TARGET)
composable_kernel | $(Agent.OS) composable_kernel | $(Agent.OS)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
@@ -99,42 +90,29 @@ jobs:
-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_HIP_FLAGS="-Wno-missing-include-dirs" -DCMAKE_HIP_FLAGS="-Wno-missing-include-dirs"
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCK_BUILD_JIT_LIB=ON
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }} -DGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
# if this artifact name is changed, please also update $ARTIFACT_URL inside miopen-get-ck-build.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: composable_kernel_testing
- job: composable_kernel_test_${{ job.target }} dependsOn: composable_kernel
timeoutInMinutes: 180 condition: succeeded()
dependsOn: composable_kernel_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: TEST_LOG_FILE - name: TEST_LOG_FILE
value: $(Pipeline.Workspace)/ckTestLog.log value: $(Pipeline.Workspace)/ckTestLog.log
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -142,14 +120,21 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: Iterate through test scripts displayName: Iterate through test scripts
inputs: inputs:
@@ -159,8 +144,3 @@ jobs:
./$file | tee -a $(TEST_LOG_FILE) ./$file | tee -a $(TEST_LOG_FILE)
done done
workingDirectory: $(Agent.BuildDirectory)/rocm/bin workingDirectory: $(Agent.BuildDirectory)/rocm/bin
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,19 +1,16 @@
parameters: parameters:
- name: jobMatrix - name: checkoutRepo
type: object type: string
default: default: 'self'
copyJobs: - name: checkoutRef
- { os: ubuntu2204, backend: amd } type: string
- { os: almalinux8, backend: amd } default: ''
- { os: ubuntu2204, backend: nvidia }
- { os: almalinux8, backend: nvidia }
# hip and clr are tightly-coupled # hip and clr are tightly-coupled
# run this same template for both repos # run this same template for both repos
# any changes for clr should just trigger HIP pipeline # any changes for clr should just trigger HIP pipeline
jobs: jobs:
- ${{ each job in parameters.jobMatrix.copyJobs }}: - job: hip_clr_combined
- job: hip_clr_combined_${{ job.os }}_${{ job.backend }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -28,17 +25,8 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml
parameters: parameters:
componentName: HIP componentName: HIP
pipelineId: $(HIP_PIPELINE_ID) pipelineId: $(hip-pipeline-id)
fileFilter: ${{ job.os }}*${{ job.backend }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
- task: Bash@3
displayName: Copy HIP artifacts
inputs:
targetType: inline
script: cp -a $(Agent.BuildDirectory)/rocm/* $(Build.BinariesDirectory)/
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters: parameters:
os: ${{ job.os }} sourceDir: $(Agent.BuildDirectory)/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
inputs:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -44,16 +39,19 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF -DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
# only run test03 because test11 has too many test cases, taking way too long # only run test03 because test11 has too many test cases, taking way too long
- task: Bash@3 - task: Bash@3
@@ -65,8 +63,3 @@ jobs:
make test03 make test03
./bin/test ./bin/test
workingDirectory: $(Build.SourcesDirectory)/test workingDirectory: $(Build.SourcesDirectory)/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: combined

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hip-tests
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -61,28 +37,9 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
# HIP with AMD backend # HIP with AMD backend
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hip_tests
- job: hip_tests_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -91,6 +48,12 @@ jobs:
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -99,18 +62,19 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# compile hip-tests # compile hip-tests
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hip-tests
cmakeSourceDir: '../catch' cmakeSourceDir: '../catch'
customBuildTarget: build_tests customBuildTarget: build_tests
extraBuildFlags: >- extraBuildFlags: >-
@@ -118,75 +82,63 @@ jobs:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DHIP_PATH=$(Agent.BuildDirectory)/rocm -DHIP_PATH=$(Agent.BuildDirectory)/rocm
-DOFFLOAD_ARCH_STR="--offload-arch=${{ job.target }}" -DOFFLOAD_ARCH_STR="--offload-arch=$(JOB_GPU_TARGET)"
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hip_tests_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: timeoutInMinutes: 180
- job: hip_tests_test_${{ job.target }} dependsOn: hip_tests
timeoutInMinutes: 240 condition: succeeded()
dependsOn: hip_tests_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: Symlink rocm_agent_enumerator displayName: Symlink rocm_agent_enumerator
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo mkdir -p /opt/rocm/bin sudo mkdir -p /opt/rocm/bin
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hip_tests
testDir: $(Agent.BuildDirectory)/rocm/share/hip testDir: $(Agent.BuildDirectory)/rocm/share/hip
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml reloadAMDGPU: true
parameters: - task: Bash@3
aptPackages: ${{ parameters.aptPackages }} displayName: Clean up symlink
environment: test inputs:
gpuTarget: ${{ job.target }} targetType: inline
optSymLink: true script: sudo rm -rf /opt/rocm

View File

@@ -1,135 +1,62 @@
parameters: parameters:
- name: componentName
type: string
default: hipBLAS-common
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- git
- wget - wget
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocminfo - llvm-project
- ROCR-Runtime - ROCR-Runtime
- clr
- name: jobMatrix - rocminfo
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
- name: downstreamComponentMatrix
type: object
default:
- hipBLASLt:
name: hipBLASLt
sparseCheckoutDir: projects/hipblaslt
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLAS_common_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipBLAS_common
- job: hipBLAS_common_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH - name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm
- template: /.azuredevops/variables-global.yml
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} # CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# extraEnvVars:
# - ROCM_PATH:::/home/user/workspace/rocm
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipBLAS
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -40,10 +16,6 @@ parameters:
- libgtest-dev - libgtest-dev
- wget - wget
- python3-pip - python3-pip
- name: pipModules
type: object
default:
- pyyaml
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -75,68 +47,39 @@ parameters:
- rocSOLVER - rocSOLVER
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
# MIOpen depends on both rocRAND and hipBLAS
# for a unified build, hipBLAS will be the one to call MIOpen
- name: downstreamComponentMatrix
type: object
default:
- MIOpen:
name: MIOpen
sparseCheckoutDir: projects/miopen
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLAS_build
unifiedBuild:
downstreamAggregateNames: hipBLAS+rocRAND
buildDependsOn:
- hipBLAS_build
- rocRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipBLAS
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -149,84 +92,49 @@ jobs:
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-DCPACK_SET_DESTDIR=OFF -DCPACK_SET_DESTDIR=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
installAOCL: true
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipBLAS_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipBLAS
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipBLAS
testExecutable: $(Agent.BuildDirectory)/rocm/bin/hipblas-test testExecutable: $(Agent.BuildDirectory)/rocm/bin/hipblas-test
testParameters: '--yaml hipblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--yaml hipblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,62 +1,27 @@
parameters: parameters:
- name: componentName
type: string
default: hipBLASLt
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- ccache
- gfortran
- git - git
- libboost-filesystem-dev
- libboost-program-options-dev
- libdrm-dev - libdrm-dev
- liblapack-dev
- libmsgpack-dev - libmsgpack-dev
- libnuma-dev - libnuma-dev
- libopenblas-dev
- ninja-build - ninja-build
- python3-pip - python3-pip
- python3-venv - python3-venv
- gfortran
- libblas-dev
- ccache
- name: pipModules - name: pipModules
type: object type: object
default: default:
- joblib - joblib
- "packaging>=22.0"
- pyyaml
- msgpack
- simplejson
- ujson
- orjson
- yappi
- --upgrade
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -65,7 +30,6 @@ parameters:
- hipBLAS-common - hipBLAS-common
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocm-cmake
- rocm_smi_lib - rocm_smi_lib
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
@@ -77,48 +41,15 @@ parameters:
- clr - clr
- hipBLAS-common - hipBLAS-common
- llvm-project - llvm-project
- rocm-cmake
- rocminfo - rocminfo
- rocm_smi_lib - rocm_smi_lib
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { pool: rocm-ci_ultra_build_pool, os: ubuntu2204, packageManager: apt, target: gfx942 }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx90a }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { pool: rocm-ci_ultra_build_pool, os: almalinux8, packageManager: dnf, target: gfx942 }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx90a }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1201 }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocBLAS:
name: rocBLAS
sparseCheckoutDir: projects/rocblas
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLASLt_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipBLASLt
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 300 timeoutInMinutes: 300
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -126,62 +57,63 @@ jobs:
value: $(Build.BinariesDirectory)/rocm value: $(Build.BinariesDirectory)/rocm
- name: TENSILE_ROCM_ASSEMBLER_PATH - name: TENSILE_ROCM_ASSEMBLER_PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang value: $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
- name: CMAKE_CXX_COMPILER
value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH - name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler
- name: ROCM_PATH - name: TENSILE_ROCM_PATH
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
- name: DAY_STRING - name: DAY_STRING
value: $[format('{0:ddMMyyyy}', pipeline.startTime)] value: $[format('{0:ddMMyyyy}', pipeline.startTime)]
pool: ${{ job.pool }} pool: ${{ variables.ULTRA_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
- task: Bash@3 ${{ elseif ne(parameters.checkoutRef, '') }}:
displayName: Add ROCm binaries to PATH dependencySource: tag-builds
inputs: - script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
targetType: inline displayName: ROCm symbolic link
script: | # Build and install gtest, lapack, hipBLAS-common
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin" # $(Pipeline.Workspace)/deps is a temporary folder for the build process
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin" # $(Pipeline.Workspace)/s/deps is part of the hipBLASLt repo
- task: Bash@3 - script: mkdir $(Pipeline.Workspace)/deps
displayName: Build and install LAPACK displayName: Create temp folder for external dependencies
inputs: # hipBLASLt already has a CMake script for external deps, so we can just run that
targetType: inline # https://github.com/ROCm/hipBLASLt/blob/develop/deps/CMakeLists.txt
script: | - script: cmake $(Pipeline.Workspace)/s/deps
mkdir -p $(Agent.BuildDirectory)/temp-deps displayName: Configure hipBLASLt external dependencies
cd $(Agent.BuildDirectory)/temp-deps workingDirectory: $(Pipeline.Workspace)/deps
# position-independent LAPACK is required for almalinux8 builds - script: make
cmake -DBUILD_GTEST=OFF -DBUILD_LAPACK=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON $(Agent.BuildDirectory)/sparse/projects/hipblaslt/deps displayName: Build hipBLASLt external dependencies
make -j workingDirectory: $(Pipeline.Workspace)/deps
sudo make install - script: sudo make install
displayName: Install hipBLASLt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps
- script: | - script: |
mkdir -p $(CCACHE_DIR) mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache" echo "##vso[task.prependpath]/usr/lib/ccache"
@@ -189,124 +121,72 @@ jobs:
- task: Cache@2 - task: Cache@2
displayName: Ccache caching displayName: Ccache caching
inputs: inputs:
key: hipBLASLt | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ key: hipBLASLt | $(Agent.OS) | $(JOB_GPU_TARGET) | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR) path: $(CCACHE_DIR)
restoreKeys: | restoreKeys: |
hipBLASLt | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING) hipBLASLt | $(Agent.OS) | $(JOB_GPU_TARGET) | $(DAY_STRING)
hipBLASLt | ${{ job.os }} | ${{ job.target }} hipBLASLt | $(Agent.OS) | $(JOB_GPU_TARGET)
hipBLASLt | ${{ job.os }} hipBLASLt | $(Agent.OS)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
cmakeSourceDir: $(Agent.BuildDirectory)/sparse/projects/hipblaslt
cmakeBuildDir: $(Agent.BuildDirectory)/sparse/projects/hipblaslt/build
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor -DCMAKE_BUILD_TYPE=Release
-DCMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/llvm/include
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DGPU_TARGETS=${{ job.target }} -DTensile_LOGIC=
-DTensile_CPU_THREADS=
-DTensile_CODE_OBJECT_VERSION=default
-DTensile_LIBRARY_FORMAT=msgpack
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DHIPBLASLT_ENABLE_ROCROLLER=ON
-DHIPBLASLT_ENABLE_FETCH=ON
-DHIPBLASLT_ENABLE_BLIS=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
installLatestCMake: true
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/amdclang
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
- ROCM_PATH:::/home/user/workspace/rocm
extraCopyDirectories:
- deps
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: hipBLASLt_testing
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} dependsOn: hipBLASLt
timeoutInMinutes: 300 condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipBLASLt
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipblaslt-test' testExecutable: './hipblaslt-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,40 +1,17 @@
parameters: parameters:
- name: componentName
type: string
default: hipCUB
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- libgtest-dev
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
@@ -51,143 +28,91 @@ parameters:
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipCUB
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_BENCHMARK=ON
-DBUILD_TEST=ON -DBUILD_TEST=ON
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
extraCxxFlags: -Wno-deprecated-declarations
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipCUB_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipCUB
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipCUB
testDir: '$(Agent.BuildDirectory)/rocm/bin/hipcub' testDir: '$(Agent.BuildDirectory)/rocm/bin/hipcub'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipFFT
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -64,27 +40,8 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipFFT
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -94,6 +51,12 @@ jobs:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -102,15 +65,16 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -119,69 +83,56 @@ jobs:
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DUSE_HIP_CLANG=ON -DUSE_HIP_CLANG=ON
-DHIP_COMPILER=clang -DHIP_COMPILER=clang
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: hipFFT_testing
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }} dependsOn: hipFFT
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipFFT
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipfft-test' testExecutable: './hipfft-test'
testParameters: '--test_prob 0.002 --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--test_prob 0.002 --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,50 +1,25 @@
parameters: parameters:
- name: componentName
type: string
default: hipRAND
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- python3-pip - googletest
- git
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project - llvm-project
- rocm-cmake - ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocRAND - rocRAND
- ROCR-Runtime
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -52,168 +27,96 @@ parameters:
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- rocRAND
- ROCR-Runtime - ROCR-Runtime
- rocRAND
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocFFT:
name: rocFFT
sparseCheckoutDir: projects/rocfft
skipUnifiedBuild: 'false'
buildDependsOn:
- hipRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipRAND
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: HIP_ROCCLR_HOME - name: HIP_ROCCLR_HOME
value: $(Build.BinariesDirectory)/rocm value: $(Build.BinariesDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
${{ if eq(job.os, 'almalinux8') }}: vmImage: ${{ variables.BASE_BUILD_POOL }}
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DBUILD_TEST=ON -DBUILD_TEST=ON
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipRAND_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipRAND
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipRAND
testDir: '$(Agent.BuildDirectory)/rocm/bin/hipRAND' testDir: '$(Agent.BuildDirectory)/rocm/bin/hipRAND'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipSOLVER
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -39,7 +15,6 @@ parameters:
- git - git
- googletest - googletest
- libgtest-dev - libgtest-dev
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -67,35 +42,22 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocSOLVER - rocSOLVER
- rocSPARSE - rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSOLVER
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -104,21 +66,21 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# build external gtest and lapack # build external gtest and lapack
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: external componentName: external
cmakeBuildDir: '$(Agent.BuildDirectory)/s/deps/build' cmakeBuildDir: 'deps/build'
cmakeSourceDir: '$(Agent.BuildDirectory)/s/deps'
installDir: '$(Pipeline.Workspace)/deps-install' installDir: '$(Pipeline.Workspace)/deps-install'
extraBuildFlags: >- extraBuildFlags: >-
-DBUILD_BOOST=OFF -DBUILD_BOOST=OFF
@@ -134,65 +96,49 @@ jobs:
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DUSE_CUDA=OFF -DUSE_CUDA=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraCopyDirectories:
# - deps-install
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipSOLVER_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipSOLVER
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipSOLVER
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsolver-test' testExecutable: './hipsolver-test'
testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,44 +1,21 @@
parameters: parameters:
- name: componentName
type: string
default: hipSPARSE
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- gfortran
- git
- libboost-program-options-dev
- libfftw3-dev
- ninja-build - ninja-build
- python3-pip - libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- gfortran
- libgtest-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -60,37 +37,9 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocSPARSE - rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipSPARSELt:
name: hipSPARSELt
sparseCheckoutDir: projects/hipsparselt
skipUnifiedBuild: 'false'
buildDependsOn:
- hipSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSPARSE
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -98,127 +47,92 @@ jobs:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} artifactName: hipSPARSE
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
publish: false publish: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters: parameters:
sourceDir: $(Agent.BuildDirectory)/s/build/clients sourceDir: $(Build.SourcesDirectory)/build/clients
contentsString: matrices/** contentsString: matrices/**
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
artifactName: testMatrices artifactName: testMatrices
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: test
# gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipSPARSE_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipSPARSE
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipSPARSE
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsparse-test' testExecutable: './hipsparse-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipSPARSELt
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -40,12 +16,10 @@ parameters:
- gfortran - gfortran
- libgfortran5 - libgfortran5
- libopenblas-dev - libopenblas-dev
- liblapack-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- joblib - joblib
- msgpack
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -54,12 +28,9 @@ parameters:
- hipSPARSE - hipSPARSE
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocm-cmake
- rocm_smi_lib
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -68,28 +39,13 @@ parameters:
- llvm-project - llvm-project
- hipBLAS-common - hipBLAS-common
- hipBLASLt - hipBLASLt
- rocm-cmake
- rocBLAS - rocBLAS
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSPARSELt
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -106,42 +62,38 @@ jobs:
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
# ignore sparse checkout for monorepo case, we want access to hipblaslt directory
# sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
# NOTE: content between `---` is for transition support between old/new build systems ${{ elseif ne(parameters.checkoutRef, '') }}:
# and should be removed once transition is complete. dependencySource: tag-builds
# -----------------------------
# Build and install gtest and lapack # Build and install gtest and lapack
# $(Pipeline.Workspace)/deps is a temporary folder for the build process # $(Pipeline.Workspace)/deps is a temporary folder for the build process
# $(Pipeline.Workspace)/s/deps is part of the hipSPARSELt repo # $(Pipeline.Workspace)/s/deps is part of the hipSPARSELt repo
- script: mkdir -p $(Pipeline.Workspace)/deps - script: mkdir $(Pipeline.Workspace)/deps
displayName: Create temp folder for external dependencies displayName: Create temp folder for external dependencies
# hipSPARSELt already has a CMake script for external deps, so we can just run that # hipSPARSELt already has a CMake script for external deps, so we can just run that
# https://github.com/ROCm/hipSPARSELt/blob/develop/deps/CMakeLists.txt # https://github.com/ROCm/hipSPARSELt/blob/develop/deps/CMakeLists.txt
- ${{ if ne(parameters.sparseCheckoutDir, '') }}: - script: cmake $(Pipeline.Workspace)/s/deps
script: cmake $(Pipeline.Workspace)/s/projects/hipsparselt/deps
${{ else }}:
script: cmake $(Pipeline.Workspace)/s/deps
displayName: Configure hipSPARSELt external dependencies displayName: Configure hipSPARSELt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps workingDirectory: $(Pipeline.Workspace)/deps
- script: make - script: make
@@ -150,115 +102,58 @@ jobs:
- script: sudo make install - script: sudo make install
displayName: Install hipSPARSELt external dependencies displayName: Install hipSPARSELt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps workingDirectory: $(Pipeline.Workspace)/deps
# -----------------------------
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
# NOTE: the following options are old build only
# and can be removed after full transition to new build
# -DAMDGPU_TARGETS=${{ job.target }}
# -DCMAKE_Fortran_COMPILER=f95
# -DTensile_LOGIC=
# -DTensile_CPU_THREADS=
# -DTensile_LIBRARY_FORMAT=msgpack
# -DROCM_PATH=$(Agent.BuildDirectory)/rocm
# -DBUILD_CLIENTS_TESTS=ON
# -DBUILD_USE_LOCAL_TENSILE=OFF
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm" -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DGPU_TARGETS=${{ job.target }}
-DAMDGPU_TARGETS=${{ job.target }}
-DCMAKE_Fortran_COMPILER=f95
-DTensile_LOGIC= -DTensile_LOGIC=
-DTensile_CPU_THREADS= -DTensile_CPU_THREADS=
-DTensile_CODE_OBJECT_VERSION=default
-DTensile_LIBRARY_FORMAT=msgpack -DTensile_LIBRARY_FORMAT=msgpack
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_USE_LOCAL_TENSILE=OFF
-DHIPSPARSELT_ENABLE_FETCH=ON
-GNinja -GNinja
${{ if ne(parameters.sparseCheckoutDir, '') }}:
cmakeSourceDir: $(Build.SourcesDirectory)/projects/hipsparselt
cmakeBuildDir: $(Build.SourcesDirectory)/projects/hipsparselt
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- deps
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang
- CMAKE_CXX_COMPILER:::/home/user/workspace/rocm/llvm/bin/hipcc
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
installLatestCMake: true
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: hipSPARSELt_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: hipSPARSELt
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: hipSPARSELt
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsparselt-test' testExecutable: './hipsparselt-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -35,29 +30,20 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipTensor
- job: hipTensor_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -68,49 +54,43 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/llvm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/llvm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DHIPTENSOR_BUILD_TESTS=ON -DHIPTENSOR_BUILD_TESTS=ON
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja multithreadFlag: -- -j32
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: hipTensor_testing
- job: hipTensor_test_${{ job.target }}
timeoutInMinutes: 90 timeoutInMinutes: 90
dependsOn: hipTensor_build_${{ job.target }} dependsOn: hipTensor
condition: condition: succeeded()
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -118,21 +98,23 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: hipTensor componentName: hipTensor
testDir: '$(Agent.BuildDirectory)/rocm/bin/hiptensor' testDir: '$(Agent.BuildDirectory)/rocm/bin/hiptensor'
testParameters: '-E ".*-extended" --output-on-failure --force-new-ctest-process --output-junit test_output.xml' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -44,44 +39,39 @@ parameters:
- rocSPARSE - rocSPARSE
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipfort
- job: hipfort_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -95,37 +85,27 @@ jobs:
-DHIPFORT_COMPILER_FLAGS="-cpp" -DHIPFORT_COMPILER_FLAGS="-cpp"
-DHIPFORT_AR=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ar -DHIPFORT_AR=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ar
-DHIPFORT_RANLIB=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ranlib -DHIPFORT_RANLIB=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ranlib
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_TESTING=ON -DBUILD_TESTING=ON
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
installLatestCMake: true
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: hipfort_testing
- job: hipfort_test_${{ job.target }} dependsOn: hipfort
dependsOn: hipfort_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -133,13 +113,21 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
@@ -148,19 +136,12 @@ jobs:
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3 - task: Bash@3
displayName: 'Test hipfort' displayName: 'Test hipfort'
inputs: inputs:
targetType: inline targetType: inline
script: PATH=$(Agent.BuildDirectory)/rocm/bin:$PATH make run_all script: PATH=$(Agent.BuildDirectory)/rocm/bin:$PATH make run_all
workingDirectory: $(Build.SourcesDirectory)/test workingDirectory: $(Build.SourcesDirectory)/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -5,79 +5,56 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- python3-pip
- libnuma-dev - libnuma-dev
- ninja-build - ninja-build
- pkg-config
- python-is-python3 - python-is-python3
- python3-pip
- zlib1g-dev - zlib1g-dev
- pkg-config
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocm-cmake - rocm-cmake
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
# - { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: llvm_project
- job: llvm_project_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
name: 'rocm-ci_high_build_pool_2404' #temporarily using 'high' pool while 'ultra' is down
${{ else }}:
name: 'rocm-ci_ultra_build_pool'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: HIP_DEVICE_LIB_PATH - name: HIP_DEVICE_LIB_PATH
value: '$(Build.BinariesDirectory)/amdgcn/bitcode' value: '$(Build.BinariesDirectory)/amdgcn/bitcode'
- name: HIP_PATH pool: ${{ variables.MEDIUM_BUILD_POOL }}
value: '$(Agent.BuildDirectory)/rocm'
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} skipLlvmSymlink: true
os: ${{ job.os }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: rocm-llvm componentName: rocm-llvm
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Build.BinariesDirectory)/llvm;$(Build.BinariesDirectory)" -DCMAKE_PREFIX_PATH="$(Build.BinariesDirectory)/llvm;$(Build.BinariesDirectory)"
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DLLVM_ENABLE_PROJECTS=clang;lld;clang-tools-extra;mlir;flang -DLLVM_ENABLE_PROJECTS=clang;lld;clang-tools-extra;mlir
-DLLVM_ENABLE_RUNTIMES=compiler-rt;libunwind;libcxx;libcxxabi -DLLVM_ENABLE_RUNTIMES=compiler-rt;libunwind;libcxx;libcxxabi
-DCLANG_ENABLE_AMDCLANG=ON -DCLANG_ENABLE_AMDCLANG=ON
-DLLVM_TARGETS_TO_BUILD=AMDGPU;X86 -DLLVM_TARGETS_TO_BUILD=AMDGPU;X86
@@ -103,24 +80,16 @@ jobs:
-DROCM_LLVM_BACKWARD_COMPAT_LINK=$(Build.BinariesDirectory)/llvm -DROCM_LLVM_BACKWARD_COMPAT_LINK=$(Build.BinariesDirectory)/llvm
-DROCM_LLVM_BACKWARD_COMPAT_LINK_TARGET=./lib/llvm -DROCM_LLVM_BACKWARD_COMPAT_LINK_TARGET=./lib/llvm
-GNinja -GNinja
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm/build' cmakeBuildDir: 'llvm/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm'
installDir: '$(Build.BinariesDirectory)/llvm' installDir: '$(Build.BinariesDirectory)/llvm'
# use llvm-lit to run unit tests for llvm, clang, and lld # use llvm-lit to run unit tests for llvm, clang, and lld
- task: Bash@3
displayName: 'Copy llvm-lit to install directory'
inputs:
targetType: inline
script: |
cp $(Build.SourcesDirectory)/llvm/build/bin/llvm-lit $(Build.BinariesDirectory)/llvm/bin/
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: check-llvm componentName: check-llvm
testDir: 'llvm/build' testDir: 'llvm/build'
testExecutable: './bin/llvm-lit' testExecutable: './bin/llvm-lit'
testParameters: '-q --xunit-xml-output=llvm_test_output.xml --filter-out="live-debug-values-spill-tracking" ./test' testParameters: '-q --xunit-xml-output=llvm_test_output.xml ./test'
testOutputFile: llvm_test_output.xml testOutputFile: llvm_test_output.xml
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: check-clang componentName: check-clang
@@ -128,7 +97,6 @@ jobs:
testExecutable: './bin/llvm-lit' testExecutable: './bin/llvm-lit'
testParameters: '-q --xunit-xml-output=clang_test_output.xml ./tools/clang/test' testParameters: '-q --xunit-xml-output=clang_test_output.xml ./tools/clang/test'
testOutputFile: clang_test_output.xml testOutputFile: clang_test_output.xml
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: check-lld componentName: check-lld
@@ -136,7 +104,6 @@ jobs:
testExecutable: './bin/llvm-lit' testExecutable: './bin/llvm-lit'
testParameters: '-q --xunit-xml-output=lld_test_output.xml ./tools/lld/test' testParameters: '-q --xunit-xml-output=lld_test_output.xml ./tools/lld/test'
testOutputFile: lld_test_output.xml testOutputFile: lld_test_output.xml
os: ${{ job.os }}
- task: CopyFiles@2 - task: CopyFiles@2
displayName: Copy FileCheck for Publishing displayName: Copy FileCheck for Publishing
inputs: inputs:
@@ -148,53 +115,28 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: device-libs componentName: device-libs
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build" -DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build"
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/device-libs/build' cmakeBuildDir: 'amd/device-libs/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/device-libs'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: comgr componentName: comgr
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build;$(Build.SourcesDirectory)/amd/device-libs/build" -DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build;$(Build.SourcesDirectory)/amd/device-libs/build"
-DCOMGR_DISABLE_SPIRV=1
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/comgr/build' cmakeBuildDir: 'amd/comgr/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/comgr'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: comgr componentName: comgr
testParameters: '--output-on-failure --force-new-ctest-process --output-junit comgr_test_output.xml' testParameters: '--output-on-failure --force-new-ctest-process --output-junit comgr_test_output.xml'
testDir: 'amd/comgr/build' testDir: 'amd/comgr/build'
testOutputFile: comgr_test_output.xml testOutputFile: comgr_test_output.xml
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: hipcc componentName: hipcc
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DHIPCC_BACKWARD_COMPATIBILITY=OFF -DHIPCC_BACKWARD_COMPATIBILITY=OFF
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/hipcc/build' cmakeBuildDir: 'amd/hipcc/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/hipcc'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
extraEnvVars:
- HIP_DEVICE_LIB_PATH:::/home/user/workspace/bin/amdgcn/bitcode
- HIP_PATH:::/home/user/workspace/rocm

View File

@@ -0,0 +1,167 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- python3-pip
- name: pipModules
type: object
default:
- astunparse==1.6.2
- colorlover
- dash>=1.12.0
- matplotlib
- numpy>=1.17.5
- pandas>=1.4.3
- pymongo
- pyyaml
- tabulate
- tqdm
- dash-svg
- dash-bootstrap-components
- kaleido
- setuptools
- plotille
- mock
- pytest
- pytest-cov
- pytest-xdist
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler
- rocprofiler-register
- roctracer
jobs:
- job: omniperf
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: omniperf_testing
dependsOn: omniperf
condition: succeeded()
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTHON_VERSION
value: 3.10
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET)
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DENABLE_TESTS=ON
-DINSTALL_TESTS=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: omniperf
testDir: $(Build.BinariesDirectory)/libexec/omniperf
testExecutable: export OMNIPERF_ARCH_OVERRIDE="MI300X"; ctest
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm compilers from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')"

View File

@@ -1,251 +0,0 @@
parameters:
- name: componentName
type: string
default: origami
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- wget
- python3
- python3-dev
- python3-pip
- libgtest-dev
- libboost-filesystem-dev
- libboost-program-options-dev
- name: pipModules
type: object
default:
- nanobind>=2.0.0
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipBLASLt:
name: hipBLASLt
sparseCheckoutDir: projects/hipblaslt
skipUnifiedBuild: 'false'
buildDependsOn:
- origami_build
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: origami_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DORIGAMI_BUILD_SHARED_LIBS=ON
-DORIGAMI_ENABLE_PYTHON=ON
-DORIGAMI_BUILD_TESTING=ON
-GNinja
- ${{ if ne(job.os, 'almalinux8') }}:
- task: PublishPipelineArtifact@1
displayName: 'Publish Build Directory Artifact'
inputs:
targetPath: '$(Agent.BuildDirectory)/s/build'
artifact: '${{ parameters.componentName }}_${{ job.os }}_build_dir'
publishLocation: 'pipeline'
- task: PublishPipelineArtifact@1
displayName: 'Publish Python Source Artifact'
inputs:
targetPath: '$(Agent.BuildDirectory)/s/python'
artifact: '${{ parameters.componentName }}_${{ job.os }}_python_src'
publishLocation: 'pipeline'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: origami_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: origami_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
- task: DownloadPipelineArtifact@2
displayName: 'Download Build Directory Artifact'
inputs:
artifact: '${{ parameters.componentName }}_${{ job.os }}_build_dir'
path: '$(Agent.BuildDirectory)/s/build'
- task: DownloadPipelineArtifact@2
displayName: 'Download Python Source Artifact'
inputs:
artifact: '${{ parameters.componentName }}_${{ job.os }}_python_src'
path: '$(Agent.BuildDirectory)/s/python'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './origami-tests'
testParameters: '--yaml origami-tests.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
- script: |
set -e
export PYTHONPATH=$(Agent.BuildDirectory)/s/build/python:$PYTHONPATH
echo "--- Running origami_test.py ---"
python3 $(Agent.BuildDirectory)/s/python/origami_test.py
echo "--- Running origami_grid_test.py ---"
python3 $(Agent.BuildDirectory)/s/python/origami_grid_test.py
displayName: 'Run Python Binding Tests'
condition: succeeded()
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -5,38 +5,32 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git
- libboost-program-options-dev
- libdrm-dev
- libfftw3-dev
- libnuma-dev
- libstdc++-12-dev
- ninja-build
- python3-pip - python3-pip
- cmake
- libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- ninja-build
- libstdc++-12-dev
- libnuma-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- rocm-core
- HIPIFY
- aomp - aomp
- aomp-extras - aomp-extras
- clr
- HIPIFY
- llvm-project
- rocm-cmake
- rocm-core
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -51,26 +45,9 @@ parameters:
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rccl
- job: rccl_build_${{ job.target }}
timeoutInMinutes: 90
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -79,68 +56,64 @@ jobs:
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
submoduleBehaviour: recursive submoduleBehaviour: recursive
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- script: chmod +x $(Agent.BuildDirectory)/rocm/bin/hipify-perl
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_TESTS=ON -DBUILD_TESTS=ON
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake;$(Agent.BuildDirectory)/rocm/libexec/hipify -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
-DGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
installLatestCMake: true
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rccl_testing
- job: rccl_test_${{ job.target }}
timeoutInMinutes: 120 timeoutInMinutes: 120
dependsOn: rccl_build_${{ job.target }} dependsOn: rccl
condition: condition: succeeded()
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -148,22 +121,24 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rccl componentName: rccl
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rccl-UnitTests' testExecutable: './rccl-UnitTests'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: rdc
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -43,27 +19,20 @@ parameters:
- libtool - libtool
- pkg-config - pkg-config
- libdrm-dev - libdrm-dev
- libnuma-dev
- libyaml-cpp-dev - libyaml-cpp-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- amdsmi - amdsmi
- clr - clr
- hipBLAS-common
- hipBLASLt
- hipRAND
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocm-cmake - rocm-cmake
- rocm-core
- rocminfo - rocminfo
- rocm_smi_lib - rocm_smi_lib
- ROCmValidationSuite - ROCmValidationSuite
- rocprofiler - rocprofiler
- rocprofiler-register - rocprofiler-register
- rocprofiler-sdk
- rocRAND
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
@@ -79,65 +48,48 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rdc
- job: ${{ parameters.componentName }}_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
parameters:
cmakeVersion: '3.25.0'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# Build grpc # Build grpc
- task: Bash@3 - task: Bash@3
displayName: 'git clone grpc' displayName: 'git clone grpc'
inputs: inputs:
targetType: inline targetType: inline
script: git clone -b v1.67.1 https://github.com/grpc/grpc --depth=1 --shallow-submodules --recurse-submodules script: git clone -b v1.61.0 https://github.com/grpc/grpc --depth=1 --shallow-submodules --recurse-submodules
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
cmakeBuildDir: $(Build.SourcesDirectory)/grpc/build cmakeBuildDir: $(Build.SourcesDirectory)/grpc/build
cmakeSourceDir: $(Build.SourcesDirectory)/grpc
installDir: $(Build.SourcesDirectory)/bin installDir: $(Build.SourcesDirectory)/bin
extraBuildFlags: >- extraBuildFlags: >-
-DgRPC_INSTALL=ON -DgRPC_INSTALL=ON
@@ -145,79 +97,65 @@ jobs:
-DBUILD_SHARED_LIBS=ON -DBUILD_SHARED_LIBS=ON
-DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DGRPC_ROOT="$(Build.SourcesDirectory)/bin" -DGRPC_ROOT="$(Build.SourcesDirectory)/bin"
-DBUILD_RVS=ON -DBUILD_RVS=ON
-DBUILD_PROFILER=ON -DBUILD_PROFILER=ON
-DBUILD_TESTS=ON -DBUILD_TESTS=ON
-DAMDGPU_TARGETS=${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rdc_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rdc
- job: ${{ parameters.componentName }}_test_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm
- name: ROCM_DIR
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: Setup test environment displayName: Setup test environment
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
sudo rm -rf /opt/rocm
sudo rm -rf /usr/sbin/rdcd
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rdcd /usr/sbin/rdcd sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rdcd /usr/sbin/rdcd
echo $(Agent.BuildDirectory)/rocm/lib/rdc/grpc/lib | sudo tee /etc/ld.so.conf.d/grpc.conf echo $(Agent.BuildDirectory)/rocm/lib/rdc/grpc/lib | sudo tee /etc/ld.so.conf.d/grpc.conf
sudo ldconfig -v sudo ldconfig -v
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3 - task: Bash@3
displayName: Test rdc displayName: Test rdc
inputs: inputs:
@@ -227,9 +165,3 @@ jobs:
--batch_mode --batch_mode
--start_rdcd --start_rdcd
--unauth_comm --unauth_comm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/bin

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -26,8 +21,6 @@ parameters:
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libdlpack-dev
- libsndfile1-dev
- libswscale-dev - libswscale-dev
- libturbojpeg-dev - libturbojpeg-dev
- libjpeg-turbo-official=3.0.2-20240124 - libjpeg-turbo-official=3.0.2-20240124
@@ -43,15 +36,15 @@ parameters:
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- half
- llvm-project
- MIVisionX
- rocDecode
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime - ROCR-Runtime
- clr
- rocDecode
- half
- rpp - rpp
- MIVisionX
- aomp
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -60,35 +53,25 @@ parameters:
- half - half
- llvm-project - llvm-project
- MIVisionX - MIVisionX
- rocDecode
- rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rpp - rpp
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocAL
- job: rocAL_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- task: Bash@3 - task: Bash@3
displayName: 'Register libjpeg-turbo packages' displayName: 'Register libjpeg-turbo packages'
@@ -124,7 +107,6 @@ jobs:
parameters: parameters:
componentName: PyBind11 componentName: PyBind11
cmakeBuildDir: '$(Build.SourcesDirectory)/pybind11/build' cmakeBuildDir: '$(Build.SourcesDirectory)/pybind11/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/pybind11'
customInstallPath: false customInstallPath: false
installEnabled: false installEnabled: false
extraBuildFlags: >- extraBuildFlags: >-
@@ -141,7 +123,6 @@ jobs:
parameters: parameters:
componentName: RapidJSON componentName: RapidJSON
cmakeBuildDir: '$(Build.SourcesDirectory)/rapidjson/build' cmakeBuildDir: '$(Build.SourcesDirectory)/rapidjson/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/rapidjson'
customInstallPath: false customInstallPath: false
installEnabled: false installEnabled: false
extraBuildFlags: >- extraBuildFlags: >-
@@ -154,10 +135,14 @@ jobs:
workingDirectory: '$(Build.SourcesDirectory)/rapidjson/build' workingDirectory: '$(Build.SourcesDirectory)/rapidjson/build'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -165,33 +150,14 @@ jobs:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;/opt/libjpeg-turbo -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;/opt/libjpeg-turbo
-DCMAKE_INSTALL_PREFIX_PYTHON=$Python3_STDARCH -DCMAKE_INSTALL_PREFIX_PYTHON=$Python3_STDARCH
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# registerJPEGPackages: true
# gpuTarget: ${{ job.target }}
# extraCopyDirectories:
# - /opt/libjpeg-turbo
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocAL_testing
- job: rocAL_test_${{ job.target }} dependsOn: rocAL
dependsOn: rocAL_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -199,10 +165,14 @@ jobs:
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm
- name: CMAKE_INCLUDE_PATH - name: CMAKE_INCLUDE_PATH
value: $(Agent.BuildDirectory)/rocm/include/rocal value: $(Agent.BuildDirectory)/rocm/include/rocal
pool: pool: $(JOB_TEST_POOL)
name: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- task: Bash@3 - task: Bash@3
displayName: 'Register libjpeg-turbo packages' displayName: 'Register libjpeg-turbo packages'
@@ -220,13 +190,21 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: Link libjpeg-turbo displayName: Link libjpeg-turbo
inputs: inputs:
@@ -242,26 +220,15 @@ jobs:
mkdir rocAL-tests mkdir rocAL-tests
cd rocAL-tests cd rocAL-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocal/test cmake $(Agent.BuildDirectory)/rocm/share/rocal/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocAL componentName: rocAL
testDir: rocAL-tests testDir: rocAL-tests
reloadAMDGPU: true
- task: Bash@3 - task: Bash@3
displayName: Clean up libjpeg-turbo displayName: Clean up libjpeg-turbo
condition: always()
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
sudo rm /etc/ld.so.conf.d/libjpeg-turbo.conf sudo rm /etc/ld.so.conf.d/libjpeg-turbo.conf
sudo ldconfig -v sudo ldconfig -v
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerJPEGPackages: true
environment: test
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- /opt/libjpeg-turbo
# docker image will be missing the ldconfig to libjpeg-turbo

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -20,7 +15,6 @@ parameters:
- git - git
- mpich - mpich
- ninja-build - ninja-build
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -49,33 +43,24 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- rocSPARSE - rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocALUTION
- job: rocALUTION_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: HIP_ROCCLR_HOME - name: HIP_ROCCLR_HOME
value: $(Build.BinariesDirectory)/rocm value: $(Build.BinariesDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -86,10 +71,14 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -97,40 +86,29 @@ jobs:
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/ -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocALUTION_testing
- job: rocALUTION_test_${{ job.target }} dependsOn: rocALUTION
dependsOn: rocALUTION_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -138,22 +116,24 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocALUTION componentName: rocALUTION
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocalution-test' testExecutable: './rocalution-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,47 +1,25 @@
parameters: parameters:
- name: componentName
type: string
default: rocBLAS
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- gfortran
- libdrm-dev
- libmsgpack-dev
- libopenblas-dev
- ninja-build - ninja-build
- python3-pip
- python3-venv - python3-venv
- git
- libmsgpack-dev
- gfortran
- libopenblas-dev
- googletest
- libgtest-dev
- wget - wget
- python3-pip
- libdrm-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
@@ -50,18 +28,18 @@ parameters:
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocm-cmake - rocm-cmake
- rocm-core - llvm-project
- rocm_smi_lib - ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - rocm_smi_lib
- roctracer - rocm-core
- aomp
- aomp-extras
- hipBLAS-common
- hipBLASLt
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -70,67 +48,13 @@ parameters:
- hipBLAS-common - hipBLAS-common
- hipBLASLt - hipBLASLt
- llvm-project - llvm-project
- rocm-cmake
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- rocm_smi_lib - rocm_smi_lib
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocSPARSE:
name: rocSPARSE
sparseCheckoutDir: projects/rocsparse
skipUnifiedBuild: 'false'
buildDependsOn:
- rocBLAS_build
# rocSOLVER depends on both rocBLAS and rocPRIM
# for a unified build, rocBLAS will be the one to call rocSOLVER
# - rocSOLVER:
# name: rocSOLVER
# sparseCheckoutDir: projects/rocsolver
# skipUnifiedBuild: 'false'
# buildDependsOn:
# - rocBLAS_build
# unifiedBuild:
# downstreamAggregateNames: rocBLAS+rocPRIM
# buildDependsOn:
# - rocBLAS_build
# - rocPRIM_build
# temporary rocblas->hipblas downstream path while the SOLVERs are disabled
- hipBLAS:
name: hipBLAS
sparseCheckoutDir: projects/hipblas
skipUnifiedBuild: 'false'
buildDependsOn:
- rocBLAS_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocBLAS
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -138,102 +62,77 @@ jobs:
value: $(Build.BinariesDirectory)/rocm value: $(Build.BinariesDirectory)/rocm
- name: TENSILE_ROCM_ASSEMBLER_PATH - name: TENSILE_ROCM_ASSEMBLER_PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang
- name: CMAKE_CXX_COMPILER
value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH - name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
cmakeSourceDir: $(Agent.BuildDirectory)/sparse/projects/rocblas
cmakeBuildDir: $(Agent.BuildDirectory)/sparse/projects/rocblas/build
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor -DCMAKE_TOOLCHAIN_FILE=toolchain-linux.cmake
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DTensile_CODE_OBJECT_VERSION=default
-DTensile_LOGIC=asm_full
-DTensile_SEPARATE_ARCHITECTURES=ON
-DTensile_LAZY_LIBRARY_LOADING=ON
-DTensile_LIBRARY_FORMAT=msgpack
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
installAOCL: true
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
- ROCM_PATH:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocBLAS_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocBLAS
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
@@ -241,47 +140,24 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocBLAS
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocblas-test' testExecutable: './rocblas-test'
testParameters: '--yaml rocblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--yaml rocblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,218 +1,135 @@
parameters: parameters:
- name: componentName
type: string
default: rocDecode
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
- name: rocPyDecodeRepo
type: string
default: rocpydecode_repo
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- python3-pip
- cmake - cmake
- ninja-build
- pkg-config
- ffmpeg - ffmpeg
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libdrm-dev
- libstdc++-12-dev - libstdc++-12-dev
- libva-amdgpu-dev - libva-dev
- mesa-amdgpu-va-drivers - mesa-amdgpu-va-drivers
- ninja-build - libdrm-dev
- pkg-config
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocm-core - llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo
- rocm-core
- rocprofiler-register
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
- clr - clr
- llvm-project - llvm-project
- rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocPyDecode:
name: rocPyDecode
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- rocDecode_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocDecode
- job: ${{ parameters.componentName }}_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
steps: steps:
# Since mesa-amdgpu-multimedia-devel is not directly available from apt, register it
- task: Bash@3
displayName: 'Register ROCm packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/amdgpu/${{ variables.KEYRING_VERSION }}/ubuntu jammy main" | sudo tee /etc/apt/sources.list.d/amdgpu.list
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/${{ variables.KEYRING_VERSION }} jammy main" | sudo tee --append /etc/apt/sources.list.d/rocm.list
echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | sudo tee /etc/apt/preferences.d/rocm-pin-600
sudo apt update
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# registerROCmPackages: true
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocDecode_testing
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} dependsOn: rocDecode
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
- task: Bash@3 dependencySource: staging
displayName: Build rocDecode tests ${{ elseif ne(parameters.checkoutRef, '') }}:
inputs: dependencySource: tag-builds
targetType: inline # anything in /opt may be persistent across runs
script: | # so we need to remove the symlink if it already exists
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }} - script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
mkdir rocDecode-tests mkdir rocDecode-tests
cd rocDecode-tests cd rocDecode-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocdecode/test cmake /opt/rocm/share/rocdecode/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocDecode componentName: rocDecode
testDir: 'rocDecode-tests' testDir: 'rocDecode-tests'
os: ${{ job.os }} - script: sudo rm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.rocPyDecodeRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: rocFFT
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -65,36 +41,8 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
- name: downstreamComponentMatrix
type: object
default:
- hipFFT:
name: hipFFT
sparseCheckoutDir: projects/hipfft
skipUnifiedBuild: 'false'
buildDependsOn:
- rocFFT_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocFFT
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_ubuntu2204_${{ job.target }} # todo: un-hardcode OS
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -103,6 +51,12 @@ jobs:
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -111,15 +65,16 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -127,83 +82,56 @@ jobs:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DUSE_HIP_CLANG=ON -DUSE_HIP_CLANG=ON
-DHIP_COMPILER=clang -DHIP_COMPILER=clang
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocFFT_testing
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }} dependsOn: rocFFT
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocFFT
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocfft-test' testExecutable: './rocfft-test'
testParameters: '--test_prob 0.004 --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--test_prob 0.004 --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,183 +0,0 @@
parameters:
- name: componentName
type: string
default: rocJPEG
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-dev
- libstdc++-12-dev
- libva-amdgpu-dev
- mesa-amdgpu-va-drivers
- ninja-build
- pkg-config
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- rocm-core
- rocprofiler-register
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
name: rocm-ci_medium_build_pool_2404
${{ else }}:
name: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- task: Bash@3
displayName: Build rocJPEG tests
inputs:
targetType: inline
script: |
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
mkdir rocJPEG-tests
cd rocJPEG-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocjpeg/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocJPEG
testDir: 'rocJPEG-tests'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -5,50 +5,30 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- libdrm-dev
- libstdc++-12-dev
- ninja-build - ninja-build
- git
- python3-pip - python3-pip
- libdrm-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- hip-python --extra-index-url https://test.pypi.org/simple
- ml_dtypes
- numpy
- tomli - tomli
- scipy
- pybind11
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project - llvm-project
- rocm-cmake - rocm-cmake
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocMLIR_build - job: rocMLIR_library
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -66,9 +46,13 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -76,33 +60,22 @@ jobs:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_FAT_LIBROCKCOMPILER=1 -DBUILD_FAT_LIBROCKCOMPILER=1
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
# compiling and running test on the test system together # compiling and running test on the test system together
- job: rocMLIR_test_${{ job.target }} - job: rocMLIR_testing
dependsOn: rocMLIR_build
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: ${{ variables.GFX942_TEST_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -113,19 +86,26 @@ jobs:
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3 - task: Bash@3
displayName: Symlink rocm_agent_enumerator displayName: ROCm symbolic link
inputs: inputs:
targetType: inline targetType: inline
script: | script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo mkdir -p /opt/rocm/bin
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
installEnabled: false installEnabled: false
@@ -133,23 +113,13 @@ jobs:
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/lib -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_TEST_CHIPSET=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=${{ job.target }}
-DROCM_TEST_CHIPSET=${{ job.target }}
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocMLIR componentName: rocMLIR
testDir: $(Build.SourcesDirectory)/build testDir: $(Build.SourcesDirectory)/build
testExecutable: ninja testExecutable: ninja
testParameters: check-rocmlir testParameters: check-rocmlir
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,49 +1,26 @@
parameters: parameters:
- name: componentName
type: string
default: rocPRIM
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- libgtest-dev
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocminfo - llvm-project
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -53,179 +30,88 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 1, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 2, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 3, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 1, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 2, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 3, shardCount: 3 }
- name: downstreamComponentMatrix
type: object
default:
- rocThrust:
name: rocThrust
sparseCheckoutDir: projects/rocthrust
skipUnifiedBuild: 'false'
buildDependsOn:
- rocPRIM_build
- hipCUB:
name: hipCUB
sparseCheckoutDir: projects/hipcub
skipUnifiedBuild: 'false'
buildDependsOn:
- rocPRIM_build
# rocSOLVER depends on both rocBLAS and rocPRIM
# for a unified build, rocBLAS will be the one to call rocSOLVER
# - rocSOLVER:
# name: rocSOLVER
# sparseCheckoutDir: projects/rocsolver
# skipUnifiedBuild: 'true'
# buildDependsOn:
# - rocPRIM_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocPRIM
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DAMDGPU_TARGETS=${{ job.target }}
-DBUILD_BENCHMARK=ON -DBUILD_BENCHMARK=ON
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_TEST=ON -DBUILD_TEST=ON
-GNinja -GNinja
extraCxxFlags: -Wno-deprecated-declarations
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocPRIM_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocPRIM
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}_shard_${{ job.shard }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocPRIM
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocprim' testDir: '$(Agent.BuildDirectory)/rocm/bin/rocprim'
extraTestParameters: '-I ${{ job.shard }},,${{ job.shardCount }}'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -5,260 +5,142 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- python3-pip
- cmake - cmake
- ninja-build
- pkg-config
- ffmpeg - ffmpeg
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libdlpack-dev
- libdrm-dev
- libva-dev - libva-dev
- ninja-build - libdrm-dev
- pkg-config - pybind11-dev
- python3-pip - python3-pybind11
- libdlpack-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- hip-python --extra-index-url https://test.pypi.org/simple - -i
- numpy - https://test.pypi.org/simple
- pybind11 - hip-python
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocDecode
- rocJPEG
- rocm-cmake - rocm-cmake
- rocm-core - llvm-project
- rocminfo
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo
- rocm-core
- rocprofiler-register - rocprofiler-register
- rocDecode
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocPyDecode
- job: rocPyDecode_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
- task: Bash@3 # manual build case: triggered by ROCm/ROCm repo
displayName: 'Save Python Package Paths' ${{ elseif ne(parameters.checkoutRef, '') }}:
inputs: dependencySource: tag-builds
targetType: inline
script: |
echo "##vso[task.setvariable variable=PYTHON_USER_SITE;]$(python3 -m site --user-site)"
echo "##vso[task.setvariable variable=PYTHON_DIST_PACKAGES;]$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "##vso[task.setvariable variable=PYBIND11_PATH;]$(python3 -c 'import pybind11; print(pybind11.get_cmake_dir())')"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
installEnabled: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(PYTHON_USER_SITE)/pybind11;$(PYTHON_DIST_PACKAGES)/pybind11;$(PYBIND11_PATH) -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_INSTALL_PREFIX_PYTHON=$(Build.BinariesDirectory) -DCMAKE_INSTALL_PREFIX_PYTHON=$(Build.BinariesDirectory)
-GNinja -GNinja
- task: Bash@3
displayName: 'rocPyDecode install'
inputs:
targetType: inline
script: |
sudo cmake --build . --target install
sudo chown -R $(whoami):$(id -gn) $(Build.BinariesDirectory)
workingDirectory: $(Build.SourcesDirectory)/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
publish: false
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: |
export HIP_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include/hip
sudo python3 setup.py bdist_wheel
sudo chown -R $(whoami):$(id -gn) $(find . -name "*.whl")
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Build.SourcesDirectory)/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- task: Bash@3
displayName: Save pipeline artifact file names
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.ArtifactStagingDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocPyDecode_testing
- job: rocPyDecode_test_${{ job.target }} dependsOn: rocPyDecode
dependsOn: rocPyDecode_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- task: Bash@3
displayName: Ensure pybind11-dev is not installed
inputs:
targetType: inline
script: |
if dpkg -l | grep -q pybind11-dev; then
echo "Removing pybind11-dev..."
sudo apt remove -y pybind11-dev
else
echo "pybind11-dev is not installed."
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
setupHIPLibrarySymlinks: true ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- task: Bash@3 - task: Bash@3
displayName: pip install displayName: Setup test environment
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
pip uninstall -y rocPyDecode sudo rm -rf /opt/rocm
find -name *.whl -exec pip install {} \; sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
workingDirectory: $(Agent.BuildDirectory) cd $(Build.SourcesDirectory)
- task: Bash@3 sudo pip install .
displayName: Setup search paths cmake -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) .
inputs:
targetType: inline
script: |
echo "##vso[task.setvariable variable=PYTHON_USER_SITE;]$(python3 -m site --user-site)"
echo "##vso[task.setvariable variable=PYTHON_DIST_PACKAGES;]$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "##vso[task.setvariable variable=PYBIND11_PATH;]$(python3 -c 'import pybind11; print(pybind11.get_cmake_dir())')"
- task: CMake@1
displayName: 'rocPyDecode Test CMake Flags'
inputs:
workingDirectory: $(Agent.BuildDirectory)/rocm/share/rocpydecode/tests
cmakeArgs: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(PYTHON_USER_SITE)/pybind11;$(PYTHON_DIST_PACKAGES)/pybind11;$(PYBIND11_PATH)
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
.
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocPyDecode componentName: rocPyDecode
testDir: $(Agent.BuildDirectory)/rocm/share/rocpydecode/tests testDir: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml # sudo required for pip install but screws up permissions for next pipeline run
parameters: - task: Bash@3
aptPackages: ${{ parameters.aptPackages }} displayName: Clean up test environment
pipModules: ${{ parameters.pipModules }} inputs:
environment: test targetType: inline
gpuTarget: ${{ job.target }} script: |
pythonEnvVars: true sudo rm -rf $(Build.SourcesDirectory)/*
# note that this docker won't have hip-python installed via pip

View File

@@ -1,49 +1,26 @@
parameters: parameters:
- name: componentName
type: string
default: rocRAND
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- python3-pip - googletest
- libgtest-dev
- git
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocminfo - llvm-project
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -53,171 +30,90 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipRAND:
name: hipRAND
sparseCheckoutDir: projects/hiprand
skipUnifiedBuild: 'false'
buildDependsOn:
- rocRAND_build
- MIOpen:
name: MIOpen
sparseCheckoutDir: projects/miopen
skipUnifiedBuild: 'true'
buildDependsOn:
- rocRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocRAND
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: HIP_ROCCLR_HOME - name: HIP_ROCCLR_HOME
value: $(Build.BinariesDirectory)/rocm value: $(Build.BinariesDirectory)/rocm
pool: pool:
${{ if eq(job.os, 'ubuntu2404') }}: vmImage: ${{ variables.BASE_BUILD_POOL }}
vmImage: 'ubuntu-24.04'
${{ else }}:
name: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DAMDGPU_TARGETS=${{ job.target }}
-DBUILD_TEST=ON -DBUILD_TEST=ON
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocRAND_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocRAND
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocRAND
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocRAND' testDir: '$(Agent.BuildDirectory)/rocm/bin/rocRAND'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,54 +1,34 @@
parameters: parameters:
- name: componentName
type: string
default: rocSOLVER
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- gfortran
- git
- libfmt-dev
- libsuitesparse-dev
- ninja-build - ninja-build
- libsuitesparse-dev
- gfortran
- libfmt-dev
- git
- googletest
- libgtest-dev
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr - clr
- hipSPARSE
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocm-cmake - rocm-cmake
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- ROCR-Runtime - ROCR-Runtime
- rocSPARSE
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -64,205 +44,111 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocSPARSE - rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipBLAS:
name: hipBLAS
sparseCheckoutDir: projects/hipblas
skipUnifiedBuild: 'false'
buildDependsOn:
- rocSOLVER_build
# hipSOLVER depends on both rocSOLVER and rocSPARSE
# for a unified build, rocSOLVER will be the one to call hipSOLVER
# - hipSOLVER:
# name: hipSOLVER
# sparseCheckoutDir: projects/hipsolver
# skipUnifiedBuild: 'false'
# buildDependsOn:
# - rocSOLVER_build
# unifiedBuild:
# downstreamAggregateNames: rocSOLVER+rocSPARSE
# buildDependsOn:
# - rocSOLVER_build
# - rocSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocSOLVER
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- task: Bash@3 - task: Bash@3
displayName: 'Clone lapack' displayName: 'Clone lapack'
inputs: inputs:
targetType: inline targetType: inline
script: git clone --depth 1 --branch v3.9.1 https://github.com/Reference-LAPACK/lapack script: git clone --depth 1 --branch v3.9.1 https://github.com/Reference-LAPACK/lapack
workingDirectory: '$(Build.SourcesDirectory)' workingDirectory: '$(Build.SourcesDirectory)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: lapack componentName: lapack
os: ${{ job.os }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DCMAKE_Fortran_FLAGS=-fno-optimize-sibling-calls -DCMAKE_Fortran_FLAGS=-fno-optimize-sibling-calls
-DBUILD_TESTING=OFF -DBUILD_TESTING=OFF
-DCBLAS=ON -DCBLAS=ON
-DLAPACKE=OFF -DLAPACKE=OFF
-GNinja -GNinja
cmakeBuildDir: '$(Build.SourcesDirectory)/lapack/build' cmakeBuildDir: '$(Build.SourcesDirectory)/lapack/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/lapack'
installDir: '$(Pipeline.Workspace)/deps-install' installDir: '$(Pipeline.Workspace)/deps-install'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install;$(Agent.BuildDirectory)/vendor -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- deps-install
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocSOLVER_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocSOLVER
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocSOLVER
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocsolver-test' testExecutable: './rocsolver-test'
testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,57 +1,34 @@
parameters: parameters:
- name: componentName
type: string
default: rocSPARSE
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- gfortran
- git
- libboost-program-options-dev
- libdrm-dev
- libfftw3-dev
- ninja-build
- python3-pip - python3-pip
- cmake
- ninja-build
- libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- gfortran
- libgtest-dev
- libdrm-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocBLAS
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocBLAS
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -63,45 +40,9 @@ parameters:
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipSPARSE:
name: hipSPARSE
sparseCheckoutDir: projects/hipsparse
skipUnifiedBuild: 'false'
buildDependsOn:
- rocSPARSE_build
# hipSOLVER depends on both rocSOLVER and rocSPARSE
# for a unified build, rocSOLVER will be the one to call hipSOLVER
# - hipSOLVER:
# name: hipSOLVER
# sparseCheckoutDir: projects/hipsolver
# skipUnifiedBuild: 'true'
# buildDependsOn:
# - rocSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocSPARSE
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -110,134 +51,99 @@ jobs:
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang++ -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/hip/cmake -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/hip/cmake
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} artifactName: rocSPARSE
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
publish: false publish: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters: parameters:
sourceDir: $(Agent.BuildDirectory)/s/build/clients sourceDir: $(Build.SourcesDirectory)/build/clients
contentsString: matrices/** contentsString: matrices/**
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
artifactName: testMatrices artifactName: testMatrices
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocSPARSE_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: timeoutInMinutes: 90
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} dependsOn: rocSPARSE
timeoutInMinutes: 120 condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocSPARSE
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin' testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocsparse-test' testExecutable: './rocsparse-test'
testParameters: '--gtest_filter="*quick*" --gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_filter="*quick*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,49 +1,26 @@
parameters: parameters:
- name: componentName
type: string
default: rocThrust
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- libboost-program-options-dev - libboost-program-options-dev
- googletest
- libfftw3-dev - libfftw3-dev
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr - clr
- hipRAND
- llvm-project - llvm-project
- rocm-cmake
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- ROCR-Runtime - ROCR-Runtime
@@ -54,142 +31,92 @@ parameters:
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- hipRAND
- name: jobMatrix - rocprofiler-register
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocThrust
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >- extraBuildFlags: >-
-GNinja -GNinja
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DBUILD_TEST=ON -DBUILD_TEST=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocThrust_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocThrust
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocThrust
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocthrust' testDir: '$(Agent.BuildDirectory)/rocm/bin/rocthrust'
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml --exclude-regex "scan.hip"'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -17,7 +12,6 @@ parameters:
- cmake - cmake
- ninja-build - ninja-build
- libboost-program-options-dev - libboost-program-options-dev
- libdrm-dev
- libgtest-dev - libgtest-dev
- googletest - googletest
- libfftw3-dev - libfftw3-dev
@@ -50,29 +44,20 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocWMMA
- job: rocWMMA_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -83,10 +68,14 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -95,39 +84,28 @@ jobs:
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DROCWMMA_BUILD_TESTS=ON -DROCWMMA_BUILD_TESTS=ON
-DROCWMMA_BUILD_SAMPLES=OFF -DROCWMMA_BUILD_SAMPLES=OFF
-DGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON
-DROCM_PLATFORM_VERSION=$(NEXT_RELEASE_VERSION)
-GNinja -GNinja
# gfx1030 not supported in documentation # gfx1030 not supported in documentation
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocWMMA_testing
- job: rocWMMA_test_${{ job.target }} timeoutInMinutes: 90
timeoutInMinutes: 270 dependsOn: rocWMMA
dependsOn: rocWMMA_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -135,20 +113,22 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocWMMA componentName: rocWMMA
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocwmma' testDir: '$(Agent.BuildDirectory)/rocm/bin/rocwmma'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,49 +5,26 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- doxygen - doxygen
- doxygen-doc - doxygen-doc
- ninja-build - ninja-build
- python3-sphinx
- name: pipModules - name: pipModules
type: object type: object
default: default:
- cget - cget
- cmake==3.20.5
- ninja - ninja
- rocm-docs-core
- sphinx
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_cmake
- job: rocm_cmake_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
vmImage: 'ubuntu-24.04'
${{ else }}:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
@@ -55,43 +32,19 @@ jobs:
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- task: Bash@3
displayName: Add CMake to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(python3 -m site --user-base)/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: # extra steps for ctest suite
os: ${{ job.os }} - script: |
useAmdclang: false
- task: Bash@3
displayName: CTest setup
inputs:
targetType: inline
script: |
python -m pip install -r $(Build.SourcesDirectory)/docs/requirements.txt python -m pip install -r $(Build.SourcesDirectory)/docs/requirements.txt
python -m pip install -r $(Build.SourcesDirectory)/test/docsphinx/docs/.sphinx/requirements.txt python -m pip install -r $(Build.SourcesDirectory)/test/docsphinx/docs/.sphinx/requirements.txt
git config --global user.email "you@example.com" git config --global user.email "you@example.com"
git config --global user.name "Your Name" git config --global user.name "Your Name"
displayName: "ctest setup"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocm-cmake componentName: rocm-cmake
testParameters: '-E "pass-version-parent" --output-on-failure --force-new-ctest-process --output-junit test_output.xml'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# environment: combined

View File

@@ -1,84 +1,27 @@
parameters: parameters:
- name: componentName
type: string
default: rocm-core
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_core
- job: rocm_core_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
vmImage: 'ubuntu-24.04'
${{ else }}:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CURRENT_BINARY_DIR=$PWD -DCMAKE_CURRENT_BINARY_DIR=$PWD
-DCMAKE_CURRENT_SOURCE_DIR=$PWD/../ -DCMAKE_CURRENT_SOURCE_DIR=$PWD/../
@@ -86,18 +29,5 @@ jobs:
-DCPACK_GENERATOR=DEB -DCPACK_GENERATOR=DEB
-DCPACK_DEBIAN_PACKAGE_RELEASE="local.9999~99.99" -DCPACK_DEBIAN_PACKAGE_RELEASE="local.9999~99.99"
-DCPACK_RPM_PACKAGE_RELEASE="local.9999" -DCPACK_RPM_PACKAGE_RELEASE="local.9999"
-DROCM_VERSION="$(NEXT_RELEASE_VERSION)" -DROCM_VERSION="$(next-release)"
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -5,21 +5,11 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- libdw-dev
- libglfw3-dev - libglfw3-dev
- libmsgpack-dev
- libtbb-dev
- libva-amdgpu-dev
- ninja-build
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
@@ -35,21 +25,16 @@ parameters:
- hipRAND - hipRAND
- hipSOLVER - hipSOLVER
- hipSPARSE - hipSPARSE
- hipTensor
- llvm-project - llvm-project
- MIOpen
- rocBLAS - rocBLAS
- rocFFT - rocFFT
- rocJPEG
- rocPRIM - rocPRIM
- rocprofiler-register - rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- rocSOLVER - rocSOLVER
- rocSPARSE - rocSPARSE
- rocThrust - rocThrust
- rocWMMA
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
@@ -64,64 +49,50 @@ parameters:
- hipRAND - hipRAND
- hipSOLVER - hipSOLVER
- hipSPARSE - hipSPARSE
- hipTensor
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocFFT - rocFFT
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocJPEG
- rocprofiler-register - rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- rocSOLVER - rocSOLVER
- rocSPARSE - rocSPARSE
- rocThrust - rocThrust
- roctracer
- rocWMMA
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_examples
- job: rocm_examples_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
parameters:
cmakeVersion: '3.25.0'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
# https://github.com/ROCm/HIP/issues/2203 # https://github.com/ROCm/HIP/issues/2203
@@ -129,10 +100,9 @@ jobs:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_ROOT=$(Agent.BuildDirectory)/rocm -DROCM_ROOT=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }} -DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET)
-DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc -DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc
-GNinja
- task: Bash@3 - task: Bash@3
displayName: Move rocm-examples binaries to rocm/examples displayName: Move rocm-examples binaries to rocm/examples
inputs: inputs:
@@ -140,72 +110,62 @@ jobs:
script: | script: |
mkdir -p $(Build.BinariesDirectory)/examples mkdir -p $(Build.BinariesDirectory)/examples
mv $(Build.BinariesDirectory)/bin/* $(Build.BinariesDirectory)/examples mv $(Build.BinariesDirectory)/bin/* $(Build.BinariesDirectory)/examples
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocm_examples_testing
- job: rocm_examples_test_${{ job.target }} dependsOn: rocm_examples
dependsOn: rocm_examples_build_${{ job.target }} condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: TEST_LOG_FILE - name: TEST_LOG_FILE
value: $(Pipeline.Workspace)/rocm-examplesTestLog.log value: $(Pipeline.Workspace)/rocm-examplesTestLog.log
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
parameters:
cmakeVersion: '3.25.0'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml ${{ if eq(parameters.checkoutRef, '') }}:
parameters: dependencySource: staging
# https://github.com/ROCm/HIP/issues/2203 ${{ elseif ne(parameters.checkoutRef, '') }}:
extraBuildFlags: >- dependencySource: tag-builds
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ - task: Bash@3
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm displayName: Unload and reload AMDGPU
-DROCM_ROOT=$(Agent.BuildDirectory)/rocm inputs:
-DAMDGPU_TARGETS=${{ job.target }} targetType: inline
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }} script: |
-DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc sudo modprobe -r amdgpu
-GNinja sudo modprobe amdgpu
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml - task: Bash@3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml displayName: Iterate through examples
parameters: inputs:
componentName: rocm-examples targetType: inline
testDir: $(Build.SourcesDirectory)/build script: |
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml' for file in *; do
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml echo Now running: $file
parameters: ./$file | tee -a $(TEST_LOG_FILE)
aptPackages: ${{ parameters.aptPackages }} done
environment: test workingDirectory: $(Agent.BuildDirectory)/rocm/examples
gpuTarget: ${{ job.target }}

View File

@@ -1,181 +0,0 @@
parameters:
- name: componentName
type: string
default: rocm_libraries
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- ccache
- gfortran
- git
- libdrm-dev
- liblapack-dev
- libmsgpack-dev
- libnuma-dev
- libopenblas-dev
- ninja-build
- python3-pip
- python3-venv
- name: pipModules
type: object
default:
- joblib
- "packaging>=22.0"
- pytest
- pytest-cmake
- --upgrade
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocminfo
- rocm-cmake
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { pool: rocm-ci_ultra_build_pool, os: ubuntu2204, packageManager: apt, target: gfx942 }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 300
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: DAY_STRING
value: $[format('{0:ddMMyyyy}', pipeline.startTime)]
pool: ${{ job.pool }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- script: |
mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache"
displayName: Update path for ccache
- task: Cache@2
displayName: Ccache caching
inputs:
key: rocm-libraries | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR)
restoreKeys: |
rocm-libraries | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING)
rocm-libraries | ${{ job.os }} | ${{ job.target }}
rocm-libraries | ${{ job.os }}
- task: Bash@3
displayName: Add paths for CMake and Python site-packages binaries
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.setvariable variable=PytestCmakePath]$USER_BASE/share/Pytest/cmake"
displayName: Set cmake configure paths
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
extraBuildFlags: >-
-D CMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor;$(PytestCmakePath)
-D CMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/llvm/include
-D CMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-D CMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-D CMAKE_CXX_COMPILER_LAUNCHER=ccache
-D CMAKE_C_COMPILER_LAUNCHER=ccache
-G Ninja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
installLatestCMake: true
extraCopyDirectories:
- deps

View File

@@ -5,12 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -33,21 +27,11 @@ parameters:
- name: rocmTestDependencies - name: rocmTestDependencies
type: object type: object
default: default:
- rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocm_bandwidth_test_build - job: rocm_bandwidth_test
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -70,9 +54,13 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -80,32 +68,22 @@ jobs:
-DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules -DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/include;$(Agent.BuildDirectory)/rocm/include/hsa -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/include;$(Agent.BuildDirectory)/rocm/include/hsa
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# extraEnvVars:
# - ROCR_INC_DIR:::/home/user/workspace/rocm
# - ROCR_LIB_DIR:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocm_bandwidth_test_testing
- job: rocm_bandwidth_test_test_${{ job.target }} dependsOn: rocm_bandwidth_test
dependsOn: rocm_bandwidth_test_build condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -115,10 +93,12 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocm_bandwidth_test componentName: rocm_bandwidth_test
@@ -126,9 +106,3 @@ jobs:
testExecutable: './rocm/bin/rocm-bandwidth-test' testExecutable: './rocm/bin/rocm-bandwidth-test'
testParameters: '' testParameters: ''
testPublishResults: false testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,145 +1,52 @@
parameters: parameters:
- name: componentName
type: string
default: rocm_smi_lib
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-dev
- ninja-build
- pkg-config
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_smi_lib
- job: rocm_smi_lib_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
vmImage: 'ubuntu-24.04'
${{ else }}:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DBUILD_TESTS=ON -DBUILD_TESTS=ON
-DROCM_DEP_ROCMCORE=ON -DROCM_DEP_ROCMCORE=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocm_smi_lib_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocm_smi_lib
- job: rocm_smi_lib_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: rocm_smi_lib_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocm_smi_lib
testDir: '$(Agent.BuildDirectory)' testDir: '$(Agent.BuildDirectory)'
testExecutable: 'sudo ./rocm/share/rocm_smi/rsmitst_tests/rsmitst' testExecutable: './rocm/share/rocm_smi/rsmitst_tests/rsmitst'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,41 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: rocminfo
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-amdgpu-dev
- libdrm-dev
- ninja-build
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -47,118 +16,69 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocminfo
- job: ${{ parameters.componentName }}_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} skipLlvmSymlink: true
os: ${{ job.os }} # CI case: download latest default branch build
${{ if parameters.triggerDownstreamJobs }}: ${{ if eq(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCRTST_BLD_TYPE=release -DROCRTST_BLD_TYPE=release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocminfo_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocminfo
- job: rocminfo_test_${{ job.target }} condition: succeeded()
dependsOn: rocminfo_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocminfo
testDir: '$(Agent.BuildDirectory)' testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/rocminfo' testExecutable: './rocm/bin/rocminfo'
testParameters: '' testParameters: ''
testPublishResults: false testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocm_agent_enumerator componentName: rocm_agent_enumerator
@@ -166,10 +86,3 @@ jobs:
testExecutable: './rocm/bin/rocm_agent_enumerator' testExecutable: './rocm/bin/rocm_agent_enumerator'
testParameters: '' testParameters: ''
testPublishResults: false testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,221 +0,0 @@
parameters:
- name: componentName
type: string
default: rocprofiler-compute
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdw-dev
- libtbb-dev
- locales
- ninja-build
- python3-pip
- name: pipModules
type: object
default:
- astunparse==1.6.2
- colorlover
- dash-bootstrap-components
- dash-svg
- "dash>=3.0.0"
- kaleido==0.2.1
- matplotlib
- "numpy>=1.17.5"
- "pandas>=1.4.3"
- plotext
- plotille
- pymongo
- pyyaml
- setuptools
- sqlalchemy
- tabulate
- textual
- textual_plotext
- textual-fspicker
- tqdm
- mock
- pytest
- pytest-cov
- pytest-xdist
- name: rocmTestDependencies
type: object
default:
- amdsmi
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- rocm_smi_lib
- ROCR-Runtime
- rocprofiler
- rocprofiler-sdk
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_compute_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_compute_test_${{ job.target }}
timeoutInMinutes: 120
dependsOn: rocprofiler_compute_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTHON_VERSION
value: 3.10
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add en_US.UTF-8 locale
inputs:
targetType: inline
script: |
sudo locale-gen en_US.UTF-8
sudo update-locale
locale -a
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }}
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DENABLE_TESTS=ON
-DINSTALL_TESTS=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Build.BinariesDirectory)/libexec/rocprofiler-compute
testExecutable: ROCM_PATH=$(Agent.BuildDirectory)/rocm ctest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,104 +1,37 @@
parameters: parameters:
- name: componentName
type: string
default: rocprofiler-register
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocprofiler_register
- job: rocprofiler_register_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
vmImage: 'ubuntu-24.04'
${{ else }}:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocprofiler-register
os: ${{ job.os }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
useAmdclang: false parameters:
componentName: rocprofiler-register-tests
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Build.BinariesDirectory) -DCMAKE_PREFIX_PATH=$(Build.BinariesDirectory)
-DROCPROFILER_REGISTER_BUILD_TESTS=ON cmakeBuildDir: 'tests/build'
-DROCPROFILER_REGISTER_BUILD_SAMPLES=ON installEnabled: false
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: rocprofiler-register
testDir: $(Agent.BuildDirectory)/s/build testDir: 'tests/build'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: combined

View File

@@ -1,223 +0,0 @@
parameters:
- name: componentName
type: string
default: rocprofiler-sdk
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- build-essential
- cmake
- libdrm-amdgpu-dev
- libdrm-dev
- libdw-dev
- libelf-dev
- libsqlite3-dev
- libva-dev
- ninja-build
- pkg-config
- python3-pip
- name: pipModules
type: object
default:
- black
- clang-format
- clang-tidy
- cmake
- cmake-format
- dataclasses
- numpy
- otf2
- pandas
- perfetto
- pycobertura
- "pytest>=6.2.5"
- pyyaml
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rccl
- rocDecode
- rocJPEG
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_sdk_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFILER_BUILD_TESTS=ON
-DROCPROFILER_BUILD_SAMPLES=ON
-DROCPROFILER_BUILD_RELEASE=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_sdk_test_${{ job.target }}
dependsOn: rocprofiler_sdk_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add Python and ROCm binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFILER_BUILD_TESTS=ON
-DROCPROFILER_BUILD_SAMPLES=ON
-DROCPROFILER_BUILD_RELEASE=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH}}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/s/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -1,271 +0,0 @@
# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: componentName
type: string
default: rocprofiler-systems
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- autoconf
- autotools-dev
- bison
- build-essential
- bzip2
- clang
- cmake
- chrpath
- environment-modules
- ffmpeg
- g++-12
- libavcodec-dev
- libavformat-dev
- libavutil-dev
- libdrm-amdgpu-dev
- libdrm-dev
- libdw-dev
- libfabric-dev
- libiberty-dev
- libpapi-dev
- libpfm4-dev
- libtool
- libopenmpi-dev
- libsqlite3-dev
- m4
- ninja-build
- openmpi-bin
- pkg-config
- python3-pip
- software-properties-common
- texinfo
- zlib1g-dev
- name: pipModules
type: object
default:
- numpy
- perfetto
- dataclasses
- name: rocmDependencies
type: object
default:
- amdsmi
- aomp
- clr
- llvm-project
- rccl
- rocDecode
- rocJPEG
- rocm-core
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_systems_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DROCPROFSYS_BUILD_TESTING=ON
-DROCPROFSYS_BUILD_DYNINST=ON
-DROCPROFSYS_BUILD_LIBUNWIND=ON
-DROCPROFSYS_DISABLE_EXAMPLES="openmp-target"
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DROCPROFSYS_USE_PAPI=ON
-DROCPROFSYS_USE_MPI=ON
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/include/rocjpeg
-DGPU_TARGETS=${{ job.target }}
-GNinja
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
registerROCmPackages: true
extraPaths: /home/user/workspace/rocm/bin:/home/user/workspace/rocm/llvm/bin
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_systems_test_${{ job.target }}
dependsOn: rocprofiler_systems_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
timeoutInMinutes: 180
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
cmakeSourceDir: $(Agent.BuildDirectory)/s/projects/rocprofiler-systems
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocprofiler-systems
-DROCPROFSYS_USE_PYTHON=ON
-DROCPROFSYS_BUILD_TESTING=ON
-DROCPROFSYS_BUILD_DYNINST=ON
-DROCPROFSYS_BUILD_LIBUNWIND=ON
-DROCPROFSYS_DISABLE_EXAMPLES="openmp-target"
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DROCPROFSYS_USE_PAPI=ON
-DROCPROFSYS_USE_MPI=ON
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/include/rocjpeg
-DGPU_TARGETS=${{ job.target }}
-GNinja
- task: Bash@3
displayName: Set up rocprofiler-systems env
inputs:
targetType: inline
script: source $(Agent.BuildDirectory)/rocprofiler-systems/share/rocprofiler-systems/setup-env.sh
workingDirectory: $(Agent.BuildDirectory)/rocprofiler-systems/share/rocprofiler-systems
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/s/build/tests/
testParameters: '--output-on-failure'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
registerROCmPackages: true
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/bin:/home/user/workspace/rocm/llvm/bin

View File

@@ -1,39 +1,15 @@
parameters: parameters:
- name: componentName
type: string
default: rocprofiler
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- libgtest-dev
- libdrm-dev - libdrm-dev
- libdw-dev - libdw-dev
- libsystemd-dev - libsystemd-dev
@@ -44,13 +20,13 @@ parameters:
- name: pipModules - name: pipModules
type: object type: object
default: default:
- barectf
- Cppheaderparser
- lxml
- matplotlib
- pandas
- pyyaml==5.3.1 - pyyaml==5.3.1
- Cppheaderparser
- websockets - websockets
- matplotlib
- lxml
- barectf
- pandas
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -59,37 +35,14 @@ parameters:
- ROCdbgapi - ROCdbgapi
- rocm-cmake - rocm-cmake
- rocm-core - rocm-core
- rocminfo
- rocm_smi_lib - rocm_smi_lib
- rocprofiler-register - rocminfo
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocprofiler
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -98,136 +51,106 @@ jobs:
- name: ROCM_PATH - name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} # CI case: download latest default branch build
aggregatePipeline: ${{ parameters.aggregatePipeline }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules;$(Agent.BuildDirectory)/rocm/lib/cmake;$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/lib64/cmake;$(Agent.BuildDirectory)/rocm/lib64/cmake/hip -DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules;$(Agent.BuildDirectory)/rocm/lib/cmake;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DENABLE_LDCONFIG=OFF -DENABLE_LDCONFIG=OFF
-DUSE_PROF_API=1 -DUSE_PROF_API=1
-DGPU_TARGETS=${{ job.target }} -DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
multithreadFlag: -- -j32 multithreadFlag: -- -j32
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ROCM_PATH:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: rocprofiler_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: rocprofiler
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH pool: $(JOB_TEST_POOL)
value: $(Agent.BuildDirectory)/rocm
- name: LD_LIBRARY_PATH
value: $(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1/test:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests
pool: ${{ job.target }}_test_pool
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters: parameters:
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocprofilerV1 componentName: rocprofilerV1
testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1/test" ./run.sh
testDir: $(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1 testDir: $(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1
testExecutable: ./run.sh
testParameters: '' testParameters: ''
testPublishResults: false testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocprofilerV2 componentName: rocprofilerV2
testDir: $(Agent.BuildDirectory)/rocm testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests" share/rocprofiler/tests/runUnitTests
testExecutable: share/rocprofiler/tests/runUnitTests
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }} testDir: $(Agent.BuildDirectory)/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -19,7 +14,6 @@ parameters:
- libdw-dev - libdw-dev
- libstdc++-12-dev - libstdc++-12-dev
- python-is-python3 - python-is-python3
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -40,17 +34,8 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocr_debug_agent_build - job: rocr_debug_agent
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -68,9 +53,13 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -79,28 +68,22 @@ jobs:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}: - job: rocr_debug_agent_testing
- job: rocr_debug_agent_test_${{ job.target }} dependsOn: rocr_debug_agent
dependsOn: rocr_debug_agent_build condition: succeeded()
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -108,11 +91,19 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: rocr_debug_agent-tests componentName: rocr_debug_agent-tests
@@ -123,13 +114,7 @@ jobs:
cmakeBuildDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test' cmakeBuildDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test'
cmakeSourceDir: '.' cmakeSourceDir: '.'
installEnabled: false installEnabled: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: rocr_debug_agent componentName: rocr_debug_agent
testDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test' testDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,43 +1,17 @@
parameters: parameters:
- name: componentName
type: string
default: roctracer
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build
- doxygen - doxygen
- graphviz - graphviz
- libdrm-amdgpu-dev
- ninja-build
- zlib1g-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
@@ -56,150 +30,98 @@ parameters:
default: default:
- clr - clr
- llvm-project - llvm-project
- rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: roctracer
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: HIP_ROCCLR_HOME
value: $(Build.BinariesDirectory)/rocm
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }} pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} # manual build case: triggered by ROCm/ROCm repo
# the linker flags will not affect ubuntu2204 builds as the paths do not exist ${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=release -DCMAKE_BUILD_TYPE=release
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/lib64/cmake/hip -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DGPU_TARGETS=${{ job.target }} -DGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ if eq(parameters.unifiedBuild, False) }}: - job: roctracer_testing
- ${{ each job in parameters.jobMatrix.testJobs }}: dependsOn: roctracer
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }} condition: succeeded()
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool pool: $(JOB_TEST_POOL)
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters: parameters:
preTargetFilter: ${{ parameters.componentName }} gpuTarget: $(JOB_GPU_TARGET)
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
os: ${{ job.os }} ${{ if eq(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: dependencySource: staging
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} ${{ elseif ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} componentName: roctracer
testExecutable: $(Agent.BuildDirectory)/rocm/share/roctracer/run_tests.sh testExecutable: $(Agent.BuildDirectory)/rocm/share/roctracer/run_tests.sh
testParameters: '' testParameters: ''
testDir: $(Agent.BuildDirectory) testDir: $(Agent.BuildDirectory)
testPublishResults: false testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -5,28 +5,12 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- clang
- cmake - cmake
- imagemagick
- libopencv-dev
- libsndfile1-dev
- libstdc++-12-dev
- ninja-build - ninja-build
- python3-pip - clang
- name: pipModules
type: object
default:
- openpyxl
- pandas
- sphinx
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -34,48 +18,24 @@ parameters:
- clr - clr
- half - half
- llvm-project - llvm-project
- rocm-cmake
- rocminfo - rocminfo
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- half
- hipTensor
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rpp
- job: rpp_build_${{ job.target }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
@@ -86,115 +46,25 @@ jobs:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
aggregatePipeline: ${{ parameters.aggregatePipeline }} # CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include -DHALF_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DROCM_PLATFORM_VERSION=$(NEXT_RELEASE_VERSION)
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
gpuTarget: ${{ job.target }} gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rpp_test_${{ job.target }}
dependsOn: rpp_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
# Dependencies from: https://github.com/ROCm/rpp/blob/develop/utilities/test_suite/README.md
- task: Bash@3
displayName: Build and install Turbo JPEG
inputs:
targetType: 'inline'
script: |
sudo apt-get install nasm
sudo apt-get install wget
git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git
cd libjpeg-turbo
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_BUILD_TYPE=RELEASE \
-DENABLE_STATIC=FALSE \
-DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \
-DWITH_JPEG8=TRUE \
..
make -j$nproc
sudo make install
- task: Bash@3
displayName: Build and install Nifti
inputs:
targetType: 'inline'
script: |
git clone -b v3.0.1 https://github.com/NIFTI-Imaging/nifti_clib.git
cd nifti_clib
mkdir build
cd build
cmake ..
sudo make -j$nproc install
- task: Bash@3
displayName: Build rpp tests
inputs:
targetType: 'inline'
script: |
mkdir rpp-tests
cd rpp-tests
cmake $(Agent.BuildDirectory)/rocm/share/rpp/test \
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ \
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rpp
testExecutable: 'CMAKE_VERBOSE_MAKEFILE=ON VERBOSE=1 ctest'
testDir: 'rpp-tests'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,64 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: boostVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- git
- name: rocmDependencies
type: object
default:
- llvm-project
jobs:
- job: boost
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: 'git clone boost'
inputs:
targetType: inline
script: git clone -b ${{ parameters.boostVersion }} https://github.com/boostorg/boost --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- task: Bash@3
displayName: 'Build Boost with clang'
inputs:
targetType: inline
script: |
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
./bootstrap.sh --with-toolset=clang --prefix=$(Build.BinariesDirectory)
./b2 --toolset=clang threading=multi link=shared --prefix=$(Build.BinariesDirectory) cxxflags="-std=c++20"
./b2 install
workingDirectory: $(Build.SourcesDirectory)/boost
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -1,63 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: catch2Version
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: catch2_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone catch2 ${{ parameters.catch2Version }}
inputs:
targetType: inline
script: git clone https://github.com/catchorg/Catch2.git -b ${{ parameters.catch2Version }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/Catch2/build
cmakeSourceDir: $(Agent.BuildDirectory)/Catch2
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,67 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: fmtlibVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- libfmt-dev
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: fmtlib_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone fmtlib ${{ parameters.fmtlibVersion }}
inputs:
targetType: inline
script: git clone https://github.com/fmtlib/fmt.git -b ${{ parameters.fmtlibVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/fmt/build
cmakeSourceDir: $(Agent.BuildDirectory)/fmt
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DFMT_SYSTEM_HEADERS=ON
-DFMT_INSTALL=ON
-DFMT_TEST=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -38,7 +38,6 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
cmakeBuildDir: $(Agent.BuildDirectory)/grpc/build cmakeBuildDir: $(Agent.BuildDirectory)/grpc/build
cmakeSourceDir: $(Agent.BuildDirectory)/grpc
extraBuildFlags: >- extraBuildFlags: >-
-DgRPC_INSTALL=ON -DgRPC_INSTALL=ON
-DgRPC_BUILD_TESTS=OFF -DgRPC_BUILD_TESTS=OFF

View File

@@ -11,54 +11,35 @@ parameters:
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git - git
- cmake
- ninja-build - ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: gtest
- job: gtest_${{ job.os }}
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: pool:
vmImage: 'ubuntu-22.04' vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace: workspace:
clean: all clean: all
steps: steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3 - task: Bash@3
displayName: Clone GTest ${{ parameters.gtestVersion }} displayName: 'git clone gtest'
inputs: inputs:
targetType: inline targetType: inline
script: git clone https://github.com/google/googletest -b ${{ parameters.gtestVersion }} --depth=1 --shallow-submodules --recurse-submodules script: git clone -b ${{ parameters.gtestVersion }} https://github.com/google/googletest --depth=1 --shallow-submodules --recurse-submodules
workingDirectory: $(Agent.BuildDirectory) workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/googletest/build cmakeBuildDir: $(Agent.BuildDirectory)/googletest/build
cmakeSourceDir: $(Agent.BuildDirectory)/googletest
useAmdclang: false
extraBuildFlags: >- extraBuildFlags: >-
-DGTEST_FORCE_SHARED_CRT=ON -DGTEST_FORCE_SHARED_CRT=ON
-DCMAKE_DEBUG_POSTFIX=d -DCMAKE_DEBUG_POSTFIX=d
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -19,7 +19,7 @@ jobs:
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
container: container:
image: rocm/dev-ubuntu-22.04:${{ variables.LATEST_RELEASE_VERSION }} image: ${{ variables.DOCKER_IMAGE_NAME }}:${{ variables.LATEST_DOCKER_VERSION }}
workspace: workspace:
clean: all clean: all
steps: steps:

View File

@@ -1,64 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: libdivideVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: libdivide_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone libdivide ${{ parameters.libdivideVersion }}
inputs:
targetType: inline
script: git clone https://github.com/ridiculousfish/libdivide.git -b ${{ parameters.libdivideVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/libdivide/build
cmakeSourceDir: $(Agent.BuildDirectory)/libdivide
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DLIBDIVIDE_BUILD_TESTS=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,71 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: spdlogVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: spdlog_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- fmtlib
- task: Bash@3
displayName: Clone spdlog ${{ parameters.spdlogVersion }}
inputs:
targetType: inline
script: git clone https://github.com/gabime/spdlog.git -b ${{ parameters.spdlogVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/spdlog/build
cmakeSourceDir: $(Agent.BuildDirectory)/spdlog
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/vendor
-DCMAKE_BUILD_TYPE=Release
-DSPDLOG_USE_STD_FORMAT=OFF
-DSPDLOG_FMT_EXTERNAL_HO=ON
-DSPDLOG_INSTALL=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -4,80 +4,78 @@ parameters:
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- autoconf - build-essential
- git
- ninja-build
- openjdk-8-jdk
- ca-certificates
- bc - bc
- bridge-utils - bridge-utils
- build-essential - cmake
- ca-certificates
- ccache
- devscripts - devscripts
- dkms - dkms
- doxygen - doxygen
- fakeroot
- ffmpeg
- gfortran
- git
- gnutls-bin
- libamd2
- libavformat-dev
- libblas3
- libcamd2
- libccolamd2
- libcholmod3
- libcolamd2
- libdpkg-dev - libdpkg-dev
- libdpkg-perl - libdpkg-perl
- libdrm-amdgpu1
- libdrm-dev
- libelf-dev - libelf-dev
- libfreetype-dev
- libgfortran5
- libgomp1
- libjpeg-dev
- libjpeg-turbo-official
- liblapack-dev
- liblapack3
- libmetis5
- libncurses-dev
- libnuma-dev
- libopenblas-dev
- libpth-dev
- libquadmath0
- libssh-dev
- libstdc++-12-dev
- libsuitesparseconfig5
- libswscale-dev
- libtinfo-dev
- libunwind-dev
- libwebp-dev
- llvm-dev
- ncurses-base
- ninja-build
- numactl
- openjdk-8-jdk
- python-is-python3
- python3-dev - python3-dev
- python3-pip - python3-pip
- python3-venv - python3-venv
- wget
- ncurses-base
- libncurses-dev
- numactl
- libnuma-dev
- libssh-dev
- libunwind-dev
- llvm-dev
- libpth-dev
- qemu-kvm - qemu-kvm
- re2c - re2c
- subversion - subversion
- wget - fakeroot
- autoconf
- libgomp1
- libtinfo-dev
- libcholmod3
- libsuitesparseconfig5
- libstdc++-12-dev
- python-is-python3
- gfortran
- libgfortran5
- liblapack3
- libblas3
- libquadmath0
- libmetis5
- libamd2
- libcamd2
- libcolamd2
- libccolamd2
- libdrm-amdgpu1
- ccache
- zip - zip
- libjpeg-turbo-official
- libjpeg-dev
- libwebp-dev
- libfreetype-dev
- gnutls-bin
- ffmpeg
- libopenblas-dev
- liblapack-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- astunparse - astunparse
- "expecttest>=0.3.0" - expecttest!=0.2.0
- hypothesis - hypothesis
- numpy - numpy
- psutil - psutil
- pyyaml - pyyaml
- requests - requests
- setuptools==75.8.0 - setuptools
- types-dataclasses - types-dataclasses
- "typing-extensions>=4.10.0" - typing-extensions>=4.8.0
- "sympy>=1.13.3" - sympy>=1.13.0
- filelock - filelock
- networkx - networkx
- jinja2 - jinja2
@@ -85,70 +83,40 @@ parameters:
- lintrunner - lintrunner
- ninja - ninja
- packaging - packaging
- "optree>=0.13.0" - optree>=0.12.0
- "click>=8.0.3"
# list for vision # list for vision
- auditwheel - auditwheel
- future - future
- pytest # list from https://github.com/pytorch/builder/blob/main/manywheel/build_rocm.sh
- pytest-azurepipelines
- pillow
# list from https://github.com/pytorch/pytorch/blob/main/.ci/manywheel/build_rocm.sh
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocminfo
- MIOpen
- clr - clr
- hipBLAS - hipBLAS
- hipBLASLt
- hipFFT - hipFFT
- hipRAND - hipRAND
- hipSOLVER - hipSOLVER
- hipSPARSE - hipSPARSE
- hipSPARSELt - ROCR-Runtime
- llvm-project - llvm-project
- MIOpen
- rccl - rccl
- rocBLAS - rocBLAS
- rocFFT - rocFFT
- rocm-core
- rocminfo
- rocm_smi_lib - rocm_smi_lib
- rocPRIM
- rocprofiler-register
- rocRAND - rocRAND
- ROCR-Runtime
- rocSOLVER - rocSOLVER
- rocSPARSE - rocSPARSE
- roctracer - roctracer
- hipBLASLt
- rocprofiler-register
- rocm-core
- rocPRIM
# below are additional dependencies not called out by build script, but throw errors during cmake # below are additional dependencies not called out by build script, but throw errors during cmake
- composable_kernel
- hipBLAS-common
- hipCUB - hipCUB
- rocThrust - rocThrust
- name: rocmTestDependencies - hipBLAS-common
type: object
default:
# rocroller.so needed and is not included in the wheel
- hipBLASLt
- rocminfo
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_pytorch_test1.sh#L54
# Will iterate through this list using pytest
- name: torchTestList
type: object
default:
- nn
- torch
# - cuda seg faults and might need cuda installed on test system
# - ops takes too long
- unary_ufuncs
- binary_ufuncs
- autograd
# - inductor/torchinductor takes too long
# set to false to disable torchvision build and test
- name: includeVision
type: boolean
default: false
trigger: none trigger: none
pr: none pr: none
@@ -168,18 +136,23 @@ jobs:
amd-staging-gfx942: amd-staging-gfx942:
ROCM_BRANCH: amd-staging ROCM_BRANCH: amd-staging
JOB_GPU_TARGET: gfx942 JOB_GPU_TARGET: gfx942
amd-staging-gfx90a:
ROCM_BRANCH: amd-staging
JOB_GPU_TARGET: gfx90a
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
# various flags/parameters expected by bash scripts in pytorch repo's .ci directory # various flags/parameters expected by bash scripts in pytorch builder repo
- name: ROCM_VERSION - name: ROCM_VERSION
value: 6.5.0 value: 6.3.0
- name: ROCM_PATH - name: ROCM_PATH
value: /opt/rocm value: /opt/rocm
- name: DESIRED_CUDA - name: DESIRED_CUDA
value: 6.5.0 value: 6.3.0
- name: MAGMA_ROCM - name: MKLROOT
value: 6.3 value: /opt/intel
- name: AOTRITON_INSTALLED_PREFIX
value: /opt/rocm/aotriton
- name: DESIRED_PYTHON - name: DESIRED_PYTHON
value: 3.10 value: 3.10
- name: PYTORCH_ROOT - name: PYTORCH_ROOT
@@ -190,7 +163,7 @@ jobs:
workspace: workspace:
clean: all clean: all
steps: steps:
# copy environment setup from https://github.com/pytorch/pytorch/blob/main/.ci/docker/manywheel/Dockerfile # copy environment setup from https://github.com/pytorch/builder/blob/main/manywheel/Dockerfile
# but instead of centos, use ubuntu environment # but instead of centos, use ubuntu environment
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- task: Bash@3 - task: Bash@3
@@ -219,29 +192,26 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
gpuTarget: $(JOB_GPU_TARGET) gpuTarget: $(JOB_GPU_TARGET)
setupHIPLibrarySymlinks: true
- task: Bash@3 - task: Bash@3
displayName: ROCm symbolic link displayName: ROCm symbolic link
inputs: inputs:
targetType: inline targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- checkout: self - checkout: self
- task: Bash@3
displayName: git clone pytorch builder
inputs:
targetType: inline
script: git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3 - task: Bash@3
displayName: git clone upstream pytorch displayName: git clone upstream pytorch
inputs: inputs:
targetType: inline targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
# builder clone still needed due to run_tests.sh at end of build_common.sh call
- task: Bash@3
displayName: git clone pytorch builder
inputs:
targetType: inline
script: |
git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules
sudo ln -s $(Build.SourcesDirectory)/builder /builder
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3 - task: Bash@3
displayName: Install patchelf displayName: Install patchelf
inputs: inputs:
@@ -249,6 +219,13 @@ jobs:
script: | script: |
sudo bash pytorch/.ci/docker/common/install_patchelf.sh sudo bash pytorch/.ci/docker/common/install_patchelf.sh
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install mkl dependency for magma
inputs:
targetType: inline
script: |
sudo bash pytorch/.ci/docker/common/install_mkl.sh
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3 - task: Bash@3
displayName: Install rocm drm displayName: Install rocm drm
inputs: inputs:
@@ -261,38 +238,39 @@ jobs:
inputs: inputs:
targetType: inline targetType: inline
script: | script: |
sudo bash pytorch/.ci/docker/common/install_rocm_magma.sh $(MAGMA_ROCM) sudo PYTORCH_ROCM_ARCH=$(JOB_GPU_TARGET) MKLROOT=$(MKLROOT) bash pytorch/.ci/docker/common/install_rocm_magma.sh
workingDirectory: $(Build.SourcesDirectory) workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3 - task: Bash@3
displayName: Install targeted typing_extensions for build displayName: Install AOTriton Shared Library
inputs: inputs:
targetType: inline targetType: inline
script: pip install --target=$(Build.SourcesDirectory)/pytorch/torch/.. typing_extensions script: sudo bash ./common/install_aotriton.sh /opt/rocm
workingDirectory: $(Build.SourcesDirectory)/pytorch/.ci/docker
- task: Bash@3 - task: Bash@3
displayName: Run ROCm Build Script displayName: Run ROCm Build Script
inputs: inputs:
targetType: inline targetType: inline
script: >- script: >-
sudo sudo
PYTHONPATH=/home/AzDevOps/.local/lib/python3.10/site-packages:/usr/lib/python3/dist-packages
DESIRED_CUDA=$(DESIRED_CUDA) DESIRED_CUDA=$(DESIRED_CUDA)
PYTORCH_ROCM_ARCH=$(JOB_GPU_TARGET) PYTORCH_ROCM_ARCH=$(JOB_GPU_TARGET)
GPU_TARGET=$(JOB_GPU_TARGET) GPU_TARGET=$(JOB_GPU_TARGET)
DESIRED_PYTHON=$(DESIRED_PYTHON) DESIRED_PYTHON=$(DESIRED_PYTHON)
PYTORCH_ROOT=$(PYTORCH_ROOT) PYTORCH_ROOT=$(PYTORCH_ROOT)
CMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm CMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
AOTRITON_INSTALLED_PREFIX=$(AOTRITON_INSTALLED_PREFIX)
DESIRED_DEVTOOLSET=$(DESIRED_DEVTOOLSET) DESIRED_DEVTOOLSET=$(DESIRED_DEVTOOLSET)
TORCH_PACKAGE_NAME=torch.$(ROCM_BRANCH).$(JOB_GPU_TARGET)
PYTORCH_BUILD_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1) PYTORCH_BUILD_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)
PYTORCH_BUILD_NUMBER=$(date -u +%Y%m%d) PYTORCH_BUILD_NUMBER=$(date -u +%Y%m%d)
SKIP_ALL_TESTS=1 SKIP_ALL_TESTS=1
bash ./.ci/manywheel/build_rocm.sh bash ./manywheel/build_rocm.sh
workingDirectory: $(Build.SourcesDirectory)/pytorch workingDirectory: $(Build.SourcesDirectory)/builder
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters: parameters:
sourceDir: /remote/wheelhouserocm$(ROCM_VERSION) sourceDir: /remote/wheelhouserocm$(ROCM_VERSION)
contentsString: '*.whl' contentsString: '*.whl'
# common helper source for pytorch vision and audio # common helper source for pytorch vision and audio
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3 - task: Bash@3
displayName: git clone pytorch test-infra displayName: git clone pytorch test-infra
inputs: inputs:
@@ -322,8 +300,7 @@ jobs:
inputs: inputs:
targetType: inline targetType: inline
script: >- script: >-
PYTORCH_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)post$(date -u +%Y%m%d) BUILD_VERSION=$(ROCM_BRANCH).$(JOB_GPU_TARGET)-$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)$(date -u +%Y%m%d)
BUILD_VERSION=$(cat $(Build.SourcesDirectory)/vision/version.txt | cut -da -f1)post$(date -u +%Y%m%d)
python3 setup.py bdist_wheel python3 setup.py bdist_wheel
workingDirectory: $(Build.SourcesDirectory)/vision workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3 - task: Bash@3
@@ -337,215 +314,8 @@ jobs:
sourceDir: $(Build.SourcesDirectory)/vision/dist sourceDir: $(Build.SourcesDirectory)/vision/dist
contentsString: '*.whl' contentsString: '*.whl'
clean: false clean: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- task: PublishPipelineArtifact@1 - task: PublishPipelineArtifact@1
displayName: 'wheel file Publish' displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3 retryCountOnTaskFailure: 3
inputs: inputs:
targetPath: $(Build.BinariesDirectory) targetPath: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Save pipeline artifact file name
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.BinariesDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- job: pytorch_testing
dependsOn: pytorch
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTORCH_TEST_WITH_ROCM
value: 1
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
# pytorch tests require an updated version of click, even if requirements is not called outright
- task: Bash@3
displayName: 'pip update click'
inputs:
targetType: inline
script: pip install --upgrade click
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# get sources to run test scripts
- task: Bash@3
displayName: git clone upstream pytorch
inputs:
targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3
displayName: git clone pytorch vision
inputs:
targetType: inline
script: git clone https://github.com/pytorch/vision.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip install --no-index --find-links=. --no-dependencies -v {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Show Updated pip List
inputs:
targetType: inline
script: pip list -v
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- task: Bash@3
displayName: Add torch libs to ldconfig
inputs:
targetType: inline
script: |
echo $(python3 -m site --user-site)/torch/lib | sudo tee /etc/ld.so.conf.d/torch.conf
sudo ldconfig -v
ldconfig -p
# https://pytorch.org/get-started/locally/#linux-verification
# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#testing-the-pytorch-installation
- task: Bash@3
displayName: Simple Import Torch Tests
inputs:
targetType: inline
script: |
python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'
python3 -c 'import torch; x = torch.rand(5, 3); print(x)'
# Test artifact build script has too many if statements for different environments
# Based off the snippet of interest for this environment, with some adjustments
# https://github.com/pytorch/pytorch/blob/main/.ci/pytorch/build.sh#L330-L371
# Removing in-line comments since it does not fit with the yaml markup
- task: Bash@3
displayName: Build Pytorch Test Artifacts
continueOnError: true
inputs:
targetType: inline
script: |
CUSTOM_TEST_ARTIFACT_BUILD_DIR="build/custom_test_artifacts"
CUSTOM_TEST_USE_ROCM=ON
CUSTOM_TEST_MODULE_PATH="${PWD}/cmake/public"
mkdir -pv "${CUSTOM_TEST_ARTIFACT_BUILD_DIR}"
CUSTOM_OP_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-op-build"
CUSTOM_OP_TEST="${PWD}/test/custom_operator"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$CUSTOM_OP_BUILD"
pushd "$CUSTOM_OP_BUILD"
cmake "$CUSTOM_OP_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
JIT_HOOK_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/jit-hook-build"
JIT_HOOK_TEST="$PWD/test/jit_hooks"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$JIT_HOOK_BUILD"
pushd "$JIT_HOOK_BUILD"
cmake "$JIT_HOOK_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
CUSTOM_BACKEND_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-backend-build"
CUSTOM_BACKEND_TEST="${PWD}/test/custom_backend"
python --version
mkdir -p "$CUSTOM_BACKEND_BUILD"
pushd "$CUSTOM_BACKEND_BUILD"
cmake "$CUSTOM_BACKEND_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
workingDirectory: $(Build.SourcesDirectory)/pytorch
- ${{ each torchTest in parameters.torchTestList }}:
- task: Bash@3
displayName: Test ${{ torchTest }}
continueOnError: true
inputs:
targetType: inline
workingDirectory: $(Build.SourcesDirectory)/pytorch
${{ if contains(torchTest, '/') }}:
script: pytest test/${{ split(torchTest, '/')[0] }}/test_${{ split(torchTest, '/')[1] }}.py
${{ else }}:
script: pytest test/test_${{ torchTest }}.py
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_torchvision.sh#L51
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3
displayName: Test vision/transforms
continueOnError: true
inputs:
targetType: inline
script: pytest test/test_transforms.py
workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3
displayName: Uninstall Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip uninstall -y {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Remove Python site-packages binaries from path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"

View File

@@ -1,23 +1,5 @@
parameters: parameters:
# using a parameter instead of a strategy matrix allows job variables to be eval'd at compile-time rather than runtime # currently excludes clr
- name: jobList
type: object
default:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2404, packageManager: apt, target: gfx942 }
- { os: ubuntu2404, packageManager: apt, target: gfx90a }
- { os: ubuntu2404, packageManager: apt, target: gfx1201 }
- { os: ubuntu2404, packageManager: apt, target: gfx1100 }
- { os: ubuntu2404, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -25,9 +7,9 @@ parameters:
- amdsmi - amdsmi
- aomp-extras - aomp-extras
- aomp - aomp
- clr
- composable_kernel - composable_kernel
- half - half
- HIP
- hip-tests - hip-tests
- hipBLAS - hipBLAS
- hipBLAS-common - hipBLAS-common
@@ -44,6 +26,7 @@ parameters:
- llvm-project - llvm-project
- MIOpen - MIOpen
- MIVisionX - MIVisionX
- omniperf
- rccl - rccl
- rdc - rdc
- rocAL - rocAL
@@ -53,7 +36,6 @@ parameters:
- rocDecode - rocDecode
- rocFFT - rocFFT
- ROCgdb - ROCgdb
- rocJPEG
- rocm-cmake - rocm-cmake
- rocm-core - rocm-core
- rocm-examples - rocm-examples
@@ -63,10 +45,7 @@ parameters:
- rocm_bandwidth_test - rocm_bandwidth_test
- rocm_smi_lib - rocm_smi_lib
- rocPRIM - rocPRIM
- rocprofiler-compute
- rocprofiler-register - rocprofiler-register
- rocprofiler-sdk
- rocprofiler-systems
- rocprofiler - rocprofiler
- rocPyDecode - rocPyDecode
- ROCR-Runtime - ROCR-Runtime
@@ -78,7 +57,6 @@ parameters:
- roctracer - roctracer
- rocWMMA - rocWMMA
- rpp - rpp
- TransferBench
trigger: none trigger: none
pr: none pr: none
@@ -91,15 +69,19 @@ schedules:
always: true always: true
jobs: jobs:
- ${{ each job in parameters.jobList }}: - job: rocm_nightly
- job: nightly_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps: steps:
- task: DeleteFiles@1 - task: DeleteFiles@1
displayName: 'Cleanup checkout space' displayName: 'Cleanup checkout space'
@@ -117,13 +99,13 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
gpuTarget: ${{ job.target }} skipLibraryLinking: true
gpuTarget: $(JOB_GPU_TARGET)
- script: df -h - script: df -h
displayName: System disk space after ROCm displayName: System disk space after ROCm
- script: du -sh $(Agent.BuildDirectory)/rocm - script: du -sh $(Agent.BuildDirectory)/rocm
displayName: Uncompressed ROCm size displayName: Uncompressed ROCm size
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- task: ArchiveFiles@2 - task: ArchiveFiles@2
displayName: Compress rocm-nightly displayName: Compress rocm-nightly
inputs: inputs:
@@ -131,109 +113,11 @@ jobs:
includeRootFolder: false includeRootFolder: false
archiveType: tar archiveType: tar
tarCompression: gz tarCompression: gz
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.DefinitionName)_$(Build.BuildNumber)_${{ job.os }}_${{ job.target }}.tar.gz archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.DefinitionName)_$(Build.BuildNumber)_ubuntu2204_$(JOB_GPU_TARGET).tar.gz
- script: du -sh $(Build.ArtifactStagingDirectory) - script: du -sh $(Build.ArtifactStagingDirectory)
displayName: Compressed ROCm size displayName: Compressed ROCm size
- task: PublishPipelineArtifact@1 - task: PublishPipelineArtifact@1
displayName: 'Publish ROCm Nightly Artifact' displayName: 'Public ROCm Nightly Artifact'
retryCountOnTaskFailure: 3 retryCountOnTaskFailure: 3
inputs: inputs:
targetPath: '$(Build.ArtifactStagingDirectory)' targetPath: '$(Build.ArtifactStagingDirectory)'
- task: Bash@3
displayName: Save pipeline artifact file name
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: echo "$(Build.DefinitionName)_$(Build.BuildNumber)_${{ job.os }}_${{ job.target }}.tar.gz" >> pipelineArtifacts.txt
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.packageManager, 'apt') }}:
- task: Bash@3
displayName: Create Dockerfile
inputs:
workingDirectory: $(Agent.BuildDirectory)
targetType: inline
script: |
cat <<'EOF' > Dockerfile
${{ iif(eq(job.os, 'ubuntu2204'), 'FROM ubuntu:22.04', '') }}
${{ iif(eq(job.os, 'ubuntu2404'), 'FROM ubuntu:24.04', '') }}
WORKDIR /root
RUN mkdir rocm
RUN apt update \
&& apt upgrade -y \
&& apt install -y cmake curl git gcc g++ gpg lsb-release lsof ninja-build pkg-config python3 python3-pip wget zip libdrm-dev libelf-dev libgtest-dev libhsakmt-dev libhwloc-dev libnuma-dev libstdc++-12-dev libtbb-dev jq \
&& apt clean all
RUN PACKAGE_NAME=$(curl -s https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/ | grep -oP "href=\"\K[^\"]*$(lsb_release -rs)[^\"]*\.deb") \
&& wget -nv --retry-connrefused https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/$PACKAGE_NAME \
&& mkdir hsa-amd-aqlprofile \
&& dpkg-deb -R $PACKAGE_NAME hsa-amd-aqlprofile \
&& cp -R hsa-amd-aqlprofile/opt/rocm-*/* rocm
RUN ARTIFACT_URL="https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/$(Build.BuildId)/artifacts?artifactName=nightly${{ job.os }}${{ job.target }}&api-version=7.1" \
&& DOWNLOAD_URL=$(curl -s $ARTIFACT_URL | jq ".resource.downloadUrl" | tr -d '"') \
&& wget -nv --retry-connrefused $DOWNLOAD_URL -O nightly.zip \
&& unzip nightly.zip \
&& tar -xf nightly${{ job.os }}${{ job.target }}/rocm-nightly*${{ job.os }}*${{ job.target }}*.tar.gz -C rocm
RUN echo /root/rocm/lib | tee /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN ldconfig -v
ENV PATH="$PATH:/root/rocm/bin"
ENTRYPOINT ["/bin/bash"]
EOF
cat Dockerfile
- ${{ elseif eq(job.packageManager, 'dnf') }}:
- task: Bash@3
displayName: Create Dockerfile
inputs:
workingDirectory: $(Agent.BuildDirectory)
targetType: inline
script: |
cat <<'EOF' > Dockerfile
${{ iif(eq(job.os, 'almalinux8'), 'FROM almalinux:8', '') }}
WORKDIR /root
RUN mkdir rocm
RUN dnf install -y cmake curl git gcc gcc-c++ gnupg2 redhat-lsb-core lsof pkgconf python3 python3-pip wget zip libdrm-devel elfutils-libelf-devel numactl-devel libstdc++-devel tbb-devel jq \
&& dnf clean all
RUN PACKAGE_NAME=$(curl -s https://repo.radeon.com/rocm/rhel8/$(REPO_RADEON_VERSION)/main/ | grep -oP "hsa-amd-aqlprofile-[^\"]+\.rpm" | head -n1) \
&& wget -nv --retry-connrefused https://repo.radeon.com/rocm/rhel8/$(REPO_RADEON_VERSION)/main/$PACKAGE_NAME \
&& mkdir hsa-amd-aqlprofile \
&& dnf -y install rpm-build cpio \
&& rpm2cpio $PACKAGE_NAME | (cd hsa-amd-aqlprofile && cpio -idmv) \
&& cp -R hsa-amd-aqlprofile/opt/rocm-*/* rocm
RUN ARTIFACT_URL="https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/$(Build.BuildId)/artifacts?artifactName=nightly${{ job.os }}${{ job.target }}&api-version=7.1" \
&& DOWNLOAD_URL=$(curl -s $ARTIFACT_URL | jq ".resource.downloadUrl" | tr -d '"') \
&& wget -nv --retry-connrefused $DOWNLOAD_URL -O nightly.zip \
&& UNZIP_DISABLE_ZIPBOMB_DETECTION=TRUE unzip nightly.zip \
&& tar -xf nightly${{ job.os }}${{ job.target }}/rocm-nightly*${{ job.os }}*${{ job.target }}*.tar.gz -C rocm
RUN echo /root/rocm/lib | tee /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN ldconfig -v
ENV PATH="$PATH:/root/rocm/bin"
ENTRYPOINT ["/bin/bash"]
EOF
cat Dockerfile
- task: Docker@2
displayName: Build and upload Docker image
retryCountOnTaskFailure: 3
inputs:
containerRegistry: ContainerService3
repository: 'nightly-${{ job.os }}-${{ job.target }}'
Dockerfile: '$(Agent.BuildDirectory)/Dockerfile'
buildContext: '$(Agent.BuildDirectory)'
- task: Bash@3
displayName: '!! Docker Run Command !!'
inputs:
targetType: inline
script: echo "docker run -it --network=host --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined rocmexternalcicd.azurecr.io/nightly-${{ job.os }}-${{ job.target }}:$(Build.BuildId)" | tr '[:upper:]' '[:lower:]'

View File

@@ -1,68 +0,0 @@
#!/usr/bin/env python3
import json
import requests
import argparse
from pathlib import Path
def get_builds(entries, gpu_target, output):
already_downloaded = {}
for entry in entries:
already_downloaded = _get_builds(entry, gpu_target, already_downloaded, output)
def _get_builds(entry, gpu_target, already_downloaded, output):
print()
print(f"{entry['buildNumber']} - {entry['buildId']} - {entry['repoName']}")
if already_downloaded.get(entry['buildId']):
print('Skipping, already downloaded from build ' + entry['buildId'])
return already_downloaded
artifacts_url = f"https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/{entry['buildId']}/artifacts?api-version=7.1"
artifacts = requests.get(artifacts_url).json()
for artifact in artifacts['value']:
if 'gfx' in artifact['name'] and gpu_target not in artifact['name']:
continue
print('Artifact name: ' + artifact['name'])
print('File size: ~' +
str(round(int(artifact['resource']['properties']['artifactsize'])/1000000, 2)) + ' MB')
download_url = f"{artifact['resource']['downloadUrl']}"
download = requests.get(download_url)
zip_file = Path(output) / f"{artifact['name']}.zip"
with open(zip_file, 'wb') as f:
f.write(download.content)
already_downloaded[entry['buildId']] = True
return already_downloaded
def main():
parser = argparse.ArgumentParser(description="Command line tool for downloading external ci artifacts")
parser.add_argument('--target', type=str, dest="target", choices=["gfx90a", "gfx942"], help="Target gfx")
parser.add_argument('--manifest', type=str, dest="manifest", help='JSON manifest url or path to local manifest')
parser.add_argument('--output_dir', type=str, dest="output", help='Path to download directory')
args = parser.parse_args()
manifest = args.manifest
gpu_target = args.target
if not gpu_target:
print("Enter the GPU target (gfx942, gfx90a)")
gpu_target = input()
if not manifest:
print("Enter the manifest file (URL or local path)")
manifest = input()
if 'http' in manifest:
data = requests.get(manifest).json()
else:
with open(manifest, 'r') as f:
data = json.load(f)
entries = [e for e in data['current']]
entries.extend([e for e in data['dependencies']])
get_builds(entries, gpu_target, args.output)
if __name__ == "__main__":
main()

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/Tensile
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/Tensile.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/TransferBench
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/TransferBench.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -3,16 +3,7 @@ variables:
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
parameters: parameters:
- name: aompRef - name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: extrasRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: flangRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: llvmRef
type: string type: string
default: refs/tags/$(LATEST_RELEASE_TAG) default: refs/tags/$(LATEST_RELEASE_TAG)
@@ -22,22 +13,22 @@ resources:
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp name: ROCm/aomp
ref: ${{ parameters.aompRef }} ref: ${{ parameters.checkoutRef }}
- repository: aomp-extras_repo - repository: aomp-extras_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp-extras name: ROCm/aomp-extras
ref: ${{ parameters.extrasRef }} ref: ${{ parameters.checkoutRef }}
- repository: flang_repo - repository: flang_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/flang name: ROCm/flang
ref: ${{ parameters.flangRef }} ref: ${{ parameters.checkoutRef }}
- repository: llvm-project_repo - repository: llvm-project_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/llvm-project name: ROCm/llvm-project
ref: ${{ parameters.llvmRef }} ref: amd-staging
trigger: none trigger: none
pr: none pr: none
@@ -46,4 +37,4 @@ jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml - template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml
parameters: parameters:
checkoutRepo: aomp_repo checkoutRepo: aomp_repo
checkoutRef: ${{ parameters.aompRef }} checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: boostVersion
type: string
default: 'master'
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/boost.yml
parameters:
boostVersion: ${{ parameters.boostVersion }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: catch2Version
type: string
default: "v3.7.0"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/catch2.yml
parameters:
catch2Version: ${{ parameters.catch2Version }}

View File

@@ -28,22 +28,12 @@ resources:
endpoint: ROCm endpoint: ROCm
name: ROCm/hipother name: ROCm/hipother
ref: ${{ parameters.checkoutRef }} ref: ${{ parameters.checkoutRef }}
pipelines:
- pipeline: hip_pipeline
source: \experimental\HIP
trigger: true
- pipeline: hipother_pipeline
source: \experimental\hipother
trigger: true
trigger: none trigger: none
pr: none pr: none
jobs: jobs:
- ${{ if eq(variables['Build.Reason'], 'ResourceTrigger') }}: - template: ${{ variables.CI_COMPONENT_PATH }}/HIP.yml
- template: ${{ variables.CI_COMPONENT_PATH }}/copyHIP.yml@pipelines_repo
- ${{ if ne(variables['Build.Reason'], 'ResourceTrigger') }}:
- template: ${{ variables.CI_COMPONENT_PATH }}/HIP.yml@pipelines_repo
parameters: parameters:
checkoutRepo: release_repo checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }} checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: fmtlibVersion
type: string
default: "11.1.3"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/fmtlib.yml
parameters:
fmtlibVersion: ${{ parameters.fmtlibVersion }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: libdivideVersion
type: string
default: master
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/libdivide.yml
parameters:
libdivideVersion: ${{ parameters.libdivideVersion }}

View File

@@ -16,14 +16,14 @@ resources:
- repository: release_repo - repository: release_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/rocJPEG name: ROCm/omniperf
ref: ${{ parameters.checkoutRef }} ref: ${{ parameters.checkoutRef }}
trigger: none trigger: none
pr: none pr: none
jobs: jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocJPEG.yml - template: ${{ variables.CI_COMPONENT_PATH }}/omniperf.yml
parameters: parameters:
checkoutRepo: release_repo checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }} checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/omnitrace
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/omnitrace.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-compute
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-compute.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-sdk
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-sdk.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-systems
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-systems.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: spdlogVersion
type: string
default: "v1.15.1"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/spdlog.yml
parameters:
spdlogVersion: ${{ parameters.spdlogVersion }}

View File

@@ -6,49 +6,116 @@ parameters:
- name: pipelineId - name: pipelineId
type: string type: string
default: '' default: ''
- name: branchName - name: useDefaultBranch
type: string type: boolean
default: '$(Build.SourceBranchName)' # for tagged builds default: true
- name: latestFromBranch
type: boolean
default: true
- name: extractToMnt
type: boolean
default: false
- name: fileFilter - name: fileFilter
type: string type: string
default: '' default: ''
- name: extractAndDeleteFiles - name: defaultBranchList
type: boolean type: object
default: true default:
# set to true if doing full build of ROCm stack AMDMIGraphX: develop
# and dependencies are pulled from same pipeline amdsmi: develop
- name: aggregatePipeline aomp-extras: aomp-dev
type: boolean aomp: aomp-dev
default: false clr: amd-staging
composable_kernel: develop
half: rocm
HIP: amd-staging
hipBLAS: develop
hipBLASLt: develop
hipBLAS-common: develop
hipCUB: develop
hipFFT: develop
hipfort: develop
HIPIFY: amd-staging
hipRAND: develop
hipSOLVER: develop
hipSPARSE: develop
hipSPARSELt: develop
hipTensor: develop
llvm-project: amd-staging
MIOpen: develop
MIVisionX: develop
rccl: develop
rdc: develop
rocAL: develop
rocALUTION: develop
rocBLAS: develop
ROCdbgapi : amd-staging
rocDecode: develop
rocFFT: develop
ROCgdb: amd-staging
rocm-cmake: develop
rocm-core: master
rocm-examples: develop
rocminfo: amd-staging
rocMLIR: develop
ROCmValidationSuite: master
rocm_bandwidth_test: master
rocm_smi_lib: develop
rocPRIM: develop
rocprofiler-register: amd-mainline
rocprofiler: amd-staging
ROCR-Runtime: amd-staging
rocRAND: develop
rocr_debug_agent: amd-staging
rocSOLVER: develop
rocSPARSE: develop
rocThrust: develop
roctracer: amd-staging
rocWMMA: develop
rpp: develop
- name: allowPartiallySucceededBuilds
type: object
default:
- amdsmi
- aomp
- HIPIFY
- rccl
- rdc
- rocm-cmake
- rocm_smi_lib
- rocFFT
- MIVisionX
# BELOW REQUIRED IF useDefaultBranch false
- name: branchName
type: string
default: '$(Build.SourceBranchName)' # for tagged builds
steps: steps:
- task: DownloadPipelineArtifact@2 - task: DownloadPipelineArtifact@2
displayName: Download ${{ parameters.componentName }} displayName: Download ${{ parameters.componentName }}
retryCountOnTaskFailure: 3
inputs: inputs:
${{ if eq(parameters.componentName, 'clr') }}:
itemPattern: '**/*${{ parameters.componentName }}*${{ parameters.fileFilter }}*amd*' # filter out nvidia clr artifacts
${{ else }}:
itemPattern: '**/*${{ parameters.componentName }}*${{ parameters.fileFilter }}*'
targetPath: '$(Pipeline.Workspace)/d'
allowPartiallySucceededBuilds: true
${{ if parameters.aggregatePipeline }}:
buildType: 'current'
${{ else }}:
buildType: 'specific' buildType: 'specific'
project: ROCm-CI project: ROCm-CI
specificBuildWithTriggering: true
definition: ${{ parameters.pipelineId }} definition: ${{ parameters.pipelineId }}
branchName: refs/heads/${{ parameters.branchName }} specificBuildWithTriggering: true
${{ if eq(parameters.componentName, 'aomp') }}: itemPattern: '**/*${{ parameters.fileFilter }}*'
buildVersionToDownload: latest # aomp trigger lives in ROCm/ROCm, so cannot use ROCm/aomp branch names ${{ if eq(parameters.latestFromBranch, true) }}:
${{ if notIn(parameters.componentName, 'aomp') }}: # remove this once these pipelines are functional + up-to-date
buildVersionToDownload: latestFromBranch # default is 'latest'
${{ if eq(parameters.useDefaultBranch, true) }}:
branchName: refs/heads/${{ parameters.defaultBranchList[parameters.componentName] }}
${{ else }}: ${{ else }}:
buildVersionToDownload: latestFromBranch branchName: ${{ parameters.branchName }}
- ${{ if eq(parameters.extractAndDeleteFiles, true) }}: ${{ if containsValue(parameters.allowPartiallySucceededBuilds, parameters.componentName) }}:
allowPartiallySucceededBuilds: true
targetPath: '$(Pipeline.Workspace)/d'
- task: ExtractFiles@1 - task: ExtractFiles@1
displayName: Extract ${{ parameters.componentName }} displayName: Extract ${{ parameters.componentName }}
inputs: inputs:
archiveFilePatterns: '$(Pipeline.Workspace)/d/**/*.tar.gz' archiveFilePatterns: '$(Pipeline.Workspace)/d/**/*.tar.gz'
${{ if parameters.extractToMnt }}:
destinationFolder: '/mnt/rocm'
${{ else }}:
destinationFolder: '$(Agent.BuildDirectory)/rocm' destinationFolder: '$(Agent.BuildDirectory)/rocm'
cleanDestinationFolder: false cleanDestinationFolder: false
overwriteExistingFiles: true overwriteExistingFiles: true

View File

@@ -1,32 +0,0 @@
# Every publish artifact call should be coupled with a script that
# prints out the artifact name to a common text file
# This template parses that text file line by line and prints out the download URL
# for each artifact, so that they are easily accessible to the public
# replace trailing '=' with their count in the encoded string
steps:
- task: Bash@3
displayName: "!! Download Links !!"
condition: always()
continueOnError: true
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
URL_BEGIN="https://artprodcus3.artifacts.visualstudio.com/"
URL_MIDDLE="/_apis/artifact/"
URL_END="/content?format=file&subPath=%2F"
ARTIFACT_NAME="$(Agent.JobName)_$(System.JobAttempt)"
ARTIFACT_STRING="pipelineartifact://ROCm-CI/projectId/$(DOWNLOAD_PROJECT_ID)/buildId/$(Build.BuildId)/artifactName/${ARTIFACT_NAME}"
ENCODED_STRING=$(echo -n "${ARTIFACT_STRING}" | base64 -w 0)
PADDING_COUNT=$(echo -n "${ENCODED_STRING}" | awk -F= '{print NF-1}')
if [ "$PADDING_COUNT" -gt 0 ]; then
FINAL_ENCODED_STRING=$(echo -n "${ENCODED_STRING}" | sed "s/=*$/${PADDING_COUNT}/")
else
FINAL_ENCODED_STRING="${ENCODED_STRING}0"
fi
while IFS= read -r fileName; do
echo "File Name:"
echo "$fileName"
printf "Download Link:\n%s%s/%s%s%s%s%s\n\n" "${URL_BEGIN}" "${DOWNLOAD_ORGANIZATION_ID}" "${DOWNLOAD_PROJECT_ID}" "${URL_MIDDLE}" "${FINAL_ENCODED_STRING}" "${URL_END}" "${fileName}"
done < pipelineArtifacts.txt
rm pipelineArtifacts.txt

View File

@@ -3,21 +3,15 @@
# publish can be toggled off for jobs that produce multiple tarballs # publish can be toggled off for jobs that produce multiple tarballs
# for those cases, only publish the last call which puts all the tarballs in one container folder # for those cases, only publish the last call which puts all the tarballs in one container folder
parameters: parameters:
- name: componentName
type: string
default: $(Build.DefinitionName)
- name: gpuTarget
type: string
default: ''
- name: artifactName - name: artifactName
type: string type: string
default: drop default: 'drop'
- name: publish - name: publish
type: boolean type: boolean
default: true default: true
- name: os - name: gpuTarget
type: string type: string
default: 'ubuntu2204' default: ''
steps: steps:
- task: ArchiveFiles@2 - task: ArchiveFiles@2
@@ -26,26 +20,17 @@ steps:
includeRootFolder: false includeRootFolder: false
archiveType: 'tar' archiveType: 'tar'
tarCompression: 'gz' tarCompression: 'gz'
archiveFile: '$(Build.ArtifactStagingDirectory)/${{ parameters.componentName }}_$(Build.BuildId)_$(Build.BuildNumber)_${{ parameters.os }}_${{ parameters.gpuTarget }}_${{ parameters.artifactName }}_$(System.JobAttempt).tar.gz' archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.DefinitionName)_$(Build.SourceBranchName)_$(Build.BuildId)_$(Build.BuildNumber)_ubuntu2204_${{ parameters.artifactName }}_${{ parameters.gpuTarget }}.tar.gz'
- task: DeleteFiles@1 - task: DeleteFiles@1
displayName: 'Cleanup Staging Area' displayName: 'Cleanup Staging Area'
inputs: inputs:
SourceFolder: '$(Build.BinariesDirectory)' SourceFolder: '$(Build.BinariesDirectory)'
Contents: '/**/*' Contents: '/**/*'
RemoveDotFiles: true RemoveDotFiles: true
- task: Bash@3
displayName: Save pipeline artifact file name
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: echo "${{ parameters.componentName }}_$(Build.BuildId)_$(Build.BuildNumber)_${{ parameters.os }}_${{ parameters.gpuTarget }}_${{ parameters.artifactName }}_$(System.JobAttempt).tar.gz" >> pipelineArtifacts.txt
# then publish it # then publish it
- ${{ if parameters.publish }}: - ${{ if parameters.publish }}:
- task: PublishPipelineArtifact@1 - task: PublishPipelineArtifact@1
condition: always()
displayName: '${{ parameters.artifactName }} Publish' displayName: '${{ parameters.artifactName }} Publish'
retryCountOnTaskFailure: 3 retryCountOnTaskFailure: 3
inputs: inputs:
# if this artifact name is changed, please also update $ARTIFACT_URL inside miopen-get-ck-build.yml
artifactName: $(Agent.JobName)_$(System.JobAttempt)
targetPath: '$(Build.ArtifactStagingDirectory)' targetPath: '$(Build.ArtifactStagingDirectory)'

View File

@@ -1,7 +1,4 @@
parameters: parameters:
- name: os
type: string
default: 'ubuntu2204'
- name: componentName - name: componentName
type: string type: string
default: '' default: ''
@@ -23,23 +20,17 @@ steps:
displayName: '${{ parameters.componentName }} configure flags' displayName: '${{ parameters.componentName }} configure flags'
inputs: inputs:
targetType: inline targetType: inline
script: ./configure --prefix=${{ parameters.installDir }} ${{ parameters.configureFlags }}
workingDirectory: ${{ parameters.buildDir }} workingDirectory: ${{ parameters.buildDir }}
script: |
${{ iif(eq(parameters.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
./configure --prefix=${{ parameters.installDir }} ${{ parameters.configureFlags }}
- task: Bash@3 - task: Bash@3
displayName: '${{ parameters.componentName }} make' displayName: '${{ parameters.componentName }} make'
inputs: inputs:
targetType: inline targetType: inline
script: ${{ parameters.makeCallPrefix }} make -j$(nproc)
workingDirectory: ${{ parameters.buildDir }} workingDirectory: ${{ parameters.buildDir }}
script: |
${{ iif(eq(parameters.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
${{ parameters.makeCallPrefix }} make -j$(nproc)
- task: Bash@3 - task: Bash@3
displayName: '${{ parameters.componentName }} make install' displayName: '${{ parameters.componentName }} make install'
inputs: inputs:
targetType: inline targetType: inline
script: make install
workingDirectory: ${{ parameters.buildDir }} workingDirectory: ${{ parameters.buildDir }}
script: |
${{ iif(eq(parameters.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
make install

View File

@@ -1,25 +1,19 @@
parameters: parameters:
- name: os
type: string
default: 'ubuntu2204'
- name: componentName - name: componentName
type: string type: string
default: '' default: ''
- name: extraBuildFlags - name: extraBuildFlags
type: string type: string
default: '' default: ''
- name: extraCxxFlags
type: string
default: ''
- name: multithreadFlag - name: multithreadFlag
type: string type: string
default: '' default: ''
- name: cmakeBuildDir - name: cmakeBuildDir
type: string type: string
default: $(Agent.BuildDirectory)/s/build default: 'build'
- name: cmakeSourceDir - name: cmakeSourceDir
type: string type: string
default: $(Agent.BuildDirectory)/s default: '..'
- name: customBuildTarget - name: customBuildTarget
type: string type: string
default: '' default: ''
@@ -38,81 +32,36 @@ parameters:
- name: installEnabled - name: installEnabled
type: boolean type: boolean
default: true default: true
# for jobs that rebuild during install step and use ninja
# set to true to save time, only applies for almalinux8
- name: consolidateBuildAndInstall
type: boolean
default: false
- name: printDiskSpace
type: boolean
default: true
# todo: make this control cxx and c compiler flags
- name: useAmdclang
type: boolean
default: true
# for cmake calls, set env variables for AlmaLinux 8
# to simulate running source /opt/rh/gcc-toolset-14/enable for the session
steps: steps:
# create workingDirectory if it does not exist and change into it # create workingDirectory if it does not exist and change into it
# call cmake from within that directory using $cmakeArgs as its parameters # call cmake from within that directory using $cmakeArgs as its parameters
- task: CMake@1 - task: CMake@1
displayName: '${{parameters.componentName }} CMake Flags' displayName: '${{parameters.componentName }} CMake Flags'
${{ if eq(parameters.os, 'almalinux8')}}:
env:
PATH: "/opt/rh/gcc-toolset-14/root/usr/bin:$(PATH)"
MANPATH: "/opt/rh/gcc-toolset-14/root/usr/share/man:$(MANPATH)"
INFOPATH: "/opt/rh/gcc-toolset-14/root/usr/share/info:$(INFOPATH)"
PCP_DIR: "/opt/rh/gcc-toolset-14/root"
LD_LIBRARY_PATH: "/opt/rh/gcc-toolset-14/root/usr/lib64:/opt/rh/gcc-toolset-14/root/usr/lib:$(LD_LIBRARY_PATH)"
PKG_CONFIG_PATH: "/opt/rh/gcc-toolset-14/root/usr/lib64/pkgconfig:$(PKG_CONFIG_PATH)"
inputs: inputs:
workingDirectory: ${{ parameters.cmakeBuildDir }} workingDirectory: ${{ parameters.cmakeBuildDir }}
cmakeArgs: >- ${{ if eq(parameters.customInstallPath, true) }}:
${{ iif(parameters.customInstallPath, join('', format('-DCMAKE_INSTALL_PREFIX={0}', parameters.installDir)), '') }} cmakeArgs: -DCMAKE_INSTALL_PREFIX=${{ parameters.installDir }} ${{ parameters.extraBuildFlags }} ${{ parameters.cmakeSourceDir }}
${{ iif(eq(parameters.os, 'almalinux8'), '-DCMAKE_SHARED_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/lib64 -L/opt/rh/gcc-toolset-14/root/usr/lib/gcc/x86_64-redhat-linux/14/"', '') }} ${{ else }}:
${{ iif(eq(parameters.os, 'almalinux8'), '-DCMAKE_EXE_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/lib64 -L/opt/rh/gcc-toolset-14/root/usr/lib/gcc/x86_64-redhat-linux/14/"', '') }} cmakeArgs: ${{ parameters.extraBuildFlags }} ..
-DCMAKE_CXX_FLAGS="${{ parameters.extraCxxFlags }} ${{ iif(and(eq(parameters.os, 'almalinux8'), parameters.useAmdclang), '--gcc-toolchain=/opt/rh/gcc-toolset-14/root', '') }}"
${{ parameters.extraBuildFlags }}
${{ parameters.cmakeSourceDir }}
- ${{ if parameters.printDiskSpace }}:
- script: df -h - script: df -h
displayName: Disk space before build displayName: Disk space before build
# equivalent to running make $cmakeTargetDir from $cmakeBuildDir # equivalent to running make $cmakeTargetDir from $cmakeBuildDir
# i.e., cd $cmakeBuildDir; make $cmakeTargetDir # i.e., cd $cmakeBuildDir; make $cmakeTargetDir
- task: CMake@1 - task: CMake@1
${{ if and( eq(parameters.os, 'almalinux8'), eq(parameters.consolidateBuildAndInstall , true)) }}: displayName: '${{parameters.componentName }} Build'
displayName: '${{ parameters.componentName }} CMake Build and Install'
${{ else }}:
displayName: '${{ parameters.componentName }} CMake Build'
${{ if eq(parameters.os, 'almalinux8')}}:
env:
PATH: "/opt/rh/gcc-toolset-14/root/usr/bin:$(PATH)"
MANPATH: "/opt/rh/gcc-toolset-14/root/usr/share/man:$(MANPATH)"
INFOPATH: "/opt/rh/gcc-toolset-14/root/usr/share/info:$(INFOPATH)"
PCP_DIR: "/opt/rh/gcc-toolset-14/root"
LD_LIBRARY_PATH: "/opt/rh/gcc-toolset-14/root/usr/lib64:/opt/rh/gcc-toolset-14/root/usr/lib:$(LD_LIBRARY_PATH)"
PKG_CONFIG_PATH: "/opt/rh/gcc-toolset-14/root/usr/lib64/pkgconfig:$(PKG_CONFIG_PATH)"
inputs: inputs:
workingDirectory: ${{ parameters.cmakeBuildDir }} workingDirectory: ${{ parameters.cmakeBuildDir }}
${{ if eq(parameters.os, 'almalinux8') }}: ${{ if eq(parameters.customBuildTarget, '') }}:
cmakeArgs: >- cmakeArgs: '--build ${{ parameters.cmakeTargetDir }} ${{ parameters.multithreadFlag }}'
--build ${{ parameters.cmakeTargetDir }} ${{ else }}:
${{ iif(and(eq(parameters.consolidateBuildAndInstall, true), ne(parameters.cmakeTarget, '')), format('--target {0}', parameters.cmakeTarget), '') }} cmakeArgs: '--build ${{ parameters.cmakeTargetDir }} --target ${{ parameters.customBuildTarget }} ${{ parameters.multithreadFlag }}'
${{ iif(and(ne(parameters.customBuildTarget, ''), ne(parameters.consolidateBuildAndInstall, true)), format('--target {0}', parameters.customBuildTarget), '') }} retryCountOnTaskFailure: 10
${{ parameters.multithreadFlag }}
${{ if ne(parameters.os, 'almalinux8') }}:
cmakeArgs: >-
--build ${{ parameters.cmakeTargetDir }}
${{ iif(ne(parameters.customBuildTarget, ''), format('--target {0}', parameters.customBuildTarget), '') }}
${{ parameters.multithreadFlag }}
- ${{ if parameters.printDiskSpace }}:
- script: df -h - script: df -h
displayName: Disk space after build displayName: Disk space after build
# equivalent to running make $cmakeTarget from $cmakeBuildDir # equivalent to running make $cmakeTarget from $cmakeBuildDir
# e.g., make install # e.g., make install
- ${{ if and(eq(parameters.installEnabled, true), or(ne(parameters.os, 'almalinux8'), eq(parameters.consolidateBuildAndInstall, false))) }}: - ${{ if eq(parameters.installEnabled, true) }}:
- task: CMake@1 - task: CMake@1
displayName: '${{parameters.componentName }} ${{ parameters.cmakeTarget }}' displayName: '${{parameters.componentName }} ${{ parameters.cmakeTarget }}'
inputs: inputs:

Some files were not shown because too many files have changed in this diff Show More