Compare commits

..

1 Commits

Author SHA1 Message Date
Young Hui
3204d37a1a add third column to compat matrix 2024-06-25 21:21:08 -04:00
563 changed files with 19977 additions and 74393 deletions

View File

@@ -23,40 +23,8 @@ ROCm-CI Azure DevOps Pipelines contains markup language files that orchestrate b
- [YAML schema](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines&viewFallbackFrom=azure-devops) - [YAML schema](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines&viewFallbackFrom=azure-devops)
- [Azure Pipelines Task Index](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/?view=azure-pipelines) - [Azure Pipelines Task Index](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/?view=azure-pipelines)
## VMSS Setup
The Azure VMSS used for build jobs have docker installed during provisioning through the Azure Portal's settings.
Select the VMSS, then go to Settings, Operating system.
In the Custom Data section, select to Modify Custom Data. Enter the code block below and Apply.
```bash
#cloud-config
bootcmd:
- mkdir -p /etc/systemd/system/walinuxagent.service.d
- echo "[Unit]\nAfter=cloud-final.service" > /etc/systemd/system/walinuxagent.service.d/override.conf
- sed "s/After=multi-user.target//g" /lib/systemd/system/cloud-final.service > /etc/systemd/system/cloud-final.service
- systemctl daemon-reload
apt:
sources:
docker.list:
source: deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
packages:
- docker-ce
- docker-ce-cli
groups:
- docker
runcmd:
- usermod -aG docker $USER
- systemctl restart docker
- systemctl enable docker
```
## Disclaimer ## Disclaimer
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard versionchanges, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated.AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.THIS INFORMATION IS PROVIDED AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard versionchanges, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated.AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.THIS INFORMATION IS PROVIDED AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.
© 2024 Advanced Micro Devices, Inc. All Rights Reserved.

View File

@@ -4,21 +4,11 @@ variables:
resources: resources:
repositories: repositories:
- repository: aomp_repo - repository: release_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp name: ROCm/aomp
ref: aomp-dev ref: aomp-dev
- repository: aomp-extras_repo
type: github
endpoint: ROCm
name: ROCm/aomp-extras
ref: aomp-dev
- repository: flang_repo
type: github
endpoint: ROCm
name: ROCm/flang
ref: aomp-dev
- repository: llvm-project_repo - repository: llvm-project_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
@@ -27,10 +17,7 @@ resources:
pipelines: pipelines:
- pipeline: rocr-runtime_pipeline - pipeline: rocr-runtime_pipeline
source: \ROCR-Runtime source: \ROCR-Runtime
trigger: trigger: true
branches:
include:
- amd-staging
# this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime # this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime
trigger: none trigger: none
@@ -39,4 +26,4 @@ pr: none
jobs: jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml - template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml
parameters: parameters:
checkoutRepo: aomp_repo checkoutRepo: release_repo

View File

@@ -1,33 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml@pipelines_repo
parameters:
- name: pipelinesRepoRef
type: string
default: refs/heads/develop
- name: librariesRepoRef
type: string
default: refs/heads/develop
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
ref: ${{ parameters.pipelinesRepoRef }}
- repository: libraries_repo
type: github
endpoint: ROCm
name: ROCm/rocm-libraries
ref: ${{ parameters.librariesRepoRef }}
trigger: none
pr: none
jobs:
- template: /.azuredevops/ci-builds/mathlibs.yml@pipelines_repo
parameters:
checkoutRepo: libraries_repo
buildDependsOn: false

View File

@@ -1,38 +0,0 @@
# entrypoint for kicking off a unified build of the mathlibs
# this template is designed to be called by another pipeline (llvm, clr, etc.)
# `buildDependsOn` will need to be set when calling this template
# passes a `unifiedBuild` param to downstream pipelines, which will prevent duplicate jobs
# logic needs to be added in individual mathlib pipelines for handling `unifiedBuild`
parameters:
- name: checkoutRepo
type: string
default: monorepo
- name: buildDependsOn
type: object
default: false
- name: downstreamComponentMatrix
type: object
default:
- rocRAND:
name: rocRAND
sparseCheckoutDir: projects/rocrand
- rocPRIM:
name: rocPRIM
sparseCheckoutDir: projects/rocprim
- hipBLAS-common:
name: hipBLAS-common
sparseCheckoutDir: projects/hipblas-common
# - composable_kernel:
# name: composable_kernel
# sparseCheckoutDir: projects/composablekernel
jobs:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ parameters.buildDependsOn }}
triggerDownstreamJobs: true
unifiedBuild: true

View File

@@ -1,240 +1,97 @@
parameters: parameters:
- name: componentName
type: string
default: AMDMIGraphX
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
# - name: sparseCheckoutDir
# type: string
# default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git - git
- libdnnl-dev - cmake
- libdrm-dev
- libmsgpack-dev
- libnuma-dev
- libprotobuf-dev
- libsqlite3-dev
- libtbb-dev
- ninja-build - ninja-build
- nlohmann-json3-dev - libnuma-dev
- protobuf-compiler
- python3-pip - python3-pip
- python3-pybind11
- python3-venv - python3-venv
- libtbb-dev
- nlohmann-json3-dev
- libmsgpack-dev
- libsqlite3-dev
- libprotobuf-dev
- protobuf-compiler
- python3-pybind11
- name: pipModules - name: pipModules
type: object type: object
default: default:
- https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz - https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
- "onnx>=1.14.1" - onnx==1.14.1
- "numpy>=1.21.6" - numpy==1.21.6
- "typing>=3.7.4" - typing==3.7.4
- "pytest>=6.0.1" - pytest==6.0.1
- "packaging>=23.0" - packaging==23.0
- "protobuf>=3.20.2" - protobuf==3.20.2
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- aomp-extras
- clr
- composable_kernel
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime - ROCR-Runtime
- rocBLAS - clr
- rocminfo - rocminfo
- rocMLIR - rocMLIR
- rocprofiler-register - MIOpen
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp - aomp
- aomp-extras - aomp-extras
- clr
- composable_kernel
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocm-cmake
- ROCR-Runtime
- rocBLAS - rocBLAS
- rocminfo - composable_kernel
- rocMLIR
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: AMDMIGraphX
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_ubuntu2204_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all pipModules: ${{ parameters.pipModules }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} checkoutRepo: ${{ parameters.checkoutRepo }}
pipModules: ${{ parameters.pipModules }} # half version should be fixed to 5.6.0
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters:
parameters: dependencySource: fixed
checkoutRepo: ${{ parameters.checkoutRepo }} fixedComponentName: half
# half version should be fixed to 5.6.0 fixedPipelineIdentifier: $(half560-pipeline-id)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
buildType: specific
definitionId: ${{ variables.HALF560_PIPELINE_ID }}
buildId: ${{ variables.HALF560_BUILD_ID }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-DAMDGPU_TARGETS=${{ job.target }}
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DBUILD_TESTING=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: MIGRAPHX_TRACE_BENCHMARKING
value: 1
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
# half version should be fixed to 5.6.0
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
buildType: specific
definitionId: ${{ variables.HALF560_PIPELINE_ID }}
buildId: ${{ variables.HALF560_BUILD_ID }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
${{ if parameters.triggerDownstreamJobs }}: parameters:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} extraBuildFlags: >-
- task: CMake@1 -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
displayName: MIGraphXTest CMake Flags -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
inputs: -DCMAKE_BUILD_TYPE=Release
cmakeArgs: >- -DAMDGPU_TARGETS=gfx1030;gfx1100
-DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DGPU_TARGETS=${{ job.target }} -DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DAMDGPU_TARGETS=${{ job.target }} -DMIGRAPHX_USE_COMPOSABLEKERNEL=OFF
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DBUILD_TESTING=ON
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm -GNinja
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include # REFERENCE: https://github.com/ROCm/composable_kernel/issues/782
-DBUILD_TESTING=ON - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
-DMIGRAPHX_ENABLE_C_API_TEST=ON
..
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testExecutable: make
testParameters: -j$(nproc) check
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
extraEnvVars:
- MIGRAPHX_TRACE_BENCHMARKING:::1

View File

@@ -1,45 +1,15 @@
parameters: parameters:
- name: componentName
type: string
default: hip_clr_combined
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- libnuma-dev - libnuma-dev
- mesa-common-dev - mesa-common-dev
- ninja-build
- ocl-icd-libopencl1
- ocl-icd-opencl-dev
- opencl-headers
- zlib1g-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
@@ -48,87 +18,124 @@ parameters:
type: object type: object
default: default:
- llvm-project - llvm-project
- ROCT-Thunk-Interface
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - rocprofiler-register
- name: rocmDependenciesNvidia - name: rocmDependenciesNvidia
type: object type: object
default: default:
- llvm-project - llvm-project
- ROCR-Runtime
- name: jobMatrix # hip and clr are tightly-coupled
type: object # run this same template for both repos
default: # any changes for clr should just trigger HIP pipeline
buildJobs: # similarly for hipother repo, for Nvidia backend
- { os: ubuntu2204, packageManager: apt, platform: amd }
- { os: ubuntu2204, packageManager: apt, platform: nvidia }
- { os: almalinux8, packageManager: dnf, platform: amd }
- { os: almalinux8, packageManager: dnf, platform: nvidia }
# HIP with AMD backend
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hip_clr_combined_amd
- job: ${{ parameters.componentName }}_${{ job.os }}_${{ job.platform }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
vmImage: 'ubuntu-22.04' clean: all
${{ if eq(job.os, 'almalinux8') }}: steps:
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest parameters:
endpoint: ContainerService3 aptPackages: ${{ parameters.aptPackages }}
variables: pipModules: ${{ parameters.pipModules }}
- group: common # checkout triggering repo (either HIP or clr)
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: # if this is triggered by HIP repo, matching repo is clr
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml # if this is triggered by clr repo, matching repo is HIP
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
pipModules: ${{ parameters.pipModules }} checkoutRepo: matching_repo
packageManager: ${{ job.packageManager }} # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - ${{ if eq(parameters.checkoutRef, '') }}:
# full checkout of rocm-systems superrepo, we need clr, hip, and hipother
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependenciesAMD }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
${{ if eq(job.platform, 'amd') }}: - ${{ if ne(parameters.checkoutRef, '') }}:
dependencyList: ${{ parameters.rocmDependenciesAMD }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
${{ elseif eq(job.platform, 'nvidia') }}:
dependencyList: ${{ parameters.rocmDependenciesNvidia }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: clr dependencyList: ${{ parameters.rocmDependenciesAMD }}
cmakeBuildDir: $(Agent.BuildDirectory)/s/projects/clr/build dependencySource: tag-builds
cmakeSourceDir: $(Agent.BuildDirectory)/s/projects/clr # compile clr
os: ${{ job.os }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
useAmdclang: false parameters:
extraBuildFlags: >- componentName: clr
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm cmakeBuildDir: 'clr/build'
-DROCM_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin -DHIP_COMMON_DIR=$(Build.SourcesDirectory)/HIP
-DHIP_COMMON_DIR=$(Agent.BuildDirectory)/s/projects/hip -DHIP_PLATFORM=amd
-DHIPNV_DIR=$(Agent.BuildDirectory)/s/projects/hipother/hipnv -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DHIP_PLATFORM=${{ job.platform }} -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCLR_BUILD_HIP=ON -DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin
-DCLR_BUILD_OCL=ON -DCLR_BUILD_HIP=ON
-GNinja -DCLR_BUILD_OCL=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
artifactName: amd
# HIP with Nvidia backend
- job: hip_clr_combined_nvidia
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
# checkout triggering repo (either HIP or clr)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# if this is triggered by HIP repo, matching repo is clr
# if this is triggered by clr repo, matching repo is HIP
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: matching_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: hipother_repo
# CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
artifactName: ${{ job.platform }} dependencyList: ${{ parameters.rocmDependenciesNvidia }}
os: ${{ job.os }} dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml # manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
artifactName: ${{ job.platform }} dependencyList: ${{ parameters.rocmDependenciesNvidia }}
os: ${{ job.os }} dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml - script: 'ls -1R $(Agent.BuildDirectory)/rocm'
displayName: 'Artifact listing'
# compile clr
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: clr
cmakeBuildDir: 'clr/build'
extraBuildFlags: >-
-DHIP_COMMON_DIR=$(Build.SourcesDirectory)/HIP
-DHIP_PLATFORM=nvidia
-DHIPCC_BIN_DIR=$(Agent.BuildDirectory)/rocm/bin
-DCLR_BUILD_HIP=ON
-DCLR_BUILD_OCL=OFF
-DHIPNV_DIR=$(Build.SourcesDirectory)/hipother/hipnv
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
artifactName: nvidia

View File

@@ -1,155 +1,55 @@
parameters: parameters:
- name: componentName
type: string
default: HIPIFY
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cuda-toolkit-12-9 - cmake
- libcudnn9-dev-cuda-12
- libnuma-dev
- mesa-common-dev
- ninja-build - ninja-build
- python-is-python3 - libnuma-dev
- python3-dev
- python3-pip
- name: pipModules
type: object
default:
- lit
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- llvm-project - llvm-project
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: HIPIFY
- job: ${{ parameters.componentName }}_build_${{ job.os }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'ubuntu2404') }}: workspace:
name: rocm-ci_medium_build_pool_2404 clean: all
${{ else }}: steps:
name: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: aptPackages: ${{ parameters.aptPackages }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: # CI case: download latest default branch build
- task: Bash@3 - ${{ if eq(parameters.checkoutRef, '') }}:
displayName: 'Register CUDA packages'
inputs:
targetType: inline
${{ if eq(job.os, 'ubuntu2204') }}:
script: |
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo rm -f cuda-keyring_1.1-1_all.deb
sudo apt update
${{ if eq(job.os, 'almalinux8') }}:
script: |
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- task: Bash@3
displayName: Add lit to PATH
inputs:
targetType: inline
script: |
site_packages=$(python3 -m site --user-base)/bin
sudo ln -sf $site_packages/bin/lit $(Pipeline.Workspace)/llvm-lit
echo "##vso[task.prependpath]$site_packages"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
# cutensor is not available from apt or dnf - ${{ if ne(parameters.checkoutRef, '') }}:
- task: Bash@3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
displayName: 'Download and install cutensor'
inputs:
targetType: inline
script: |
wget -q --show-progress https://developer.download.nvidia.com/compute/cutensor/redist/libcutensor/linux-x86_64/libcutensor-linux-x86_64-2.2.0.0-archive.tar.xz
tar -xvJf libcutensor-linux-x86_64-*.tar.xz
mkdir -p $(Pipeline.Workspace)/cutensor
cp -r libcutensor-linux-x86_64-*/* $(Pipeline.Workspace)/cutensor/
- task: Bash@3
displayName: 'List downloaded CUDA files'
inputs:
targetType: inline
script: ls -la1R /usr/local/cuda-12.9
# script: cp $(Pipeline.Workspace)/llvm-project/llvm/build/bin/FileCheck $(Pipeline.Workspace)/llvm/bin
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: tag-builds
consolidateBuildAndInstall: true - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;/usr/local/cuda/targets/x86_64-linux/lib extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DCMAKE_BUILD_TYPE=Release
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm
-DHIPIFY_CLANG_TESTS=ON -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release -GNinja
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.9 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
-DCUDA_DNN_ROOT_DIR=/usr/local/cuda-12.9
-DCUDA_CUB_ROOT_DIR=/usr/local/cuda-12.9/targets/x86_64-linux/include/cub
-DCUDA_TENSOR_ROOT_DIR=$(Pipeline.Workspace)/cutensor/
multithreadFlag: -- -j32
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
# parameters:
# componentName: HIPIFY
# testDir: $(Build.SourcesDirectory)/build
# testExecutable: make
# testParameters: -j 32 test-hipify
# testPublishResults: false
# os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
registerCUDAPackages: true
extraCopyDirectories:
- llvm-project

View File

@@ -1,61 +1,30 @@
parameters: parameters:
- name: componentName
type: string
default: MIOpen
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- jq
- libdrm-dev
- libmsgpack-dev
- libsqlite3-dev
- libstdc++-12-dev
- ninja-build
- python3-pip
- python3-venv
- software-properties-common - software-properties-common
- zip - python3-pip
- name: pipModules - cmake
type: object - ninja-build
default: - libsqlite3-dev
- cget - libbz2-dev
- nlohmann-json3-dev
- libgtest-dev
- libdrm-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- half - rocMLIR
- rocRAND - rocRAND
- rocBLAS - rocBLAS
- hipBLAS - half
- hipBLASLt - composable_kernel
- hipBLAS-common
- rocm-cmake - rocm-cmake
- llvm-project - llvm-project
- ROCR-Runtime - ROCR-Runtime
@@ -63,228 +32,56 @@ parameters:
- clr - clr
- rocminfo - rocminfo
- roctracer - roctracer
- name: rocmTestDependencies
type: object
default:
- clr
- half
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- rocRAND
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
- name: downstreamComponentMatrix
type: object
default:
- MIVisionX:
name: MIVisionX
checkoutRepo: mivisionx_repo
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- MIOpen_build
- AMDMIGraphX:
name: AMDMIGraphX
checkoutRepo: amdmigraphx_repo
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- MIOpen_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: MIOpen
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_ubuntu2204_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
- name: ROCM_PATH parameters:
value: $(Agent.BuildDirectory)/rocm aptPackages: ${{ parameters.aptPackages }}
pool: ${{ variables.MEDIUM_BUILD_POOL }} # The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+)
workspace: # Upgrade boost from apt by following https://launchpad.net/~mhier/+archive/ubuntu/libboost-latest
clean: all - task: Bash@3
steps: displayName: 'update boost version'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml inputs:
parameters: targetType: inline
aptPackages: ${{ parameters.aptPackages }} script: sudo add-apt-repository ppa:mhier/libboost-latest -y
pipModules: ${{ parameters.pipModules }} - task: Bash@3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml displayName: 'install boost'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml inputs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml targetType: inline
parameters: script: sudo apt-get --yes install libboost1.83-dev libboost-system1.83-dev libboost-filesystem1.83-dev
checkoutRepo: ${{ parameters.checkoutRepo }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/miopen-get-ck-build.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
gpuTarget: ${{ job.target }} # CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Build and install other dependencies
retryCountOnTaskFailure: 3
inputs:
targetType: inline
workingDirectory: $(Agent.BuildDirectory)/s
script: |
sed -i '/composable_kernel/d' requirements.txt
mkdir -p $(Agent.BuildDirectory)/miopen-deps
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
cmake -P install_deps.cmake --prefix $(Agent.BuildDirectory)/miopen-deps --generator Ninja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DMIOPEN_BACKEND=HIP
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/miopen-deps
-DGPU_TARGETS=${{ job.target }}
-DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF
-DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF
-DCMAKE_BUILD_TYPE=Release
-DBUILD_TESTING=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- miopen-deps
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
timeoutInMinutes: 180
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/miopen-get-ck-build.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
${{ if parameters.triggerDownstreamJobs }}: parameters:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} extraBuildFlags: >-
- task: Bash@3 -DMIOPEN_BACKEND=HIP
displayName: Build and install other dependencies -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
retryCountOnTaskFailure: 3 -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
inputs: -DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF
targetType: inline -DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF
workingDirectory: $(Agent.BuildDirectory)/s -DCMAKE_BUILD_TYPE=Release
script: | -DBUILD_TESTING=ON
sed -i '/composable_kernel/d' requirements.txt -GNinja
mkdir -p $(Agent.BuildDirectory)/miopen-deps - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
cmake -P install_deps.cmake --prefix $(Agent.BuildDirectory)/miopen-deps
- task: CMake@1
displayName: 'MIOpen Test CMake Flags'
inputs:
cmakeArgs: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/s/bin;$(Agent.BuildDirectory)/miopen-deps
-DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DMIOPEN_BACKEND=HIP
-DMIOPEN_TEST_FLAGS=" --disable-verification-cache"
-DCMAKE_BUILD_TYPE=release
-DBUILD_DEV=OFF
-DMIOPEN_USE_MLIR=ON
-DMIOPEN_GPU_SYNC=OFF
$(Agent.BuildDirectory)/s
- task: Bash@3
displayName: 'MIOpen Test Build'
inputs:
targetType: inline
workingDirectory: build
script: |
cmake --build . --target tests -- -j$(nproc)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml --exclude-regex "test_rnn_seq_api|GPU_Conv2dTuningAsm_FP32|GPU_Conv2dTuningAsmBwdWrw_FP32"'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- miopen-deps
# - ${{ if parameters.triggerDownstreamJobs }}:
# - ${{ each component in parameters.downstreamComponentMatrix }}:
# - ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
# - template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
# parameters:
# checkoutRepo: ${{ component.checkoutRepo }}
# # sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
# buildDependsOn: ${{ component.buildDependsOn }}
# downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
# triggerDownstreamJobs: true
# unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,213 +1,99 @@
parameters: parameters:
- name: componentName
type: string
default: MIVisionX
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
# - name: sparseCheckoutDir
# type: string
# default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build - ninja-build
- git
- wget - wget
- unzip - unzip
- libstdc++-12-dev
- pkg-config - pkg-config
- protobuf-compiler
- libprotoc-dev
- ffmpeg - ffmpeg
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libswscale-dev - libswscale-dev
- build-essential
- libgtk2.0-dev
- libavcodec-dev
- libavformat-dev
- libswscale-dev
- libtbb2
- libtbb-dev
- libjpeg-dev
- libpng-dev
- libtiff-dev
- libdc1394-dev
- libgmp-dev - libgmp-dev
- libomp-dev
- libopencv-dev - libopencv-dev
- protobuf-compiler
- libprotoc-dev
- python3-pip
- name: pipModules - name: pipModules
type: object type: object
default: default:
- future==1.0.0 - future==0.18.2
- pytz==2022.1 - pytz==2022.1
- numpy==1.23 - numpy==1.21
- google==3.0.0 - google==3.0.0
- protobuf==3.12.4 - protobuf==3.12.4
- onnx==1.12.0 - onnx==1.12.0
- nnef==1.0.7
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- AMDMIGraphX
- clr
- half
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocBLAS
- rocDecode
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime
- rpp
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- half - half
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocBLAS - rocBLAS
- rocprofiler-register - MIOpen
- ROCR-Runtime - AMDMIGraphX
- roctracer
- rpp - rpp
- rocDecode
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: MIVisionX
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_ubuntu2204_${{ job.target }} vmImage: ${{ variables.BASE_BUILD_POOL }}
variables: workspace:
- group: common clean: all
- template: /.azuredevops/variables-global.yml steps:
pool: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
workspace: parameters:
clean: all aptPackages: ${{ parameters.aptPackages }}
steps: pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
pipModules: ${{ parameters.pipModules }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_DEP_ROCMCORE=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: LD_LIBRARY_PATH
value: $(Agent.BuildDirectory)/rocm/lib:$(Agent.BuildDirectory)/rocm/include/mivisionx/VX
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
${{ if parameters.triggerDownstreamJobs }}: parameters:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} extraBuildFlags: >-
- task: Bash@3 -DCMAKE_BUILD_TYPE=Release
displayName: Build MIVisionX tests -DROCM_PATH=$(Agent.BuildDirectory)/rocm
inputs: -DROCM_DEP_ROCMCORE=ON
targetType: inline -GNinja
script: | - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
mkdir mivisionx-tests
cd mivisionx-tests
cmake $(Agent.BuildDirectory)/rocm/share/mivisionx/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: 'mivisionx-tests'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -1,221 +1,56 @@
parameters: parameters:
- name: componentName
type: string
default: ROCR-Runtime
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - libelf-dev
- g++ - g++
- libdrm-dev - libdrm-dev
- libelf-dev
- libnuma-dev - libnuma-dev
- libsimde-dev
- ninja-build
- pkg-config
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- llvm-project - llvm-project
- ROCT-Thunk-Interface
- rocprofiler-register - rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- amdsmi
- llvm-project
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCR_Runtime
- job: ROCR_Runtime_build_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
vmImage: 'ubuntu-22.04' clean: all
${{ if eq(job.os, 'almalinux8') }}: steps:
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest parameters:
endpoint: ContainerService3 aptPackages: ${{ parameters.aptPackages }}
variables: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- group: common - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: /.azuredevops/variables-global.yml parameters:
workspace: checkoutRepo: ${{ parameters.checkoutRepo }}
clean: all # CI case: download latest default branch build
steps: - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: tag-builds
useAmdclang: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-DBUILD_SHARED_LIBS=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release cmakeBuildDir: 'src/build'
-GNinja - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ROCR_Runtime_test_${{ job.os }}_${{ job.target }}
dependsOn: ROCR_Runtime_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- task: Bash@3
displayName: Build kfdtest
inputs:
targetType: 'inline'
workingDirectory: $(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest
script: |
if [ -e /opt/rh/gcc-toolset-14/enable ]; then
source /opt/rh/gcc-toolset-14/enable
fi
mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm ..
make
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: kfdtest
testExecutable: BIN_DIR=$(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest/build ./run_kfdtest.sh
testParameters: '-p core --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Agent.BuildDirectory)/s/libhsakmt/tests/kfdtest/scripts
os: ${{ job.os }}
- task: Bash@3
displayName: Build rocrtst
inputs:
targetType: 'inline'
workingDirectory: $(Agent.BuildDirectory)/s/rocrtst/suites/test_common
script: |
echo $(Agent.BuildDirectory)/s/rocrtst/thirdparty/lib | sudo tee -a /etc/ld.so.conf.d/rocm-ci.conf
sudo cat /etc/ld.so.conf.d/rocm-ci.conf
sudo ldconfig -v
ldconfig -p
if [ -e /opt/rh/gcc-toolset-14/enable ]; then
source /opt/rh/gcc-toolset-14/enable
fi
BASE_CLANG_DIR=$(Agent.BuildDirectory)/rocm/llvm/lib/clang
export NEWEST_CLANG_VER=$(ls -1 $BASE_CLANG_DIR | sort -V | tail -n 1)
mkdir build && cd build
cmake .. \
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm \
-DTARGET_DEVICES=${{ job.target }} \
-DROCM_DIR=$(Agent.BuildDirectory)/rocm \
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin \
-DOPENCL_INC_DIR=$BASE_CLANG_DIR/$NEWEST_CLANG_VER/include
make
make rocrtst_kernels
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocrtst
testExecutable: ./rocrtst64
testParameters: '--gtest_filter="-rocrtstNeg.Memory_Negative_Tests:rocrtstFunc.Memory_Max_Mem" --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Agent.BuildDirectory)/s//rocrtst/suites/test_common/build/${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
# docker image will be missing libhwloc5

View File

@@ -16,7 +16,8 @@ jobs:
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: ${{ variables.LOW_BUILD_POOL }} pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
@@ -28,5 +29,4 @@ jobs:
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -23,57 +18,39 @@ parameters:
- rocminfo - rocminfo
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCdbgapi
- job: ROCdbgapi_build_${{ job.os }} variables:
pool: - group: common
vmImage: 'ubuntu-22.04' - template: /.azuredevops/variables-global.yml
${{ if eq(job.os, 'almalinux8') }}: pool:
container: vmImage: ${{ variables.BASE_BUILD_POOL }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest workspace:
endpoint: ContainerService3 clean: all
variables: steps:
- group: common - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
- template: /.azuredevops/variables-global.yml parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
packageManager: ${{ job.packageManager }} - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
useAmdclang: false dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_BUILD_TYPE=Release parameters:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-GNinja -DCMAKE_BUILD_TYPE=Release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
parameters: -GNinja
os: ${{ job.os }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -1,215 +1,35 @@
parameters: parameters:
- name: componentName
type: string
default: ROCgdb
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
# reference: https://github.com/ROCm/ROCgdb/blob/amd-staging/README-ROCM.md
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- bison
- cmake
- dejagnu
- flex
- libbabeltrace-dev
- libexpat-dev
- libgmp-dev - libgmp-dev
- liblzma-dev
- libmpfr-dev - libmpfr-dev
- ncurses-dev
- pkg-config
- python3-dev
- python3-pip
- texinfo - texinfo
- zlib1g-dev - bison
- name: rocmDependencies - flex
type: object
default:
- clr
- llvm-project
- ROCdbgapi
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocgdb
- job: ${{ parameters.componentName }}_build_${{ job.os }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
- name: PKG_CONFIG_PATH vmImage: ${{ variables.BASE_BUILD_POOL }}
value: $(Agent.BuildDirectory)/rocm/share/pkgconfig workspace:
pool: clean: all
${{ if eq(job.os, 'ubuntu2404') }}: steps:
name: rocm-ci_medium_build_pool_2404 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ else }}: parameters:
name: ${{ variables.MEDIUM_BUILD_POOL }} aptPackages: ${{ parameters.aptPackages }}
${{ if eq(job.os, 'almalinux8') }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest parameters:
endpoint: ContainerService3 checkoutRepo: ${{ parameters.checkoutRepo }}
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
parameters:
os: ${{ job.os }}
configureFlags: >-
--program-prefix=roc
--enable-64-bit-bfd
--enable-targets="x86_64-linux-gnu,amdgcn-amd-amdhsa"
--disable-ld
--disable-gas
--disable-gdbserver
--disable-sim
--enable-tui
--disable-gdbtk
--disable-shared
--disable-gprofng
--with-expat
--with-system-zlib
--without-guile
--with-babeltrace
--with-lzma
--with-python=python3
--with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm
LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib"
makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PKG_CONFIG_PATH
value: $(Agent.BuildDirectory)/rocm/share/pkgconfig
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
parameters:
os: ${{ job.os }}
configureFlags: >-
--program-prefix=roc
--enable-64-bit-bfd
--enable-targets="x86_64-linux-gnu,amdgcn-amd-amdhsa"
--disable-ld
--disable-gas
--disable-gdbserver
--disable-sim
--enable-tui
--disable-gdbtk
--disable-shared
--disable-gprofng
--with-expat
--with-system-zlib
--without-guile
--with-babeltrace
--with-lzma
--with-python=python3
--with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm
LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib"
makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib'
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
echo "##vso[task.prependpath]/opt/rocm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: check-gdb
continueOnError: true
inputs:
targetType: inline
script: |
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
make check-gdb TESTS=gdb.rocm/simple.exp
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: print gdb log
inputs:
targetType: inline
script: find -name gdb.log -exec cat {} \;
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
gpuTarget: ${{ job.target }}
extraEnvVars:
- PKG_CONFIG_PATH:::/home/user/workspace/rocm/share/pkgconfig

View File

@@ -5,33 +5,20 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build - ninja-build
- libdrm-dev
- libyaml-cpp-dev - libyaml-cpp-dev
- libpci-dev - libpci-dev
- libpci3 - libpci3
- libgst-dev - googletest
- libgtest-dev
- git - git
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- amdsmi
- aomp
- clr - clr
- hipBLAS-common
- hipBLASLt
- hipRAND
- llvm-project - llvm-project
- rocBLAS - rocBLAS
- rocm-cmake - rocm-cmake
@@ -39,134 +26,44 @@ parameters:
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocRAND - ROCT-Thunk-Interface
- name: rocmTestDependencies
type: object
default:
- amdsmi
- aomp
- clr
- hipBLAS-common
- hipBLASLt
- hipRAND
- llvm-project
- rocBLAS
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocRAND
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
confDir: MI300X
- gfx90a:
target: gfx90a
confDir: MI210
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: ROCmValidationSuite
- job: ROCmValidationSuite_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml - name: HIP_ROCCLR_HOME
- name: HIP_ROCCLR_HOME value: $(Build.BinariesDirectory)/rocm
value: $(Build.BinariesDirectory)/rocm pool:
- name: ROCM_PATH vmImage: ${{ variables.BASE_BUILD_POOL }}
value: $(Agent.BuildDirectory)/rocm workspace:
- name: HIP_INC_DIR clean: all
value: $(Agent.BuildDirectory)/rocm steps:
pool: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
workspace: parameters:
clean: all aptPackages: ${{ parameters.aptPackages }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/llvm/include
-DCPACK_PACKAGING_INSTALL_PREFIX=$(Build.BinariesDirectory)
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
# - ROCM_PATH:::/home/user/workspace/rocm
# - HIP_INC_DIR:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ROCmValidationSuite_test_${{ job.target }}
dependsOn: ROCmValidationSuite_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml extraBuildFlags: >-
parameters: -DROCM_PATH=$(Agent.BuildDirectory)/rocm
componentName: ROCmValidationSuite -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
testExecutable: $(Agent.BuildDirectory)/rocm/bin/rvs -c $(Agent.BuildDirectory)/rocm/share/rocm-validation-suite/conf/${{ job.confDir }}/gst_single.conf -DCPACK_PACKAGING_INSTALL_PREFIX=$(Build.BinariesDirectory)
testParameters: '' -GNinja
testDir: $(Agent.BuildDirectory) - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,221 +0,0 @@
parameters:
- name: componentName
type: string
default: Tensile
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libboost-filesystem-dev
- libboost-program-options-dev
- libmsgpack-dev
- name: pipModules
type: object
default:
- tox
- pytest
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn: ${{ parameters.buildDependsOn[job.target] }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: python3 setup.py bdist_wheel
workingDirectory: $(Agent.BuildDirectory)/s
- task: Bash@3
displayName: Rename wheel file with job OS
inputs:
targetType: inline
workingDirectory: $(Agent.BuildDirectory)/s
script: |
wheelFile=$(find "$(Agent.BuildDirectory)/s/dist" -type f -name "*.whl" | head -n 1)
newWheelFile="$(basename "$wheelFile" .whl)-${{ job.os }}.whl"
mv "$wheelFile" "$(dirname "$wheelFile")/$newWheelFile"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Agent.BuildDirectory)/s/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- task: Bash@3
displayName: Save pipeline artifact file names
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.ArtifactStagingDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: Tensile_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 180
dependsOn: Tensile_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*${{ job.os }}*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- task: Bash@3
displayName: pip install
inputs:
targetType: inline
script: find -name *.whl -exec pip install {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: tox test
inputs:
targetType: inline
script: tox run -v -e ci -- -m pre_checkin
workingDirectory: $(Agent.BuildDirectory)/s
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
pythonEnvVars: true
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
# docker image will not have python site-packages in path, but the env vars will make it easier

View File

@@ -1,142 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libnuma-dev
- ninja-build
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: TransferBench_build_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: TransferBench_test_${{ job.target }}
dependsOn: TransferBench_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: TransferBench All-to-all
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/TransferBench'
testParameters: 'a2a'
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: TransferBench Peer-to-peer
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/TransferBench'
testParameters: 'p2p'
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,110 +5,30 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- libdrm-dev - libdrm-dev
- ninja-build
- pkg-config
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: amdsmi
- job: amdsmi_build_${{ job.os }} variables:
pool: - group: common
${{ if eq(job.os, 'ubuntu2404') }}: - template: /.azuredevops/variables-global.yml
vmImage: 'ubuntu-24.04' pool:
${{ else }}: vmImage: ${{ variables.BASE_BUILD_POOL }}
vmImage: 'ubuntu-22.04' workspace:
${{ if eq(job.os, 'almalinux8') }}: clean: all
container: steps:
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
endpoint: ContainerService3 parameters:
variables: aptPackages: ${{ parameters.aptPackages }}
- group: common - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: extraBuildFlags: >-
aptPackages: ${{ parameters.aptPackages }} -DBUILD_TESTS=ON
packageManager: ${{ job.packageManager }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DBUILD_TESTS=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: amdsmi_test_${{ job.os }}_${{ job.target }}
dependsOn: amdsmi_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: amdsmi
testDir: '$(Agent.BuildDirectory)'
testExecutable: 'sudo ./rocm/share/amd_smi/tests/amdsmitst'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -40,11 +35,18 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRef: ${{ parameters.checkoutRef }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
dependencyList: ${{ parameters.rocmDependencies }} parameters:
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: aomp-extras componentName: aomp-extras
@@ -55,9 +57,4 @@ jobs:
-DAOMP_VERSION_STRING=$(LATEST_RELEASE_TAG) -DAOMP_VERSION_STRING=$(LATEST_RELEASE_TAG)
-GNinja -GNinja
installDir: $(Build.BinariesDirectory)/llvm installDir: $(Build.BinariesDirectory)/llvm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -1,280 +1,125 @@
parameters: parameters:
- name: componentName
type: string
default: aomp
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack - name: offloadEnabled
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean type: boolean
default: false default: false
# reference:
# https://github.com/ROCm/aomp/blob/aomp-dev/docs/SOURCEINSTALL_PREREQUISITE.md
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- bison
- ccache
- cmake - cmake
- flex - python3-pip
- gawk
- git
- mesa-common-dev
- ninja-build - ninja-build
- libbabeltrace-dev - pkg-config
- libbison-dev - libpci-dev
- libnuma-dev
- libffi-dev
- git
- libopenmpi-dev
- gawk
- mesa-common-dev
- libtool
- libdrm-amdgpu1 - libdrm-amdgpu1
- libdrm-dev - libdrm-dev
- libdw-dev - libdw-dev
- libffi-dev - libgtest-dev
- libgmp-dev - libsystemd-dev
- liblzma-dev
- libmpfr-dev
- libncurses5-dev
- libnuma-dev
- libopenmpi-dev
- libpci-dev
- libssl-dev - libssl-dev
- libstdc++-12-dev - libstdc++-12-dev
- libsystemd-dev
- libtool
- libudev-dev
- parallel
- pkg-config
- python3-dev
- python3-pip
- python3-setuptools
- texinfo
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocm-cmake
- llvm-project - llvm-project
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies - ROCT-Thunk-Interface
type: object
default:
- clr
- llvm-project
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: aomp
- job: ${{ parameters.componentName }}_build_${{ job.os }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'ubuntu2404') }}: workspace:
name: rocm-ci_medium_build_pool_2404 clean: all
${{ else }}: steps:
name: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: aptPackages: ${{ parameters.aptPackages }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: llvm-project_repo
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
packageManager: ${{ job.packageManager }} - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
# checkout the repos tied to openmp-extras, plus llvm-project
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: aomp-extras_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: flang_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: llvm-project_repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: llvm-openmp
extraBuildFlags: >-
-DOPENMP_ENABLE_LIBOMPTARGET=1
-DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DLIBOMPTARGET_AMDGCN_GFXLIST=gfx700;gfx701;gfx801;gfx803;gfx900;gfx902;gfx906;gfx908;gfx90a;gfx90c;gfx940;gfx941;gfx942;gfx1030;gfx1031;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103
-DDEVICELIBS_ROOT=$(Build.SourcesDirectory)/llvm-project/amd/device-libs
-DLIBOMP_COPY_EXPORTS=OFF
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm
-DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DLIBOMPTARGET_LLVM_INCLUDE_DIRS=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DCUDA_TOOLKIT_ROOT_DIR=OFF
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_SKIP_BUILD_RPATH=TRUE
-DCMAKE_SKIP_INSTALL_RPATH=TRUE
-DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN:$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-DCMAKE_MODULE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN:$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-GNinja
cmakeBuildDir: $(Build.SourcesDirectory)/llvm-project/openmp/build
installDir: $(Build.BinariesDirectory)/llvm
# offload does not exist for recent releases, so use CI conditional
- ${{ if eq(parameters.offloadEnabled, true) }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} componentName: llvm-offload
useAmdclang: false
componentName: extras
cmakeBuildDir: '$(Build.SourcesDirectory)/aomp-extras/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/aomp-extras'
installDir: '$(Build.BinariesDirectory)/llvm'
extraBuildFlags: >- extraBuildFlags: >-
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm
-DCMAKE_BUILD_TYPE=Release
-DAOMP_STANDALONE_BUILD=0
-DAOMP_VERSION_STRING=9.99.99
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
componentName: openmp
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm-project/openmp/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm-project/openmp'
installDir: '$(Build.BinariesDirectory)/llvm'
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Build.BinariesDirectory)"
-DCMAKE_BUILD_TYPE=Release
-DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DOPENMP_ENABLE_LIBOMPTARGET=1 -DOPENMP_ENABLE_LIBOMPTARGET=1
-DLIBOMP_COPY_EXPORTS=OFF
-DLIBOMP_OMPD_SUPPORT=ON
-DCMAKE_SKIP_INSTALL_RPATH=TRUE
-DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DLIBOMP_FORTRAN_MODULES_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/flang
-DLIBOMP_MODULES_INSTALL_PATH=$(Build.BinariesDirectory)/llvm/include/flang/
multithreadFlag: -- -j32
- task: Bash@3
displayName: 'ROCm symbolic link'
inputs:
targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
componentName: offload
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm-project/offload/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm-project/offload'
installDir: '$(Build.BinariesDirectory)/llvm'
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Build.BinariesDirectory)"
-DCMAKE_BUILD_TYPE=Release
-DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DOPENMP_TEST_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DOPENMP_TEST_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_SKIP_INSTALL_RPATH=TRUE -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DLIBOMPTARGET_AMDGCN_GFXLIST=gfx700;gfx701;gfx801;gfx803;gfx900;gfx902;gfx906;gfx908;gfx90a;gfx90c;gfx940;gfx941;gfx942;gfx1030;gfx1031;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm
-DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include -DLLVM_MAIN_INCLUDE_DIR=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DLIBOMPTARGET_LLVM_INCLUDE_DIRS=$(Build.SourcesDirectory)/llvm-project/llvm/include -DLIBOMPTARGET_LLVM_INCLUDE_DIRS=$(Build.SourcesDirectory)/llvm-project/llvm/include
-DCMAKE_EXE_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/llvm/lib" -DCUDA_TOOLKIT_ROOT_DIR=OFF
-DCMAKE_SHARED_LINKER_FLAGS="-L$(Agent.BuildDirectory)/rocm/llvm/lib" -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_SKIP_BUILD_RPATH=TRUE
-DCMAKE_SKIP_INSTALL_RPATH=TRUE
-DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN:$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-DCMAKE_MODULE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN:$ORIGIN/../lib:$ORIGIN/../../lib:$ORIGIN/../../../lib
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml cmakeBuildDir: $(Build.SourcesDirectory)/llvm-project/offload/build
parameters: installDir: $(Build.BinariesDirectory)/llvm
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
- task: Bash@3
displayName: ROCm symbolic link
inputs:
targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: aomp-extras_repo
# these copy steps are from the aomp prototype script for test prep
- task: CopyFiles@2
displayName: 'Copy AOMP contents'
inputs:
CleanTargetFolder: false
SourceFolder: $(Build.SourcesDirectory)/aomp
Contents: |
**
!**/.git/**
!**/.github/**
!**/.gitignore
TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests
retryCount: 3
- task: CopyFiles@2
displayName: 'Copy FileCheck'
inputs:
CleanTargetFolder: false
SourceFolder: $(Agent.BuildDirectory)/rocm/llvm/bin
Contents: FileCheck
TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests/bin
retryCount: 3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Test AOMP
continueOnError: true
inputs:
targetType: inline
script: ./run_rocm_test.sh
workingDirectory: $(Build.SourcesDirectory)/aomp/bin
env:
AOMP: $(Agent.BuildDirectory)/rocm/llvm
AOMP_REPOS_TEST: $(Build.SourcesDirectory)/aomp-test
AOMP_TEST_DIR: $(Build.SourcesDirectory)/aomp-test
SKIP_TEST_PACKAGE: 1
MAINLINE_BUILD: 1
SUITE_LIST: smoke
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -1,174 +0,0 @@
parameters:
- name: componentName
type: string
default: aqlprofile
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/aqlprofile/cmake_modules
-DAQLPROFILE_BUILD_TESTS=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/rocm/share/hsa-amd-aqlprofile/
testExecutable: ./run_tests.sh
testParameters: ''
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -17,8 +12,6 @@ parameters:
- ninja-build - ninja-build
- git - git
- python3-pip - python3-pip
- libdrm-dev
- ccache
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -28,139 +21,42 @@ parameters:
- clr - clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: composable_kernel
- job: composable_kernel_build_${{ job.target }} variables:
timeoutInMinutes: 240 - group: common
variables: - template: /.azuredevops/variables-global.yml
- group: common pool: ${{ variables.MEDIUM_BUILD_POOL }}
- template: /.azuredevops/variables-global.yml workspace:
- name: DAY_STRING clean: all
value: $[format('{0:ddMMyyyy}', pipeline.startTime)] steps:
pool: ${{ variables.ULTRA_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
workspace: parameters:
clean: all aptPackages: ${{ parameters.aptPackages }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- script: | - ${{ if ne(parameters.checkoutRef, '') }}:
mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache"
displayName: Update path for ccache
- task: Cache@2
displayName: Ccache caching
inputs:
key: composable_kernel | $(Agent.OS) | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR)
restoreKeys: |
composable_kernel | $(Agent.OS) | ${{ job.target }} | $(DAY_STRING)
composable_kernel | $(Agent.OS) | ${{ job.target }}
composable_kernel | $(Agent.OS)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_HIP_FLAGS="-Wno-missing-include-dirs"
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCK_BUILD_JIT_LIB=ON
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
# if this artifact name is changed, please also update $ARTIFACT_URL inside miopen-get-ck-build.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: composable_kernel_test_${{ job.target }}
timeoutInMinutes: 180
dependsOn: composable_kernel_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: TEST_LOG_FILE
value: $(Pipeline.Workspace)/ckTestLog.log
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml parameters:
- task: Bash@3 extraBuildFlags: >-
displayName: Iterate through test scripts -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
inputs: -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
targetType: inline -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
script: | -DCMAKE_BUILD_TYPE=Release
for file in ./test_*; do -DGPU_TARGETS=gfx1030;gfx1100
./$file | tee -a $(TEST_LOG_FILE) -GNinja
done - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
workingDirectory: $(Agent.BuildDirectory)/rocm/bin
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,44 +1,33 @@
parameters: parameters:
- name: jobMatrix - name: checkoutRepo
type: object type: string
default: default: 'self'
copyJobs: - name: checkoutRef
- { os: ubuntu2204, backend: amd } type: string
- { os: almalinux8, backend: amd } default: ''
- { os: ubuntu2204, backend: nvidia }
- { os: almalinux8, backend: nvidia }
# hip and clr are tightly-coupled # hip and clr are tightly-coupled
# run this same template for both repos # run this same template for both repos
# any changes for clr should just trigger HIP pipeline # any changes for clr should just trigger HIP pipeline
jobs: jobs:
- ${{ each job in parameters.jobMatrix.copyJobs }}: - job: hip_clr_combined
- job: hip_clr_combined_${{ job.os }}_${{ job.backend }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: vmImage: ${{ variables.BASE_BUILD_POOL }}
vmImage: ${{ variables.BASE_BUILD_POOL }} workspace:
workspace: clean: all
clean: all steps:
steps: # checkout nothing, just copy artifacts from triggering HIP job
# checkout nothing, just copy artifacts from triggering HIP job # and then publish for this clr job or for this hipother job to maintain latest
# and then publish for this clr job or for this hipother job to maintain latest - checkout: none
- checkout: none - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml parameters:
parameters: useDefaultBranch: false
componentName: HIP componentName: HIP
pipelineId: $(HIP_PIPELINE_ID) branchName: develop
fileFilter: ${{ job.os }}*${{ job.backend }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
- task: Bash@3 parameters:
displayName: Copy HIP artifacts sourceDir: $(Agent.BuildDirectory)/rocm
inputs: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
targetType: inline
script: cp -a $(Agent.BuildDirectory)/rocm/* $(Build.BinariesDirectory)/
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
inputs:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -42,31 +37,22 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRef: ${{ parameters.checkoutRef }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
dependencyList: ${{ parameters.rocmDependencies }} parameters:
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF -DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
# only run test03 because test11 has too many test cases, taking way too long
- task: Bash@3
displayName: test03
inputs:
targetType: inline
script: |
make clean
make test03
./bin/test
workingDirectory: $(Build.SourcesDirectory)/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: combined

View File

@@ -1,192 +0,0 @@
parameters:
- name: componentName
type: string
default: hip-tests
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ninja-build
- git
- libnuma-dev
- mesa-common-dev
- ocl-icd-libopencl1
- ocl-icd-opencl-dev
- opencl-headers
- libglu1-mesa-dev
- freeglut3-dev
- libboost-all-dev
- python3-pip
- name: rocmDependencies
type: object
default:
- llvm-project
- ROCR-Runtime
- clr
- rocminfo
- rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
# HIP with AMD backend
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: hip_tests_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: HIP_ROCCLR_HOME
value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
# compile hip-tests
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
cmakeSourceDir: '../catch'
customBuildTarget: build_tests
extraBuildFlags: >-
-DHIP_PLATFORM=amd
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DHIP_PATH=$(Agent.BuildDirectory)/rocm
-DOFFLOAD_ARCH_STR="--offload-arch=${{ job.target }}"
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: hip_tests_test_${{ job.target }}
timeoutInMinutes: 240
dependsOn: hip_tests_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Symlink rocm_agent_enumerator
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo mkdir -p /opt/rocm/bin
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/rocm/share/hip
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -1,135 +0,0 @@
parameters:
- name: componentName
type: string
default: hipBLAS-common
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- wget
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
- name: downstreamComponentMatrix
type: object
default:
- hipBLASLt:
name: hipBLASLt
sparseCheckoutDir: projects/hipblaslt
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLAS_common_build
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: hipBLAS_common_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# extraEnvVars:
# - ROCM_PATH:::/home/user/workspace/rocm
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipBLAS
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -40,193 +16,72 @@ parameters:
- libgtest-dev - libgtest-dev
- wget - wget
- python3-pip - python3-pip
- name: pipModules - libomp-dev
type: object
default:
- pyyaml
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- hipBLAS-common
- llvm-project
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- rocBLAS - rocBLAS
- rocSPARSE - rocSPARSE
- rocSOLVER - rocSOLVER
- name: rocmTestDependencies
type: object
default:
- aomp - aomp
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocm-cmake
- ROCR-Runtime
- rocminfo
- rocprofiler-register
- rocBLAS
- rocSPARSE
- rocSOLVER
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
# MIOpen depends on both rocRAND and hipBLAS
# for a unified build, hipBLAS will be the one to call MIOpen
- name: downstreamComponentMatrix
type: object
default:
- MIOpen:
name: MIOpen
sparseCheckoutDir: projects/miopen
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLAS_build
unifiedBuild:
downstreamAggregateNames: hipBLAS+rocRAND
buildDependsOn:
- hipBLAS_build
- rocRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipBLAS
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_ubuntu2204_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} - task: Bash@3
pipModules: ${{ parameters.pipModules }} displayName: 'Download AOCL'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml inputs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml targetType: inline
parameters: script: wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-1/aocl-linux-aocc-4.1.0_1_amd64.deb
checkoutRepo: ${{ parameters.checkoutRepo }} workingDirectory: '$(Pipeline.Workspace)'
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} - task: Bash@3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml displayName: 'Install AOCL'
inputs:
targetType: inline
script: sudo apt install --yes ./aocl-linux-aocc-4.1.0_1_amd64.deb
workingDirectory: '$(Pipeline.Workspace)'
# CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- dependencyList: ${{ parameters.rocmDependencies }}
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm dependencySource: tag-builds
-DCMAKE_BUILD_TYPE=Release - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ parameters:
-DHIP_PLATFORM=amd extraBuildFlags: >-
-DBUILD_CLIENTS_TESTS=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_CLIENTS_BENCHMARKS=OFF -DCMAKE_BUILD_TYPE=Release
-DBUILD_CLIENTS_SAMPLES=OFF -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCPACK_SET_DESTDIR=OFF -DAMDGPU_TARGETS=gfx1030;gfx1100
-GNinja -DHIP_PLATFORM=amd
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DBUILD_CLIENTS_TESTS=ON
parameters: -DBUILD_CLIENTS_BENCHMARKS=OFF
componentName: ${{ parameters.componentName }} -DBUILD_CLIENTS_SAMPLES=OFF
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} -DCPACK_SET_DESTDIR=OFF
gpuTarget: ${{ job.target }} -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
installAOCL: true
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testExecutable: $(Agent.BuildDirectory)/rocm/bin/hipblas-test
testParameters: '--yaml hipblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: '$(Agent.BuildDirectory)/rocm/bin'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,312 +1,88 @@
parameters: parameters:
- name: componentName
type: string
default: hipBLASLt
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- ccache
- gfortran
- git
- libboost-filesystem-dev
- libboost-program-options-dev
- libdrm-dev
- liblapack-dev
- libmsgpack-dev
- libnuma-dev
- libopenblas-dev
- ninja-build - ninja-build
- python3-pip
- python3-venv - python3-venv
- libmsgpack-dev
- git
- python3-pip
- libdrm-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- joblib - joblib
- "packaging>=22.0"
- pyyaml
- msgpack
- simplejson
- ujson
- orjson
- yappi
- --upgrade
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- hipBLAS-common
- llvm-project - llvm-project
- rocminfo
- rocm-cmake
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- clr - clr
- hipBLAS-common
- llvm-project
- rocm-cmake
- rocminfo - rocminfo
- rocm_smi_lib
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - hipBLAS
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { pool: rocm-ci_ultra_build_pool, os: ubuntu2204, packageManager: apt, target: gfx942 }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx90a }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { pool: rocm-ci_medium_build_pool, os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { pool: rocm-ci_ultra_build_pool, os: almalinux8, packageManager: dnf, target: gfx942 }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx90a }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1201 }
- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { pool: rocm-ci_medium_build_pool, os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocBLAS:
name: rocBLAS
sparseCheckoutDir: projects/rocblas
skipUnifiedBuild: 'false'
buildDependsOn:
- hipBLASLt_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipBLASLt
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
timeoutInMinutes: 300 - group: common
${{ if parameters.buildDependsOn }}: - template: /.azuredevops/variables-global.yml
dependsOn: - name: HIP_ROCCLR_HOME
- ${{ each build in parameters.buildDependsOn }}: value: $(Build.BinariesDirectory)/rocm
- ${{ build }}_${{ job.os }} - name: TENSILE_ROCM_ASSEMBLER_PATH
variables: value: $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
- group: common - name: CMAKE_CXX_COMPILER
- template: /.azuredevops/variables-global.yml value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: HIP_ROCCLR_HOME - name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH
value: $(Build.BinariesDirectory)/rocm value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler
- name: TENSILE_ROCM_ASSEMBLER_PATH - name: TENSILE_ROCM_PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH - name: PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
- name: ROCM_PATH pool: ${{ variables.MEDIUM_BUILD_POOL }}
value: $(Agent.BuildDirectory)/rocm workspace:
- name: DAY_STRING clean: all
value: $[format('{0:ddMMyyyy}', pipeline.startTime)] steps:
pool: ${{ job.pool }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: aptPackages: ${{ parameters.aptPackages }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest pipModules: ${{ parameters.pipModules }}
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
steps: parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml checkoutRepo: ${{ parameters.checkoutRepo }}
parameters: # CI case: download latest default branch build
aptPackages: ${{ parameters.aptPackages }} - ${{ if eq(parameters.checkoutRef, '') }}:
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
gpuTarget: ${{ job.target }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- task: Bash@3
displayName: Build and install LAPACK
inputs:
targetType: inline
script: |
mkdir -p $(Agent.BuildDirectory)/temp-deps
cd $(Agent.BuildDirectory)/temp-deps
# position-independent LAPACK is required for almalinux8 builds
cmake -DBUILD_GTEST=OFF -DBUILD_LAPACK=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON $(Agent.BuildDirectory)/sparse/projects/hipblaslt/deps
make -j
sudo make install
- script: |
mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache"
displayName: Update path for ccache
- task: Cache@2
displayName: Ccache caching
inputs:
key: hipBLASLt | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR)
restoreKeys: |
hipBLASLt | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING)
hipBLASLt | ${{ job.os }} | ${{ job.target }}
hipBLASLt | ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
cmakeSourceDir: $(Agent.BuildDirectory)/sparse/projects/hipblaslt dependencySource: tag-builds
cmakeBuildDir: $(Agent.BuildDirectory)/sparse/projects/hipblaslt/build - script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
extraBuildFlags: >- displayName: ROCm symbolic link
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/llvm/include parameters:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ extraBuildFlags: >-
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=gfx90a
-DGPU_TARGETS=${{ job.target }} -DTensile_LOGIC=
-DBUILD_CLIENTS_TESTS=ON -DTensile_CPU_THREADS=
-DHIPBLASLT_ENABLE_ROCROLLER=ON -DTensile_CODE_OBJECT_VERSION=default
-DHIPBLASLT_ENABLE_FETCH=ON -DTensile_LIBRARY_FORMAT=msgpack
-DHIPBLASLT_ENABLE_BLIS=OFF -DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
installLatestCMake: true
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/amdclang
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
- ROCM_PATH:::/home/user/workspace/rocm
extraCopyDirectories:
- deps
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 300
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipblaslt-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,40 +1,17 @@
parameters: parameters:
- name: componentName
type: string
default: hipCUB
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- googletest
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
@@ -44,150 +21,42 @@ parameters:
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocPRIM
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipCUB
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_${{ job.os }}_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
endpoint: ContainerService3 parameters:
workspace: checkoutRepo: ${{ parameters.checkoutRepo }}
clean: all # CI case: download latest default branch build
steps: - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
consolidateBuildAndInstall: true dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ parameters:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DBUILD_BENCHMARK=ON -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DBUILD_TEST=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=${{ job.target }} -DBUILD_TEST=ON
-GNinja -DAMDGPU_TARGETS=gfx1030;gfx1100
extraCxxFlags: -Wno-deprecated-declarations -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin/hipcub'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipFFT
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -38,11 +14,11 @@ parameters:
- libgtest-dev - libgtest-dev
- libfftw3-dev - libfftw3-dev
- python3-pip - python3-pip
- libomp-14-dev
# rocm dependencies should match dependencies-rocm.yml # rocm dependencies should match dependencies-rocm.yml
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- rocRAND - rocRAND
- hipRAND - hipRAND
- llvm-project - llvm-project
@@ -51,137 +27,50 @@ parameters:
- rocminfo - rocminfo
- rocFFT - rocFFT
- aomp - aomp
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- hipRAND
- llvm-project
- rocFFT
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocRAND
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipFFT
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_ubuntu2204_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: pool:
- group: common vmImage: ${{ variables.BASE_BUILD_POOL }}
- template: /.azuredevops/variables-global.yml workspace:
- name: HIP_ROCCLR_HOME clean: all
value: $(Build.BinariesDirectory)/rocm steps:
pool: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
vmImage: ${{ variables.BASE_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }}
-DUSE_HIP_CLANG=ON
-DHIP_COMPILER=clang
-DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
${{ if parameters.triggerDownstreamJobs }}: parameters:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} extraBuildFlags: >-
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
parameters: -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
componentName: ${{ parameters.componentName }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
testDir: '$(Agent.BuildDirectory)/rocm/bin' -DCMAKE_BUILD_TYPE=Release
testExecutable: './hipfft-test' -DAMDGPU_TARGETS=gfx1030;gfx1100
testParameters: '--test_prob 0.002 --gtest_output=xml:./test_output.xml --gtest_color=yes' -DUSE_HIP_CLANG=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml -DHIP_COMPILER=clang
parameters: -DBUILD_CLIENTS_TESTS=ON
aptPackages: ${{ parameters.aptPackages }} -DBUILD_CLIENTS_BENCHMARKS=OFF
environment: test -DBUILD_CLIENTS_SAMPLES=OFF
gpuTarget: ${{ job.target }} -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -1,219 +1,66 @@
parameters: parameters:
- name: componentName
type: string
default: hipRAND
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- python3-pip - googletest
- git
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project - llvm-project
- rocm-cmake - ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocRAND - rocRAND
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- rocRAND
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocFFT:
name: rocFFT
sparseCheckoutDir: projects/rocfft
skipUnifiedBuild: 'false'
buildDependsOn:
- hipRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipRAND
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: pool:
- group: common vmImage: ${{ variables.BASE_BUILD_POOL }}
- template: /.azuredevops/variables-global.yml workspace:
- name: HIP_ROCCLR_HOME clean: all
value: $(Build.BinariesDirectory)/rocm steps:
pool: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: aptPackages: ${{ parameters.aptPackages }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
consolidateBuildAndInstall: true dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ parameters:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang extraBuildFlags: >-
-DBUILD_TEST=ON -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DBUILD_TEST=ON
-DCMAKE_BUILD_TYPE=Release -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DAMDGPU_TARGETS=${{ job.target }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -DCMAKE_BUILD_TYPE=Release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DAMDGPU_TARGETS=gfx1030;gfx1100
parameters: -GNinja
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin/hipRAND'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipSOLVER
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -39,7 +15,6 @@ parameters:
- git - git
- googletest - googletest
- libgtest-dev - libgtest-dev
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -52,147 +27,55 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocSPARSE - rocSPARSE
- rocSOLVER - rocSOLVER
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- hipBLAS-common
- hipBLASLt
- hipSPARSE
- rocBLAS
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocSOLVER
- rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSOLVER
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_ubuntu2204_${{ job.target }} vmImage: ${{ variables.BASE_BUILD_POOL }}
variables: workspace:
- group: common clean: all
- template: /.azuredevops/variables-global.yml steps:
pool: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
workspace: parameters:
clean: all aptPackages: ${{ parameters.aptPackages }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
# build external gtest and lapack
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: external dependencyList: ${{ parameters.rocmDependencies }}
cmakeBuildDir: '$(Agent.BuildDirectory)/s/deps/build' dependencySource: tag-builds
cmakeSourceDir: '$(Agent.BuildDirectory)/s/deps' # build external gtest and lapack
installDir: '$(Pipeline.Workspace)/deps-install' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DBUILD_BOOST=OFF componentName: external
-DBUILD_GTEST=OFF cmakeBuildDir: 'deps/build'
-GNinja installDir: '$(Pipeline.Workspace)/deps-install'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml extraBuildFlags: >-
parameters: -DBUILD_BOOST=OFF
extraBuildFlags: >- -DBUILD_GTEST=OFF
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install -GNinja
-DCMAKE_BUILD_TYPE=Release - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ parameters:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang extraBuildFlags: >-
-DBUILD_CLIENTS_TESTS=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install
-DUSE_CUDA=OFF -DCMAKE_BUILD_TYPE=Release
-GNinja -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
parameters: -DAMDGPU_TARGETS=gfx1030;gfx1100
gpuTarget: ${{ job.target }} -DBUILD_CLIENTS_TESTS=ON
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} -DUSE_CUDA=OFF
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml -GNinja
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraCopyDirectories:
# - deps-install
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsolver-test'
testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,44 +1,21 @@
parameters: parameters:
- name: componentName
type: string
default: hipSPARSE
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- gfortran
- git
- libboost-program-options-dev
- libfftw3-dev
- ninja-build - ninja-build
- python3-pip - libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- gfortran
- libgtest-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -48,177 +25,53 @@ parameters:
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- rocSPARSE - rocSPARSE
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- hipBLAS-common
- hipBLASLt
- rocBLAS
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipSPARSELt:
name: hipSPARSELt
sparseCheckoutDir: projects/hipsparselt
skipUnifiedBuild: 'false'
buildDependsOn:
- hipSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSPARSE
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }}_${{ job.target }} vmImage: ${{ variables.BASE_BUILD_POOL }}
variables: workspace:
- group: common clean: all
- template: /.azuredevops/variables-global.yml steps:
pool: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
vmImage: ${{ variables.BASE_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
packageManager: ${{ job.packageManager }} - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
extraBuildFlags: >- dependencySource: tag-builds
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ parameters:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DBUILD_CLIENTS_TESTS=ON -DCMAKE_BUILD_TYPE=Release
-DBUILD_CLIENTS_SAMPLES=OFF -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
-GNinja -DBUILD_CLIENTS_TESTS=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DBUILD_CLIENTS_SAMPLES=OFF
parameters: -GNinja
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} parameters:
gpuTarget: ${{ job.target }} artifactName: hipSPARSE
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml publish: false
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
componentName: ${{ parameters.componentName }} parameters:
gpuTarget: ${{ job.target }} sourceDir: $(Build.SourcesDirectory)/build/clients
os: ${{ job.os }} contentsString: matrices/**
publish: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml parameters:
parameters: artifactName: testMatrices
sourceDir: $(Agent.BuildDirectory)/s/build/clients
contentsString: matrices/**
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
artifactName: testMatrices
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: test
# gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsparse-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: hipSPARSELt
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -37,228 +13,74 @@ parameters:
- libmsgpack-dev - libmsgpack-dev
- git - git
- python3-pip - python3-pip
- gfortran
- libgfortran5
- libopenblas-dev
- liblapack-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- joblib - joblib
- msgpack # rocm dependencies should match dependencies-rocm.yml
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp - llvm-project
- ROCR-Runtime
- clr - clr
- rocminfo
- rocprofiler-register
- hipSPARSE - hipSPARSE
- llvm-project
- rocBLAS - rocBLAS
- rocm-cmake
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- llvm-project
- hipBLAS-common
- hipBLASLt
- rocm-cmake
- rocBLAS
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipSPARSELt
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: - name: TENSILE_ROCM_ASSEMBLER_PATH
- group: common value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang
- template: /.azuredevops/variables-global.yml - name: CMAKE_CXX_COMPILER
- name: HIP_ROCCLR_HOME value: $(Agent.BuildDirectory)/rocm/llvm/bin/hipcc
value: $(Build.BinariesDirectory)/rocm - name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH
- name: TENSILE_ROCM_ASSEMBLER_PATH value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang - name: PATH
- name: CMAKE_CXX_COMPILER value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
value: $(Agent.BuildDirectory)/rocm/llvm/bin/hipcc pool: ${{ variables.MEDIUM_BUILD_POOL }}
- name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH workspace:
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler clean: all
- name: PATH steps:
value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all pipModules: ${{ parameters.pipModules }}
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
pipModules: ${{ parameters.pipModules }} checkoutRepo: ${{ parameters.checkoutRepo }}
packageManager: ${{ job.packageManager }} # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# ignore sparse checkout for monorepo case, we want access to hipblaslt directory
# sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
# NOTE: content between `---` is for transition support between old/new build systems
# and should be removed once transition is complete.
# -----------------------------
# Build and install gtest and lapack
# $(Pipeline.Workspace)/deps is a temporary folder for the build process
# $(Pipeline.Workspace)/s/deps is part of the hipSPARSELt repo
- script: mkdir -p $(Pipeline.Workspace)/deps
displayName: Create temp folder for external dependencies
# hipSPARSELt already has a CMake script for external deps, so we can just run that
# https://github.com/ROCm/hipSPARSELt/blob/develop/deps/CMakeLists.txt
- ${{ if ne(parameters.sparseCheckoutDir, '') }}:
script: cmake $(Pipeline.Workspace)/s/projects/hipsparselt/deps
${{ else }}:
script: cmake $(Pipeline.Workspace)/s/deps
displayName: Configure hipSPARSELt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps
- script: make
displayName: Build hipSPARSELt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps
- script: sudo make install
displayName: Install hipSPARSELt external dependencies
workingDirectory: $(Pipeline.Workspace)/deps
# -----------------------------
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
# NOTE: the following options are old build only dependencySource: tag-builds
# and can be removed after full transition to new build - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
# -DAMDGPU_TARGETS=${{ job.target }} parameters:
# -DCMAKE_Fortran_COMPILER=f95 extraBuildFlags: >-
# -DTensile_LOGIC= -DCMAKE_BUILD_TYPE=Release
# -DTensile_CPU_THREADS= -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
# -DTensile_LIBRARY_FORMAT=msgpack -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
# -DROCM_PATH=$(Agent.BuildDirectory)/rocm -DAMDGPU_TARGETS=all
# -DBUILD_CLIENTS_TESTS=ON -DTensile_LOGIC=
# -DBUILD_USE_LOCAL_TENSILE=OFF -DTensile_CPU_THREADS=
extraBuildFlags: >- -DTensile_CODE_OBJECT_VERSION=default
-DCMAKE_BUILD_TYPE=Release -DTensile_LIBRARY_FORMAT=msgpack
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm" -GNinja
-DGPU_TARGETS=${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
-DAMDGPU_TARGETS=${{ job.target }}
-DCMAKE_Fortran_COMPILER=f95
-DTensile_LOGIC=
-DTensile_CPU_THREADS=
-DTensile_LIBRARY_FORMAT=msgpack
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_CLIENTS_TESTS=ON
-DBUILD_USE_LOCAL_TENSILE=OFF
-DHIPSPARSELT_ENABLE_FETCH=ON
-GNinja
${{ if ne(parameters.sparseCheckoutDir, '') }}:
cmakeSourceDir: $(Build.SourcesDirectory)/projects/hipsparselt
cmakeBuildDir: $(Build.SourcesDirectory)/projects/hipsparselt
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- deps
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang
- CMAKE_CXX_COMPILER:::/home/user/workspace/rocm/llvm/bin/hipcc
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
installLatestCMake: true
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './hipsparselt-test'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes --gtest_filter=*pre_checkin*'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -26,113 +21,43 @@ parameters:
- clr - clr
- rocminfo - rocminfo
- composable_kernel - composable_kernel
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipTensor
- job: hipTensor_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool: ${{ variables.MEDIUM_BUILD_POOL }}
pool: ${{ variables.MEDIUM_BUILD_POOL }} workspace:
workspace: clean: all
clean: all steps:
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: aptPackages: ${{ parameters.aptPackages }}
aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
checkoutRepo: ${{ parameters.checkoutRepo }} # CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/llvm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DHIPTENSOR_BUILD_TESTS=ON
-DAMDGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: hipTensor_test_${{ job.target }}
timeoutInMinutes: 90
dependsOn: hipTensor_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml extraBuildFlags: >-
parameters: -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/llvm
componentName: hipTensor -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
testDir: '$(Agent.BuildDirectory)/rocm/bin/hiptensor' -DROCM_PATH=$(Agent.BuildDirectory)/rocm
testParameters: '-E ".*-extended" --extra-verbose --output-on-failure --force-new-ctest-process --output-junit test_output.xml' -DCMAKE_BUILD_TYPE=Release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml -DHIPTENSOR_BUILD_TESTS=ON
parameters: -DAMDGPU_TARGETS=gfx90a
aptPackages: ${{ parameters.aptPackages }} multithreadFlag: -- -j32
environment: test - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
gpuTarget: ${{ job.target }}

View File

@@ -5,162 +5,32 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- gfortran - gfortran
- git - git
- cmake - cmake
- python3-pip
- python3-dev
- ninja-build
- name: rocmDependencies
type: object
default:
- aomp
- llvm-project
- clr
- hipBLAS
- hipBLAS-common
- hipBLASLt
- hipFFT
- hipRAND
- hipSOLVER
- hipSPARSE
- rocBLAS
- rocFFT
- rocm-cmake
- rocminfo
- rocprofiler
- rocprofiler-register
- rocRAND
- ROCR-Runtime
- rocSOLVER
- rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: hipfort
- job: hipfort_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: ${{ variables.MEDIUM_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
checkoutRepo: ${{ parameters.checkoutRepo }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml parameters:
parameters: extraBuildFlags: >-
checkoutRef: ${{ parameters.checkoutRef }} -DHIPFORT_INSTALL_DIR=/tmp/hipfort
dependencyList: ${{ parameters.rocmDependencies }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/llvm
-DHIPFORT_INSTALL_DIR=$(Build.BinariesDirectory)
-DHIPFORT_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/flang
-DCMAKE_Fortran_FLAGS="-Mfree -fPIC"
-DCMAKE_Fortran_FLAGS_DEBUG=""
-DHIPFORT_COMPILER_FLAGS="-cpp"
-DHIPFORT_AR=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ar
-DHIPFORT_RANLIB=$(Agent.BuildDirectory)/rocm/llvm/bin/llvm-ranlib
-DAMDGPU_TARGETS=${{ job.target }}
-DBUILD_TESTING=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
installLatestCMake: true
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: hipfort_test_${{ job.target }}
dependsOn: hipfort_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- task: Bash@3
displayName: 'ROCm symbolic link'
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: 'Test hipfort'
inputs:
targetType: inline
script: PATH=$(Agent.BuildDirectory)/rocm/bin:$PATH make run_all
workingDirectory: $(Build.SourcesDirectory)/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
optSymLink: true

View File

@@ -5,196 +5,132 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- python3-pip
- libnuma-dev - libnuma-dev
- ninja-build - ninja-build
- pkg-config
- python-is-python3 - python-is-python3
- python3-pip
- zlib1g-dev - zlib1g-dev
- pkg-config
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocm-cmake - rocm-cmake
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
# - { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: llvm_project
- job: llvm_project_${{ job.os }} variables:
pool: - group: common
${{ if eq(job.os, 'ubuntu2404') }}: - template: /.azuredevops/variables-global.yml
name: 'rocm-ci_high_build_pool_2404' #temporarily using 'high' pool while 'ultra' is down - name: HIP_DEVICE_LIB_PATH
${{ else }}: value: '$(Build.BinariesDirectory)/amdgcn/bitcode'
name: 'rocm-ci_ultra_build_pool' pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}: workspace:
container: clean: all
image: rocmexternalcicd.azurecr.io/manylinux228:latest steps:
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
variables: parameters:
- group: common aptPackages: ${{ parameters.aptPackages }}
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- name: HIP_DEVICE_LIB_PATH - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
value: '$(Build.BinariesDirectory)/amdgcn/bitcode' parameters:
- name: HIP_PATH checkoutRepo: ${{ parameters.checkoutRepo }}
value: '$(Agent.BuildDirectory)/rocm' # CI case: download latest default branch build
workspace: - ${{ if eq(parameters.checkoutRef, '') }}:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
componentName: rocm-llvm dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: tag-builds
useAmdclang: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DCMAKE_PREFIX_PATH="$(Build.BinariesDirectory)/llvm;$(Build.BinariesDirectory)" componentName: rocm-llvm
-DCMAKE_BUILD_TYPE=Release extraBuildFlags: >-
-DLLVM_ENABLE_PROJECTS=clang;lld;clang-tools-extra;mlir;flang -DCMAKE_PREFIX_PATH="$(Build.BinariesDirectory)/llvm;$(Build.BinariesDirectory)"
-DLLVM_ENABLE_RUNTIMES=compiler-rt;libunwind;libcxx;libcxxabi -DCMAKE_BUILD_TYPE=Release
-DCLANG_ENABLE_AMDCLANG=ON -DLLVM_ENABLE_PROJECTS=clang;lld;clang-tools-extra;mlir
-DLLVM_TARGETS_TO_BUILD=AMDGPU;X86 -DLLVM_ENABLE_RUNTIMES=compiler-rt;libunwind;libcxx;libcxxabi
-DLIBCXX_ENABLE_SHARED=OFF -DCLANG_ENABLE_AMDCLANG=ON
-DLIBCXX_ENABLE_STATIC=ON -DLLVM_TARGETS_TO_BUILD=AMDGPU;X86
-DLIBCXX_INSTALL_LIBRARY=OFF -DLIBCXX_ENABLE_SHARED=OFF
-DLIBCXX_INSTALL_HEADERS=OFF -DLIBCXX_ENABLE_STATIC=ON
-DLIBCXXABI_ENABLE_SHARED=OFF -DLIBCXX_INSTALL_LIBRARY=OFF
-DLIBCXXABI_ENABLE_STATIC=ON -DLIBCXX_INSTALL_HEADERS=OFF
-DLIBCXXABI_INSTALL_STATIC_LIBRARY=OFF -DLIBCXXABI_ENABLE_SHARED=OFF
-DLLVM_BUILD_DOCS=OFF -DLIBCXXABI_ENABLE_STATIC=ON
-DLLVM_ENABLE_SPHINX=OFF -DLIBCXXABI_INSTALL_STATIC_LIBRARY=OFF
-DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_BUILD_DOCS=OFF
-DLLVM_ENABLE_Z3_SOLVER=OFF -DLLVM_ENABLE_SPHINX=OFF
-DLLVM_ENABLE_ZLIB=ON -DLLVM_ENABLE_ASSERTIONS=OFF
-DCLANG_DEFAULT_LINKER=lld -DLLVM_ENABLE_Z3_SOLVER=OFF
-DCLANG_DEFAULT_RTLIB=compiler-rt -DLLVM_ENABLE_ZLIB=ON
-DCLANG_DEFAULT_UNWINDLIB=libgcc -DCLANG_DEFAULT_LINKER=lld
-DSANITIZER_AMDGPU=OFF -DCLANG_DEFAULT_RTLIB=compiler-rt
-DPACKAGE_VENDOR=AMD -DCLANG_DEFAULT_UNWINDLIB=libgcc
-DCLANG_LINK_FLANG_LEGACY=ON -DSANITIZER_AMDGPU=OFF
-DCMAKE_CXX_STANDARD=17 -DPACKAGE_VENDOR=AMD
-DROCM_LLVM_BACKWARD_COMPAT_LINK=$(Build.BinariesDirectory)/llvm -DCLANG_LINK_FLANG_LEGACY=ON
-DROCM_LLVM_BACKWARD_COMPAT_LINK_TARGET=./lib/llvm -DCMAKE_CXX_STANDARD=17
-GNinja -DROCM_LLVM_BACKWARD_COMPAT_LINK=$(Build.BinariesDirectory)/llvm
cmakeBuildDir: '$(Build.SourcesDirectory)/llvm/build' -DROCM_LLVM_BACKWARD_COMPAT_LINK_TARGET=./lib/llvm
cmakeSourceDir: '$(Build.SourcesDirectory)/llvm' -GNinja
installDir: '$(Build.BinariesDirectory)/llvm' cmakeBuildDir: 'llvm/build'
# use llvm-lit to run unit tests for llvm, clang, and lld installDir: '$(Build.BinariesDirectory)/llvm'
- task: Bash@3 # use llvm-lit to run unit tests for llvm, clang, and lld
displayName: 'Copy llvm-lit to install directory' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
inputs: parameters:
targetType: inline componentName: check-llvm
script: | testDir: 'llvm/build'
cp $(Build.SourcesDirectory)/llvm/build/bin/llvm-lit $(Build.BinariesDirectory)/llvm/bin/ testExecutable: './bin/llvm-lit'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml testParameters: '-q --xunit-xml-output=llvm_test_output.xml ./test'
parameters: testOutputFile: llvm_test_output.xml
componentName: check-llvm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
testDir: 'llvm/build' parameters:
testExecutable: './bin/llvm-lit' componentName: check-clang
testParameters: '-q --xunit-xml-output=llvm_test_output.xml --filter-out="live-debug-values-spill-tracking" ./test' testDir: 'llvm/build'
testOutputFile: llvm_test_output.xml testExecutable: './bin/llvm-lit'
os: ${{ job.os }} testParameters: '-q --xunit-xml-output=clang_test_output.xml ./tools/clang/test'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml testOutputFile: clang_test_output.xml
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
componentName: check-clang parameters:
testDir: 'llvm/build' componentName: check-lld
testExecutable: './bin/llvm-lit' testDir: 'llvm/build'
testParameters: '-q --xunit-xml-output=clang_test_output.xml ./tools/clang/test' testExecutable: './bin/llvm-lit'
testOutputFile: clang_test_output.xml testParameters: '-q --xunit-xml-output=lld_test_output.xml ./tools/lld/test'
os: ${{ job.os }} testOutputFile: lld_test_output.xml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: check-lld componentName: device-libs
testDir: 'llvm/build' extraBuildFlags: >-
testExecutable: './bin/llvm-lit' -DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build"
testParameters: '-q --xunit-xml-output=lld_test_output.xml ./tools/lld/test' -DCMAKE_BUILD_TYPE=Release
testOutputFile: lld_test_output.xml cmakeBuildDir: 'amd/device-libs/build'
os: ${{ job.os }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- task: CopyFiles@2 parameters:
displayName: Copy FileCheck for Publishing componentName: comgr
inputs: extraBuildFlags: >-
CleanTargetFolder: false -DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build;$(Build.SourcesDirectory)/amd/device-libs/build"
SourceFolder: llvm/build/bin -DCMAKE_BUILD_TYPE=Release
Contents: FileCheck cmakeBuildDir: 'amd/comgr/build'
TargetFolder: $(Build.BinariesDirectory)/llvm/bin - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
retryCount: 3 parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml componentName: comgr
parameters: testParameters: '--output-on-failure --force-new-ctest-process --output-junit comgr_test_output.xml'
componentName: device-libs testDir: 'amd/comgr/build'
os: ${{ job.os }} testOutputFile: comgr_test_output.xml
useAmdclang: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build" componentName: hipcc
-DCMAKE_BUILD_TYPE=Release extraBuildFlags: >-
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/device-libs/build' -DCMAKE_BUILD_TYPE=Release
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/device-libs' -DHIPCC_BACKWARD_COMPATIBILITY=OFF
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml cmakeBuildDir: 'amd/hipcc/build'
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
componentName: comgr
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Build.SourcesDirectory)/llvm/build;$(Build.SourcesDirectory)/amd/device-libs/build"
-DCOMGR_DISABLE_SPIRV=1
-DCMAKE_BUILD_TYPE=Release
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/comgr/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/comgr'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: comgr
testParameters: '--output-on-failure --force-new-ctest-process --output-junit comgr_test_output.xml'
testDir: 'amd/comgr/build'
testOutputFile: comgr_test_output.xml
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: hipcc
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DHIPCC_BACKWARD_COMPATIBILITY=OFF
cmakeBuildDir: '$(Build.SourcesDirectory)/amd/hipcc/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/amd/hipcc'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: combined
extraEnvVars:
- HIP_DEVICE_LIB_PATH:::/home/user/workspace/bin/amdgcn/bitcode
- HIP_PATH:::/home/user/workspace/rocm

View File

@@ -1,251 +0,0 @@
parameters:
- name: componentName
type: string
default: origami
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- wget
- python3
- python3-dev
- python3-pip
- libgtest-dev
- libboost-filesystem-dev
- libboost-program-options-dev
- name: pipModules
type: object
default:
- nanobind>=2.0.0
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipBLASLt:
name: hipBLASLt
sparseCheckoutDir: projects/hipblaslt
skipUnifiedBuild: 'false'
buildDependsOn:
- origami_build
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: origami_build_${{ job.os }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DORIGAMI_BUILD_SHARED_LIBS=ON
-DORIGAMI_ENABLE_PYTHON=ON
-DORIGAMI_BUILD_TESTING=ON
-GNinja
- ${{ if ne(job.os, 'almalinux8') }}:
- task: PublishPipelineArtifact@1
displayName: 'Publish Build Directory Artifact'
inputs:
targetPath: '$(Agent.BuildDirectory)/s/build'
artifact: '${{ parameters.componentName }}_${{ job.os }}_build_dir'
publishLocation: 'pipeline'
- task: PublishPipelineArtifact@1
displayName: 'Publish Python Source Artifact'
inputs:
targetPath: '$(Agent.BuildDirectory)/s/python'
artifact: '${{ parameters.componentName }}_${{ job.os }}_python_src'
publishLocation: 'pipeline'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: origami_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: origami_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
- task: DownloadPipelineArtifact@2
displayName: 'Download Build Directory Artifact'
inputs:
artifact: '${{ parameters.componentName }}_${{ job.os }}_build_dir'
path: '$(Agent.BuildDirectory)/s/build'
- task: DownloadPipelineArtifact@2
displayName: 'Download Python Source Artifact'
inputs:
artifact: '${{ parameters.componentName }}_${{ job.os }}_python_src'
path: '$(Agent.BuildDirectory)/s/python'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './origami-tests'
testParameters: '--yaml origami-tests.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
- script: |
set -e
export PYTHONPATH=$(Agent.BuildDirectory)/s/build/python:$PYTHONPATH
echo "--- Running origami_test.py ---"
python3 $(Agent.BuildDirectory)/s/python/origami_test.py
echo "--- Running origami_grid_test.py ---"
python3 $(Agent.BuildDirectory)/s/python/origami_grid_test.py
displayName: 'Run Python Binding Tests'
condition: succeeded()
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,242 +1,77 @@
parameters: parameters:
- name: componentName
type: string
default: rccl
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
- name: systemsRepo
type: string
default: systems_repo
- name: systemsSparseCheckoutDir
type: string
default: 'projects/rocprofiler-sdk'
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git
- libboost-program-options-dev
- libdrm-dev
- libfftw3-dev
- libnuma-dev
- libstdc++-12-dev
- ninja-build
- python3-pip - python3-pip
- cmake
- libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- ninja-build
- libstdc++-12-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- rocm-core
- HIPIFY
- aomp - aomp
- aomp-extras - aomp-extras
- clr
- HIPIFY
- llvm-project
- rocm-cmake
- rocm-core
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- aomp-extras
- clr
- HIPIFY
- llvm-project
- rocm-cmake
- rocm-core
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocprofiler-sdk:
name: rocprofiler-sdk
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- rccl_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rccl
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
timeoutInMinutes: 120 pool: ${{ variables.MEDIUM_BUILD_POOL }}
variables: workspace:
- group: common clean: all
- template: /.azuredevops/variables-global.yml steps:
- name: HIP_ROCCLR_HOME - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
value: $(Build.BinariesDirectory)/rocm parameters:
pool: ${{ variables.MEDIUM_BUILD_POOL }} aptPackages: ${{ parameters.aptPackages }}
${{ if eq(job.os, 'almalinux8') }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest parameters:
endpoint: ContainerService3 checkoutRepo: ${{ parameters.checkoutRepo }}
workspace: # CI case: download latest default branch build
clean: all - ${{ if eq(parameters.checkoutRef, '') }}:
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
submoduleBehaviour: recursive
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
gpuTarget: ${{ job.target }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
extraBuildFlags: >- dependencySource: tag-builds
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc - script: chmod +x $(Agent.BuildDirectory)/rocm/bin/hipify-perl
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_BUILD_TYPE=Release parameters:
-DROCM_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-DBUILD_TESTS=ON -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake;$(Agent.BuildDirectory)/rocm/libexec/hipify -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DGPU_TARGETS=${{ job.target }} -DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-GNinja -DCMAKE_BUILD_TYPE=Release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DROCM_PATH=$(Agent.BuildDirectory)/rocm
parameters: -DBUILD_TESTS=ON
componentName: ${{ parameters.componentName }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} -DAMDGPU_TARGETS=gfx1030;gfx1100
os: ${{ job.os }} -GNinja
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
installLatestCMake: true
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rccl-UnitTests'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.systemsRepo }}
sparseCheckoutDir: ${{ parameters.systemsSparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,34 +1,10 @@
parameters: parameters:
- name: componentName
type: string
default: rdc
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -42,194 +18,67 @@ parameters:
- autoconf - autoconf
- libtool - libtool
- pkg-config - pkg-config
- libdrm-dev
- libnuma-dev
- libyaml-cpp-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- amdsmi
- clr
- hipBLAS-common
- hipBLASLt
- hipRAND
- llvm-project
- rocBLAS
- rocm-cmake - rocm-cmake
- rocm-core - llvm-project
- ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocm_smi_lib - rocm_smi_lib
- ROCmValidationSuite
- rocprofiler
- rocprofiler-register
- rocprofiler-sdk
- rocRAND
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- amdsmi - amdsmi
- clr
- llvm-project
- rocm-cmake
- rocminfo
- rocm_smi_lib
- ROCmValidationSuite
- rocprofiler
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rdc
- job: ${{ parameters.componentName }}_build_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
cmakeVersion: '3.25.0'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
# Build grpc
- task: Bash@3
displayName: 'git clone grpc'
inputs:
targetType: inline
script: git clone -b v1.67.1 https://github.com/grpc/grpc --depth=1 --shallow-submodules --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} dependencyList: ${{ parameters.rocmDependencies }}
cmakeBuildDir: $(Build.SourcesDirectory)/grpc/build dependencySource: tag-builds
cmakeSourceDir: $(Build.SourcesDirectory)/grpc # Build grpc
installDir: $(Build.SourcesDirectory)/bin - task: Bash@3
extraBuildFlags: >- displayName: 'git clone grpc'
-DgRPC_INSTALL=ON inputs:
-DgRPC_BUILD_TESTS=OFF targetType: inline
-DBUILD_SHARED_LIBS=ON script: git clone -b v1.61.0 https://github.com/grpc/grpc --depth=1 --shallow-submodules --recurse-submodules
-DCMAKE_INSTALL_LIBDIR=lib workingDirectory: $(Build.SourcesDirectory)
-DCMAKE_BUILD_TYPE=Release - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DAMDGPU_TARGETS=${{ job.target }} parameters:
-GNinja cmakeBuildDir: $(Build.SourcesDirectory)/grpc/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml installDir: $(Build.SourcesDirectory)/bin
parameters: extraBuildFlags: >-
componentName: ${{ parameters.componentName }} -DgRPC_INSTALL=ON
extraBuildFlags: >- -DgRPC_BUILD_TESTS=OFF
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DBUILD_SHARED_LIBS=ON
-DGRPC_ROOT="$(Build.SourcesDirectory)/bin" -DCMAKE_INSTALL_LIBDIR=lib
-DBUILD_RVS=ON -DCMAKE_BUILD_TYPE=Release
-DBUILD_PROFILER=ON -GNinja
-DBUILD_TESTS=ON - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DAMDGPU_TARGETS=${{ job.target }} parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml extraBuildFlags: >-
parameters: -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
componentName: ${{ parameters.componentName }} -DGRPC_ROOT="$(Build.SourcesDirectory)/bin"
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} -DBUILD_TESTS=ON
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
- name: ROCM_DIR
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rdcd /usr/sbin/rdcd
echo $(Agent.BuildDirectory)/rocm/lib/rdc/grpc/lib | sudo tee /etc/ld.so.conf.d/grpc.conf
sudo ldconfig -v
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Test rdc
inputs:
targetType: inline
script: >-
$(Agent.BuildDirectory)/rocm/share/rdc/rdctst_tests/rdctst
--batch_mode
--start_rdcd
--unauth_comm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/bin

View File

@@ -1,267 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- python3-pip
- python3-protobuf
- cmake
- ninja-build
- libprotobuf-dev
- libprotoc-dev
- protobuf-compiler
- liblmdb-dev
- pkg-config
- ffmpeg
- libavcodec-dev
- libavformat-dev
- libavutil-dev
- libdlpack-dev
- libsndfile1-dev
- libswscale-dev
- libturbojpeg-dev
- libjpeg-turbo-official=3.0.2-20240124
- libopencv-dev
- --allow-downgrades # for libjpeg-turbo
- name: pipModules
type: object
default:
- numpy
- opencv-python
- torch
- pillow
- name: rocmDependencies
type: object
default:
- aomp
- clr
- half
- llvm-project
- MIVisionX
- rocDecode
- rocm-cmake
- ROCR-Runtime
- rpp
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- half
- llvm-project
- MIVisionX
- rocDecode
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rpp
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocAL_build_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- task: Bash@3
displayName: 'Clone PyBind11'
inputs:
targetType: inline
script: git clone --depth 1 -b v2.11.1 https://github.com/pybind/pybind11
workingDirectory: '$(Build.SourcesDirectory)'
- task: Bash@3
displayName: 'Clone RapidJSON'
inputs:
targetType: inline
script: git clone --depth 1 https://github.com/Tencent/rapidjson.git
workingDirectory: '$(Build.SourcesDirectory)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: PyBind11
cmakeBuildDir: '$(Build.SourcesDirectory)/pybind11/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/pybind11'
customInstallPath: false
installEnabled: false
extraBuildFlags: >-
-DDOWNLOAD_CATCH=ON
-DDOWNLOAD_EIGEN=ON
-GNinja
- task: Bash@3
displayName: 'Install PyBind11'
inputs:
targetType: inline
script: sudo cmake --build . --target install
workingDirectory: '$(Build.SourcesDirectory)/pybind11/build'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: RapidJSON
cmakeBuildDir: '$(Build.SourcesDirectory)/rapidjson/build'
cmakeSourceDir: '$(Build.SourcesDirectory)/rapidjson'
customInstallPath: false
installEnabled: false
extraBuildFlags: >-
-GNinja
- task: Bash@3
displayName: 'Install RapidJSON'
inputs:
targetType: inline
script: sudo cmake --build . --target install
workingDirectory: '$(Build.SourcesDirectory)/rapidjson/build'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;/opt/libjpeg-turbo
-DCMAKE_INSTALL_PREFIX_PYTHON=$Python3_STDARCH
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# registerJPEGPackages: true
# gpuTarget: ${{ job.target }}
# extraCopyDirectories:
# - /opt/libjpeg-turbo
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocAL_test_${{ job.target }}
dependsOn: rocAL_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
- name: CMAKE_INCLUDE_PATH
value: $(Agent.BuildDirectory)/rocm/include/rocal
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
- task: Bash@3
displayName: Link libjpeg-turbo
inputs:
targetType: inline
script: |
echo /opt/libjpeg-turbo/lib64 | sudo tee /etc/ld.so.conf.d/libjpeg-turbo.conf
sudo ldconfig -v
- task: Bash@3
displayName: Build rocAL tests
inputs:
targetType: inline
script: |
mkdir rocAL-tests
cd rocAL-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocal/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocAL
testDir: rocAL-tests
- task: Bash@3
displayName: Clean up libjpeg-turbo
condition: always()
inputs:
targetType: inline
script: |
sudo rm /etc/ld.so.conf.d/libjpeg-turbo.conf
sudo ldconfig -v
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerJPEGPackages: true
environment: test
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- /opt/libjpeg-turbo
# docker image will be missing the ldconfig to libjpeg-turbo

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -20,7 +15,6 @@ parameters:
- git - git
- mpich - mpich
- ninja-build - ninja-build
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -34,126 +28,48 @@ parameters:
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- rocSPARSE - rocSPARSE
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocminfo
- rocPRIM
- rocprofiler-register
- ROCR-Runtime
- rocRAND
- rocSPARSE
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocALUTION
- job: rocALUTION_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml - name: HIP_ROCCLR_HOME
- name: HIP_ROCCLR_HOME value: $(Build.BinariesDirectory)/rocm
value: $(Build.BinariesDirectory)/rocm pool:
pool: ${{ variables.MEDIUM_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
# CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DAMDGPU_TARGETS=${{ job.target }}
-DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocALUTION_test_${{ job.target }}
dependsOn: rocALUTION_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml extraBuildFlags: >-
parameters: -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
componentName: rocALUTION -DCMAKE_BUILD_TYPE=Release
testDir: '$(Agent.BuildDirectory)/rocm/bin' -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/share/rocm/cmake/
testExecutable: './rocalution-test' -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes' -DAMDGPU_TARGETS=gfx1030;gfx1100
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml -DBUILD_CLIENTS_TESTS=ON
parameters: -DBUILD_CLIENTS_BENCHMARKS=OFF
aptPackages: ${{ parameters.aptPackages }} -DBUILD_CLIENTS_SAMPLES=OFF
environment: test -GNinja
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -1,287 +1,122 @@
parameters: parameters:
- name: componentName
type: string
default: rocBLAS
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- gfortran
- libdrm-dev
- libmsgpack-dev
- libopenblas-dev
- ninja-build - ninja-build
- python3-pip
- python3-venv - python3-venv
- git
- libmsgpack-dev
- gfortran
- libopenblas-dev
- googletest
- libgtest-dev
- wget - wget
- python3-pip
- libdrm-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
- joblib - joblib
- pyyaml
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocminfo
- rocprofiler-register
- rocm_smi_lib
- rocm-core - rocm-core
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp - aomp
- clr - aomp-extras
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- rocm_smi_lib
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocSPARSE:
name: rocSPARSE
sparseCheckoutDir: projects/rocsparse
skipUnifiedBuild: 'false'
buildDependsOn:
- rocBLAS_build
# rocSOLVER depends on both rocBLAS and rocPRIM
# for a unified build, rocBLAS will be the one to call rocSOLVER
# - rocSOLVER:
# name: rocSOLVER
# sparseCheckoutDir: projects/rocsolver
# skipUnifiedBuild: 'false'
# buildDependsOn:
# - rocBLAS_build
# unifiedBuild:
# downstreamAggregateNames: rocBLAS+rocPRIM
# buildDependsOn:
# - rocBLAS_build
# - rocPRIM_build
# temporary rocblas->hipblas downstream path while the SOLVERs are disabled
- hipBLAS:
name: hipBLAS
sparseCheckoutDir: projects/hipblas
skipUnifiedBuild: 'false'
buildDependsOn:
- rocBLAS_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocBLAS
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: - name: TENSILE_ROCM_ASSEMBLER_PATH
- group: common value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang
- template: /.azuredevops/variables-global.yml - name: CMAKE_CXX_COMPILER
- name: HIP_ROCCLR_HOME value: $(Agent.BuildDirectory)/rocm/bin/hipcc
value: $(Build.BinariesDirectory)/rocm - name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH
- name: TENSILE_ROCM_ASSEMBLER_PATH value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang pool: ${{ variables.MEDIUM_BUILD_POOL }}
- name: TENSILE_ROCM_OFFLOAD_BUNDLER_PATH workspace:
value: $(Agent.BuildDirectory)/rocm/llvm/bin/clang-offload-bundler clean: all
- name: ROCM_PATH steps:
value: $(Agent.BuildDirectory)/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: pipModules: ${{ parameters.pipModules }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters: - task: Bash@3
aptPackages: ${{ parameters.aptPackages }} displayName: 'Download AOCL'
pipModules: ${{ parameters.pipModules }} inputs:
packageManager: ${{ job.packageManager }} targetType: inline
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml script: wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-2/aocl-linux-gcc-4.2.0_1_amd64.deb
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml workingDirectory: '$(Pipeline.Workspace)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - task: Bash@3
parameters: displayName: 'Install AOCL'
checkoutRepo: ${{ parameters.checkoutRepo }} inputs:
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} targetType: inline
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aocl.yml script: sudo apt install --yes ./aocl-linux-gcc-4.2.0_1_amd64.deb
parameters: workingDirectory: '$(Pipeline.Workspace)'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
gpuTarget: ${{ job.target }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - task: Bash@3
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} displayName: 'Download AOCL'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml inputs:
targetType: inline
script: wget -nv https://download.amd.com/developer/eula/aocl/aocl-4-1/aocl-linux-aocc-4.1.0_1_amd64.deb
workingDirectory: '$(Pipeline.Workspace)'
- task: Bash@3
displayName: 'Install AOCL'
inputs:
targetType: inline
script: sudo apt install --yes ./aocl-linux-aocc-4.1.0_1_amd64.deb
workingDirectory: '$(Pipeline.Workspace)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
cmakeSourceDir: $(Agent.BuildDirectory)/sparse/projects/rocblas dependencySource: tag-builds
cmakeBuildDir: $(Agent.BuildDirectory)/sparse/projects/rocblas/build - script: echo $PATH
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor parameters:
-DCMAKE_BUILD_TYPE=Release extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang++ -DCMAKE_TOOLCHAIN_FILE=toolchain-linux.cmake
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install
-DGPU_TARGETS=${{ job.target }} -DCMAKE_BUILD_TYPE=Release
-DBUILD_CLIENTS_TESTS=ON -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-GNinja -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DAMDGPU_TARGETS=gfx1030;gfx1100
parameters: -DTensile_CODE_OBJECT_VERSION=default
componentName: ${{ parameters.componentName }} -DTensile_LOGIC=asm_full
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} -DTensile_SEPARATE_ARCHITECTURES=ON
os: ${{ job.os }} -DTensile_LAZY_LIBRARY_LOADING=ON
gpuTarget: ${{ job.target }} -DTensile_LIBRARY_FORMAT=msgpack
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml -DBUILD_CLIENTS_TESTS=ON
parameters: -DBUILD_CLIENTS_BENCHMARKS=OFF
componentName: ${{ parameters.componentName }} -DBUILD_CLIENTS_SAMPLES=OFF
os: ${{ job.os }} -DROCM_PATH=$(Agent.BuildDirectory)/rocm
gpuTarget: ${{ job.target }} -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
installAOCL: true
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- TENSILE_ROCM_ASSEMBLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang
- TENSILE_ROCM_OFFLOAD_BUNDLER_PATH:::/home/user/workspace/rocm/llvm/bin/clang-offload-bundler
- ROCM_PATH:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocblas-test'
testParameters: '--yaml rocblas_smoke.yaml --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,218 +1,82 @@
parameters: parameters:
- name: componentName
type: string
default: rocDecode
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
- name: rocPyDecodeRepo
type: string
default: rocpydecode_repo
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- python3-pip
- cmake - cmake
- ninja-build
- pkg-config
- ffmpeg - ffmpeg
- libavcodec-dev - libavcodec-dev
- libavformat-dev - libavformat-dev
- libavutil-dev - libavutil-dev
- libdrm-dev
- libstdc++-12-dev - libstdc++-12-dev
- libva-amdgpu-dev - libva-dev
- mesa-amdgpu-va-drivers - mesa-amdgpu-va-drivers
- ninja-build - libdrm-dev
- pkg-config
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocm-core
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project - llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- clr
- name: jobMatrix - rocminfo
type: object - rocm-core
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- rocPyDecode:
name: rocPyDecode
sparseCheckoutDir: ''
skipUnifiedBuild: 'false'
buildDependsOn:
- rocDecode_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocDecode
- job: ${{ parameters.componentName }}_build_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
variables: workspace:
- group: common clean: all
- template: /.azuredevops/variables-global.yml steps:
- name: ROCM_PATH # Since mesa-amdgpu-multimedia-devel is not directly available from apt, register it
value: $(Agent.BuildDirectory)/rocm - task: Bash@3
pool: displayName: 'Register ROCm packages'
vmImage: ${{ variables.BASE_BUILD_POOL }} inputs:
${{ if eq(job.os, 'almalinux8') }}: targetType: inline
container: script: |
image: rocmexternalcicd.azurecr.io/manylinux228:latest sudo mkdir --parents --mode=0755 /etc/apt/keyrings
endpoint: ContainerService3 wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
workspace: echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/amdgpu/${{ variables.KEYRING_VERSION }}/ubuntu jammy main" | sudo tee /etc/apt/sources.list.d/amdgpu.list
clean: all echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/${{ variables.KEYRING_VERSION }} jammy main" | sudo tee --append /etc/apt/sources.list.d/rocm.list
steps: echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | sudo tee /etc/apt/preferences.d/rocm-pin-600
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml sudo apt update
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
packageManager: ${{ job.packageManager }} aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
checkoutRepo: ${{ parameters.checkoutRepo }} # CI case: download latest default branch build
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# registerROCmPackages: true
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
os: ${{ job.os }} parameters:
- task: Bash@3 extraBuildFlags: >-
displayName: Build rocDecode tests -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
inputs: -DROCM_PATH=$(Agent.BuildDirectory)/rocm
targetType: inline -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
script: | -DCMAKE_BUILD_TYPE=Release
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }} -GNinja
mkdir rocDecode-tests - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
cd rocDecode-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocdecode/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocDecode
testDir: 'rocDecode-tests'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.rocPyDecodeRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,41 +1,16 @@
parameters: parameters:
- name: componentName
type: string
default: rocFFT
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build - ninja-build
- libboost-program-options-dev - libboost-program-options-dev
- libdrm-dev
- libgtest-dev - libgtest-dev
- libfftw3-dev - libfftw3-dev
- python3-pip - python3-pip
@@ -43,167 +18,57 @@ parameters:
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- clr
- hipRAND
- llvm-project - llvm-project
- rocm-cmake - ROCR-Runtime
- clr
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime
- rocRAND
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- hipRAND - hipRAND
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocRAND - rocRAND
- rocm-cmake
- name: jobMatrix - aomp
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
- name: downstreamComponentMatrix
type: object
default:
- hipFFT:
name: hipFFT
sparseCheckoutDir: projects/hipfft
skipUnifiedBuild: 'false'
buildDependsOn:
- rocFFT_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocFFT
- job: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_ubuntu2204_${{ job.target }} # todo: un-hardcode OS value: $(Build.BinariesDirectory)/rocm
variables: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- group: common workspace:
- template: /.azuredevops/variables-global.yml clean: all
- name: HIP_ROCCLR_HOME steps:
value: $(Build.BinariesDirectory)/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DAMDGPU_TARGETS=${{ job.target }}
-DUSE_HIP_CLANG=ON
-DHIP_COMPILER=clang
-DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_ubuntu2204_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_ubuntu2204_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
${{ if parameters.triggerDownstreamJobs }}: parameters:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} extraBuildFlags: >-
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
parameters: -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
componentName: ${{ parameters.componentName }} -DCMAKE_BUILD_TYPE=Release
testDir: '$(Agent.BuildDirectory)/rocm/bin' -DAMDGPU_TARGETS=gfx1030;gfx1100
testExecutable: './rocfft-test' -DUSE_HIP_CLANG=ON
testParameters: '--test_prob 0.004 --gtest_output=xml:./test_output.xml --gtest_color=yes' -DHIP_COMPILER=clang
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml -DBUILD_CLIENTS_TESTS=ON
parameters: -DBUILD_CLIENTS_BENCHMARKS=OFF
aptPackages: ${{ parameters.aptPackages }} -DBUILD_CLIENTS_SAMPLES=OFF
environment: test -GNinja
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,183 +0,0 @@
parameters:
- name: componentName
type: string
default: rocJPEG
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-dev
- libstdc++-12-dev
- libva-amdgpu-dev
- mesa-amdgpu-va-drivers
- ninja-build
- pkg-config
- python3-pip
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocminfo
- rocm-core
- rocprofiler-register
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
${{ if eq(job.os, 'ubuntu2404') }}:
name: rocm-ci_medium_build_pool_2404
${{ else }}:
name: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
consolidateBuildAndInstall: true
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm"
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- task: Bash@3
displayName: Build rocJPEG tests
inputs:
targetType: inline
script: |
${{ iif(eq(job.os, 'almalinux8'), 'source /opt/rh/gcc-toolset-14/enable', '') }}
mkdir rocJPEG-tests
cd rocJPEG-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocjpeg/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocJPEG
testDir: 'rocJPEG-tests'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -5,50 +5,21 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- libdrm-dev
- libstdc++-12-dev
- ninja-build - ninja-build
- git
- python3-pip - python3-pip
- name: pipModules
type: object
default:
- hip-python --extra-index-url https://test.pypi.org/simple
- ml_dtypes
- numpy
- tomli
- scipy
- pybind11
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project - llvm-project
- rocm-cmake - rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocMLIR_build - job: rocMLIR
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -59,16 +30,22 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
aptPackages: ${{ parameters.aptPackages }} aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRef: ${{ parameters.checkoutRef }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
dependencyList: ${{ parameters.rocmDependencies }} parameters:
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -76,80 +53,6 @@ jobs:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_FAT_LIBROCKCOMPILER=1 -DBUILD_FAT_LIBROCKCOMPILER=1
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
# compiling and running test on the test system together
- job: rocMLIR_test_${{ job.target }}
dependsOn: rocMLIR_build
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
- task: Bash@3
displayName: Symlink rocm_agent_enumerator
inputs:
targetType: inline
script: |
# Assuming that /opt is no longer persistent across runs, test environments are fully ephemeral
sudo mkdir -p /opt/rocm/bin
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
installEnabled: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/lib
-DCMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/lib
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=${{ job.target }}
-DROCM_TEST_CHIPSET=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocMLIR
testDir: $(Build.SourcesDirectory)/build
testExecutable: ninja
testParameters: check-rocmlir
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,231 +1,65 @@
parameters: parameters:
- name: componentName
type: string
default: rocPRIM
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- libgtest-dev
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocminfo
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project - llvm-project
- rocminfo
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - clr
- rocminfo
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 1, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 2, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx942, shard: 3, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 1, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 2, shardCount: 3 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a, shard: 3, shardCount: 3 }
- name: downstreamComponentMatrix
type: object
default:
- rocThrust:
name: rocThrust
sparseCheckoutDir: projects/rocthrust
skipUnifiedBuild: 'false'
buildDependsOn:
- rocPRIM_build
- hipCUB:
name: hipCUB
sparseCheckoutDir: projects/hipcub
skipUnifiedBuild: 'false'
buildDependsOn:
- rocPRIM_build
# rocSOLVER depends on both rocBLAS and rocPRIM
# for a unified build, rocBLAS will be the one to call rocSOLVER
# - rocSOLVER:
# name: rocSOLVER
# sparseCheckoutDir: projects/rocsolver
# skipUnifiedBuild: 'true'
# buildDependsOn:
# - rocPRIM_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocPRIM
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_${{ job.os }}_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
endpoint: ContainerService3 parameters:
workspace: checkoutRepo: ${{ parameters.checkoutRepo }}
clean: all # ${{ }} are resolved during compile-time
steps: # so this next step is skipped completely until
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml # we define explicit aptPackages needed to install
parameters: # CI case: download latest default branch build
aptPackages: ${{ parameters.aptPackages }} - ${{ if eq(parameters.checkoutRef, '') }}:
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
consolidateBuildAndInstall: true dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" parameters:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ extraBuildFlags: >-
-DAMDGPU_TARGETS=${{ job.target }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_BENCHMARK=ON -DBUILD_BENCHMARK=ON
-DBUILD_TEST=ON -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-GNinja -DAMDGPU_TARGETS=gfx1030;gfx1100
extraCxxFlags: -Wno-deprecated-declarations -DBUILD_TEST=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -GNinja
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}_shard_${{ job.shard }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocprim'
extraTestParameters: '-I ${{ job.shard }},,${{ job.shardCount }}'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,264 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ffmpeg
- libavcodec-dev
- libavformat-dev
- libavutil-dev
- libdlpack-dev
- libdrm-dev
- libva-dev
- ninja-build
- pkg-config
- python3-pip
- name: pipModules
type: object
default:
- hip-python --extra-index-url https://test.pypi.org/simple
- numpy
- pybind11
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocDecode
- rocJPEG
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocPyDecode_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: 'Save Python Package Paths'
inputs:
targetType: inline
script: |
echo "##vso[task.setvariable variable=PYTHON_USER_SITE;]$(python3 -m site --user-site)"
echo "##vso[task.setvariable variable=PYTHON_DIST_PACKAGES;]$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "##vso[task.setvariable variable=PYBIND11_PATH;]$(python3 -c 'import pybind11; print(pybind11.get_cmake_dir())')"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
installEnabled: false
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(PYTHON_USER_SITE)/pybind11;$(PYTHON_DIST_PACKAGES)/pybind11;$(PYBIND11_PATH)
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-DCMAKE_INSTALL_PREFIX_PYTHON=$(Build.BinariesDirectory)
-GNinja
- task: Bash@3
displayName: 'rocPyDecode install'
inputs:
targetType: inline
script: |
sudo cmake --build . --target install
sudo chown -R $(whoami):$(id -gn) $(Build.BinariesDirectory)
workingDirectory: $(Build.SourcesDirectory)/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
publish: false
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: |
export HIP_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include/hip
sudo python3 setup.py bdist_wheel
sudo chown -R $(whoami):$(id -gn) $(find . -name "*.whl")
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Build.SourcesDirectory)/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- task: Bash@3
displayName: Save pipeline artifact file names
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.ArtifactStagingDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocPyDecode_test_${{ job.target }}
dependsOn: rocPyDecode_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- task: Bash@3
displayName: Ensure pybind11-dev is not installed
inputs:
targetType: inline
script: |
if dpkg -l | grep -q pybind11-dev; then
echo "Removing pybind11-dev..."
sudo apt remove -y pybind11-dev
else
echo "pybind11-dev is not installed."
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
setupHIPLibrarySymlinks: true
- task: Bash@3
displayName: pip install
inputs:
targetType: inline
script: |
pip uninstall -y rocPyDecode
find -name *.whl -exec pip install {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Setup search paths
inputs:
targetType: inline
script: |
echo "##vso[task.setvariable variable=PYTHON_USER_SITE;]$(python3 -m site --user-site)"
echo "##vso[task.setvariable variable=PYTHON_DIST_PACKAGES;]$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')"
echo "##vso[task.setvariable variable=PYBIND11_PATH;]$(python3 -c 'import pybind11; print(pybind11.get_cmake_dir())')"
- task: CMake@1
displayName: 'rocPyDecode Test CMake Flags'
inputs:
workingDirectory: $(Agent.BuildDirectory)/rocm/share/rocpydecode/tests
cmakeArgs: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(PYTHON_USER_SITE)/pybind11;$(PYTHON_DIST_PACKAGES)/pybind11;$(PYBIND11_PATH)
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
.
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocPyDecode
testDir: $(Agent.BuildDirectory)/rocm/share/rocpydecode/tests
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
pythonEnvVars: true
# note that this docker won't have hip-python installed via pip

View File

@@ -1,223 +1,64 @@
parameters: parameters:
- name: componentName
type: string
default: rocRAND
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- python3-pip - googletest
- libgtest-dev
- git
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocm-cmake - rocm-cmake
- rocminfo
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project - llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- clr
- name: jobMatrix - rocminfo
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipRAND:
name: hipRAND
sparseCheckoutDir: projects/hiprand
skipUnifiedBuild: 'false'
buildDependsOn:
- rocRAND_build
- MIOpen:
name: MIOpen
sparseCheckoutDir: projects/miopen
skipUnifiedBuild: 'true'
buildDependsOn:
- rocRAND_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocRAND
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: pool:
- group: common vmImage: ${{ variables.BASE_BUILD_POOL }}
- template: /.azuredevops/variables-global.yml workspace:
- name: HIP_ROCCLR_HOME clean: all
value: $(Build.BinariesDirectory)/rocm steps:
pool: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
${{ if eq(job.os, 'ubuntu2404') }}: parameters:
vmImage: 'ubuntu-24.04' aptPackages: ${{ parameters.aptPackages }}
${{ else }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
name: ${{ variables.MEDIUM_BUILD_POOL }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: checkoutRepo: ${{ parameters.checkoutRepo }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest # CI case: download latest default branch build
endpoint: ContainerService3 - ${{ if eq(parameters.checkoutRef, '') }}:
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
os: ${{ job.os }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
consolidateBuildAndInstall: true dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor parameters:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ extraBuildFlags: >-
-DAMDGPU_TARGETS=${{ job.target }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_TEST=ON -DBUILD_TEST=ON
-GNinja -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DAMDGPU_TARGETS=gfx1030;gfx1100
parameters: -GNinja
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# gpuTarget: ${{ job.target }}
# extraEnvVars:
# - HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocRAND'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,268 +1,90 @@
parameters: parameters:
- name: componentName
type: string
default: rocSOLVER
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- gfortran
- git
- libfmt-dev
- libsuitesparse-dev
- ninja-build - ninja-build
- libsuitesparse-dev
- gfortran
- libfmt-dev
- git
- googletest
- libgtest-dev
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocBLAS
- rocm-cmake - rocm-cmake
- rocminfo
- rocPRIM
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- hipBLAS-common
- hipBLASLt
- hipSPARSE
- llvm-project - llvm-project
- rocBLAS
- rocm-cmake
- rocminfo
- rocPRIM
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- clr
- rocminfo
- rocBLAS
- rocPRIM
- rocSPARSE - rocSPARSE
- roctracer - hipSPARSE
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
#- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipBLAS:
name: hipBLAS
sparseCheckoutDir: projects/hipblas
skipUnifiedBuild: 'false'
buildDependsOn:
- rocSOLVER_build
# hipSOLVER depends on both rocSOLVER and rocSPARSE
# for a unified build, rocSOLVER will be the one to call hipSOLVER
# - hipSOLVER:
# name: hipSOLVER
# sparseCheckoutDir: projects/hipsolver
# skipUnifiedBuild: 'false'
# buildDependsOn:
# - rocSOLVER_build
# unifiedBuild:
# downstreamAggregateNames: rocSOLVER+rocSPARSE
# buildDependsOn:
# - rocSOLVER_build
# - rocSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocSOLVER
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_${{ job.os }}_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
endpoint: ContainerService3 parameters:
workspace: checkoutRepo: ${{ parameters.checkoutRepo }}
clean: all - task: Bash@3
steps: displayName: 'Clone lapack'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml inputs:
parameters: targetType: inline
aptPackages: ${{ parameters.aptPackages }} script: git clone --depth 1 --branch v3.9.1 https://github.com/Reference-LAPACK/lapack
packageManager: ${{ job.packageManager }} workingDirectory: '$(Build.SourcesDirectory)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- task: Bash@3
displayName: 'Clone lapack'
inputs:
targetType: inline
script: git clone --depth 1 --branch v3.9.1 https://github.com/Reference-LAPACK/lapack
workingDirectory: '$(Build.SourcesDirectory)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: staging
gpuTarget: ${{ job.target }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: lapack dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_BUILD_TYPE=Release parameters:
-DCMAKE_POSITION_INDEPENDENT_CODE=ON componentName: lapack
-DCMAKE_Fortran_FLAGS=-fno-optimize-sibling-calls extraBuildFlags: >-
-DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
-DCBLAS=ON -DCMAKE_Fortran_FLAGS=-fno-optimize-sibling-calls
-DLAPACKE=OFF -DBUILD_TESTING=OFF
-GNinja -DCBLAS=ON
cmakeBuildDir: '$(Build.SourcesDirectory)/lapack/build' -DLAPACKE=OFF
cmakeSourceDir: '$(Build.SourcesDirectory)/lapack' -GNinja
installDir: '$(Pipeline.Workspace)/deps-install' cmakeBuildDir: '$(Build.SourcesDirectory)/lapack/build'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml installDir: '$(Pipeline.Workspace)/deps-install'
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
os: ${{ job.os }} parameters:
extraBuildFlags: >- extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install;$(Agent.BuildDirectory)/vendor -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Pipeline.Workspace)/deps-install
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DAMDGPU_TARGETS=${{ job.target }} -DAMDGPU_TARGETS=gfx1030;gfx1100
-DBUILD_CLIENTS_TESTS=ON -DBUILD_CLIENTS_TESTS=ON
-DBUILD_CLIENTS_BENCHMARKS=OFF -DBUILD_CLIENTS_BENCHMARKS=OFF
-DBUILD_CLIENTS_SAMPLES=OFF -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraCopyDirectories:
- deps-install
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocsolver-test'
testParameters: '--gtest_filter="*checkin*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}
${{ if parameters.unifiedBuild }}:
buildDependsOn: ${{ component.unifiedBuild.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ component.unifiedBuild.downstreamAggregateNames }}
${{ else }}:
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}

View File

@@ -1,243 +1,87 @@
parameters: parameters:
- name: componentName
type: string
default: rocSPARSE
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- gfortran
- git
- libboost-program-options-dev
- libdrm-dev
- libfftw3-dev
- ninja-build
- python3-pip - python3-pip
- cmake
- ninja-build
- libboost-program-options-dev
- googletest
- libfftw3-dev
- git
- gfortran
- libgtest-dev
- libdrm-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr
- llvm-project
- rocBLAS
- rocm-cmake - rocm-cmake
- llvm-project
- ROCR-Runtime
- clr
- rocBLAS
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
#- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- name: downstreamComponentMatrix
type: object
default:
- hipSPARSE:
name: hipSPARSE
sparseCheckoutDir: projects/hipsparse
skipUnifiedBuild: 'false'
buildDependsOn:
- rocSPARSE_build
# hipSOLVER depends on both rocSOLVER and rocSPARSE
# for a unified build, rocSOLVER will be the one to call hipSOLVER
# - hipSOLVER:
# name: hipSOLVER
# sparseCheckoutDir: projects/hipsolver
# skipUnifiedBuild: 'true'
# buildDependsOn:
# - rocSPARSE_build
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocSPARSE
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- group: common workspace:
- template: /.azuredevops/variables-global.yml clean: all
- name: HIP_ROCCLR_HOME steps:
value: $(Build.BinariesDirectory)/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
aptPackages: ${{ parameters.aptPackages }} # CI case: download latest default branch build
packageManager: ${{ job.packageManager }} - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
extraBuildFlags: >- dependencySource: tag-builds
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang++ parameters:
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/amdclang extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/bin/hipcc
-DAMDGPU_TARGETS=${{ job.target }} -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_CLIENTS_SAMPLES=OFF -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_CLIENTS_TESTS=ON -DCMAKE_BUILD_TYPE=Release
-DBUILD_CLIENTS_BENCHMARKS=OFF -DAMDGPU_TARGETS=gfx1030;gfx1100
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/hip/cmake -DBUILD_CLIENTS_SAMPLES=OFF
-GNinja -DBUILD_CLIENTS_TESTS=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DBUILD_CLIENTS_BENCHMARKS=OFF
parameters: -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/hip/cmake
componentName: ${{ parameters.componentName }} -GNinja
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
gpuTarget: ${{ job.target }} parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml artifactName: rocSPARSE
parameters: publish: false
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
gpuTarget: ${{ job.target }} parameters:
os: ${{ job.os }} sourceDir: $(Build.SourcesDirectory)/build/clients
publish: false contentsString: matrices/**
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters: parameters:
sourceDir: $(Agent.BuildDirectory)/s/build/clients artifactName: testMatrices
contentsString: matrices/**
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
artifactName: testMatrices
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: '$(Agent.BuildDirectory)/rocm/bin'
testExecutable: './rocsparse-test'
testParameters: '--gtest_filter="*quick*" --gtest_output=xml:./test_output.xml --gtest_color=yes'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}
- ${{ if parameters.triggerDownstreamJobs }}:
- ${{ each component in parameters.downstreamComponentMatrix }}:
- ${{ if not(and(parameters.unifiedBuild, eq(component.skipUnifiedBuild, 'true'))) }}:
- template: /.azuredevops/components/${{ component.name }}.yml@pipelines_repo
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ component.sparseCheckoutDir }}
buildDependsOn: ${{ component.buildDependsOn }}
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}+${{ parameters.componentName }}
triggerDownstreamJobs: true
unifiedBuild: ${{ parameters.unifiedBuild }}

View File

@@ -1,195 +1,66 @@
parameters: parameters:
- name: componentName
type: string
default: rocThrust
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- git
- ninja-build - ninja-build
- libboost-program-options-dev - libboost-program-options-dev
- googletest
- libfftw3-dev - libfftw3-dev
- git
- python3-pip - python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- clr - clr
- llvm-project - hipRAND
- rocm-cmake
- rocminfo
- rocPRIM
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project - llvm-project
- rocminfo - rocminfo
- rocPRIM - rocPRIM
- rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocThrust
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool: ${{ variables.MEDIUM_BUILD_POOL }}
- ${{ build }}_${{ job.os }}_${{ job.target }} workspace:
variables: clean: all
- group: common steps:
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
endpoint: ContainerService3 parameters:
workspace: checkoutRepo: ${{ parameters.checkoutRepo }}
clean: all # CI case: download latest default branch build
steps: - ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
consolidateBuildAndInstall: true dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-GNinja parameters:
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -GNinja
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DAMDGPU_TARGETS=${{ job.target }} -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_TEST=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -DAMDGPU_TARGETS=gfx1030;gfx1100
parameters: -DBUILD_TEST=ON
componentName: ${{ parameters.componentName }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocthrust'
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml --exclude-regex "scan.hip"'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -17,15 +12,14 @@ parameters:
- cmake - cmake
- ninja-build - ninja-build
- libboost-program-options-dev - libboost-program-options-dev
- libdrm-dev
- libgtest-dev - libgtest-dev
- googletest - googletest
- libfftw3-dev - libfftw3-dev
- git - git
- libomp-dev
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp
- rocm-cmake - rocm-cmake
- llvm-project - llvm-project
- ROCR-Runtime - ROCR-Runtime
@@ -34,121 +28,44 @@ parameters:
- rocBLAS - rocBLAS
- aomp - aomp
- rocm_smi_lib - rocm_smi_lib
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocm-cmake
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocWMMA
- job: rocWMMA_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool: ${{ variables.MEDIUM_BUILD_POOL }}
pool: ${{ variables.MEDIUM_BUILD_POOL }} workspace:
workspace: clean: all
clean: all steps:
steps: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml parameters:
parameters: aptPackages: ${{ parameters.aptPackages }}
aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml parameters:
parameters: checkoutRepo: ${{ parameters.checkoutRepo }}
checkoutRepo: ${{ parameters.checkoutRepo }} # CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_BUILD_TYPE=Release
-DROCWMMA_BUILD_TESTS=ON
-DROCWMMA_BUILD_SAMPLES=OFF
-DGPU_TARGETS=${{ job.target }}
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON
-DROCM_PLATFORM_VERSION=$(NEXT_RELEASE_VERSION)
-GNinja
# gfx1030 not supported in documentation
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocWMMA_test_${{ job.target }}
timeoutInMinutes: 270
dependsOn: rocWMMA_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml extraBuildFlags: >-
parameters: -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
componentName: rocWMMA -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
testDir: '$(Agent.BuildDirectory)/rocm/bin/rocwmma' -DCMAKE_BUILD_TYPE=Release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml -DROCWMMA_BUILD_TESTS=ON
parameters: -DROCWMMA_BUILD_SAMPLES=OFF
aptPackages: ${{ parameters.aptPackages }} -DGPU_TARGETS=gfx1100
environment: test -GNinja
gpuTarget: ${{ job.target }} # gfx1030 not supported in documentation
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -5,93 +5,44 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- doxygen - doxygen
- doxygen-doc - doxygen-doc
- ninja-build - ninja-build
- python3-sphinx
- name: pipModules - name: pipModules
type: object type: object
default: default:
- cget - cget
- cmake==3.20.5
- ninja - ninja
- rocm-docs-core
- sphinx
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_cmake
- job: rocm_cmake_${{ job.os }} variables:
pool: - group: common
${{ if eq(job.os, 'ubuntu2404') }}: - template: /.azuredevops/variables-global.yml
vmImage: 'ubuntu-24.04' pool:
${{ else }}: vmImage: ${{ variables.BASE_BUILD_POOL }}
vmImage: 'ubuntu-22.04' workspace:
${{ if eq(job.os, 'almalinux8') }}: clean: all
container: steps:
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
endpoint: ContainerService3 parameters:
variables: aptPackages: ${{ parameters.aptPackages }}
- group: common pipModules: ${{ parameters.pipModules }}
- template: /.azuredevops/variables-global.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
clean: all parameters:
steps: checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: # extra steps for ctest suite
aptPackages: ${{ parameters.aptPackages }} - script: |
pipModules: ${{ parameters.pipModules }} python -m pip install -r $(Build.SourcesDirectory)/docs/requirements.txt
packageManager: ${{ job.packageManager }} python -m pip install -r $(Build.SourcesDirectory)/test/docsphinx/docs/.sphinx/requirements.txt
- task: Bash@3 git config --global user.email "you@example.com"
displayName: Add CMake to PATH git config --global user.name "Your Name"
inputs: displayName: "ctest setup"
targetType: inline # - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
script: echo "##vso[task.prependpath]$(python3 -m site --user-base)/bin" - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
useAmdclang: false
- task: Bash@3
displayName: CTest setup
inputs:
targetType: inline
script: |
python -m pip install -r $(Build.SourcesDirectory)/docs/requirements.txt
python -m pip install -r $(Build.SourcesDirectory)/test/docsphinx/docs/.sphinx/requirements.txt
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm-cmake
testParameters: '-E "pass-version-parent" --extra-verbose --output-on-failure --force-new-ctest-process --output-junit test_output.xml'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# environment: combined

View File

@@ -1,103 +1,33 @@
parameters: parameters:
- name: componentName
type: string
default: rocm-core
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_core
- job: rocm_core_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
${{ if eq(job.os, 'ubuntu2404') }}: clean: all
vmImage: 'ubuntu-24.04' steps:
${{ else }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
vmImage: 'ubuntu-22.04' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: checkoutRepo: ${{ parameters.checkoutRepo }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
endpoint: ContainerService3 parameters:
variables: extraBuildFlags: >-
- group: common -DCMAKE_CURRENT_BINARY_DIR=$PWD
- template: /.azuredevops/variables-global.yml -DCMAKE_CURRENT_SOURCE_DIR=$PWD/../
workspace: -DCMAKE_VERBOSE_MAKEFILE=1
clean: all -DCPACK_GENERATOR=DEB
steps: -DCPACK_DEBIAN_PACKAGE_RELEASE="local.9999~99.99"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml -DCPACK_RPM_PACKAGE_RELEASE="local.9999"
parameters: -DROCM_VERSION="$(last-release)"
aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_CURRENT_BINARY_DIR=$PWD
-DCMAKE_CURRENT_SOURCE_DIR=$PWD/../
-DCMAKE_VERBOSE_MAKEFILE=1
-DCPACK_GENERATOR=DEB
-DCPACK_DEBIAN_PACKAGE_RELEASE="local.9999~99.99"
-DCPACK_RPM_PACKAGE_RELEASE="local.9999"
-DROCM_VERSION="$(NEXT_RELEASE_VERSION)"
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}

View File

@@ -5,227 +5,67 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- libdw-dev
- libglfw3-dev - libglfw3-dev
- libmsgpack-dev
- libopencv-dev
- libtbb-dev
- libtiff-dev
- libva-amdgpu-dev
- libavcodec-dev
- libavformat-dev
- libavutil-dev
- ninja-build
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- AMDMIGraphX - AMDMIGraphX
- aomp
- aomp-extras
- clr - clr
- composable_kernel
- hipBLAS - hipBLAS
- hipBLAS-common
- hipBLASLt
- hipCUB - hipCUB
- hipFFT
- HIPIFY - HIPIFY
- hipRAND - hipRAND
- hipSOLVER - hipSOLVER
- hipSPARSE - hipSPARSE
- hipTensor
- llvm-project - llvm-project
- MIOpen
- MIVisionX
- rocALUTION
- rocBLAS - rocBLAS
- rocDecode
- rocFFT
- rocJPEG
- rocPRIM - rocPRIM
- rocprofiler-register - rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime - ROCR-Runtime
- rocRAND - rocRAND
- rocSOLVER - rocSOLVER
- rocSPARSE - rocSPARSE
- rocThrust - rocThrust
- rocWMMA
- rpp
- name: rocmTestDependencies
type: object
default:
- AMDMIGraphX
- aomp
- aomp-extras
- clr
- composable_kernel
- hipBLAS
- hipBLAS-common
- hipBLASLt
- hipCUB
- hipFFT
- HIPIFY
- hipRAND
- hipSOLVER
- hipSPARSE
- hipTensor
- llvm-project
- MIOpen
- MIVisionX
- rocALUTION
- rocBLAS
- rocDecode
- rocFFT
- rocminfo
- rocPRIM
- rocJPEG
- rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime
- rocRAND
- rocSOLVER
- rocSPARSE
- rocThrust
- roctracer
- rocWMMA
- rpp
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_examples
- job: rocm_examples_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: ${{ variables.MEDIUM_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
steps: steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
registerROCmPackages: true checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters: parameters:
cmakeVersion: '3.25.0' aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml # CI case: download latest default branch build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - ${{ if eq(parameters.checkoutRef, '') }}:
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
# https://github.com/ROCm/HIP/issues/2203
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_ROOT=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=${{ job.target }}
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }}
-DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc
-GNinja
- task: Bash@3
displayName: Move rocm-examples binaries to rocm/examples
inputs:
targetType: inline
script: |
mkdir -p $(Build.BinariesDirectory)/examples
mv $(Build.BinariesDirectory)/bin/* $(Build.BinariesDirectory)/examples
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocm_examples_test_${{ job.target }}
dependsOn: rocm_examples_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: TEST_LOG_FILE
value: $(Pipeline.Workspace)/rocm-examplesTestLog.log
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
parameters:
cmakeVersion: '3.25.0'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml parameters:
parameters: # https://github.com/ROCm/HIP/issues/2203
# https://github.com/ROCm/HIP/issues/2203 extraBuildFlags: >-
extraBuildFlags: >- -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DROCM_ROOT=$(Agent.BuildDirectory)/rocm
-DROCM_ROOT=$(Agent.BuildDirectory)/rocm -DCMAKE_HIP_ARCHITECTURES=gfx1030;gfx1100
-DAMDGPU_TARGETS=${{ job.target }} -DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
-DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm-examples
testDir: $(Build.SourcesDirectory)/build
testParameters: '--output-on-failure --force-new-ctest-process --output-junit test_output.xml'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,193 +0,0 @@
parameters:
- name: componentName
type: string
default: rocm_libraries
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- ccache
- gfortran
- git
- libdrm-dev
- liblapack-dev
- libmsgpack-dev
- libnuma-dev
- libopenblas-dev
- ninja-build
- python3-pip
- python3-venv
- googletest
- libgtest-dev
- libgmock-dev
- libboost-filesystem-dev
- name: pipModules
type: object
default:
- msgpack
- joblib
- "packaging>=22.0"
- pytest
- pytest-cmake
- --upgrade
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocminfo
- rocm-cmake
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { pool: rocm-ci_ultra_build_pool, os: ubuntu2204, packageManager: apt, target: gfx942 }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 300
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: DAY_STRING
value: $[format('{0:ddMMyyyy}', pipeline.startTime)]
pool: ${{ job.pool }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-custom.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- gtest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- script: |
mkdir -p $(CCACHE_DIR)
echo "##vso[task.prependpath]/usr/lib/ccache"
displayName: Update path for ccache
- task: Cache@2
displayName: Ccache caching
inputs:
key: rocm-libraries | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING) | $(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
path: $(CCACHE_DIR)
restoreKeys: |
rocm-libraries | ${{ job.os }} | ${{ job.target }} | $(DAY_STRING)
rocm-libraries | ${{ job.os }} | ${{ job.target }}
rocm-libraries | ${{ job.os }}
- task: Bash@3
displayName: Add paths for CMake and Python site-packages binaries
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.setvariable variable=PytestCmakePath]$USER_BASE/share/Pytest/cmake"
displayName: Set cmake configure paths
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
extraBuildFlags: >-
-D CMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor;$(PytestCmakePath)
-D CMAKE_INCLUDE_PATH=$(Agent.BuildDirectory)/rocm/llvm/include
-D CMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-D CMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-D CMAKE_CXX_COMPILER_LAUNCHER=ccache
-D CMAKE_C_COMPILER_LAUNCHER=ccache
-G Ninja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/llvm/bin:/home/user/workspace/rocm/bin
installLatestCMake: true
extraCopyDirectories:
- deps

View File

@@ -5,12 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -30,24 +24,10 @@ parameters:
- rocminfo - rocminfo
- rocprofiler-register - rocprofiler-register
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies - ROCT-Thunk-Interface
type: object
default:
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocm_bandwidth_test_build - job: rocm_bandwidth_test
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -68,11 +48,18 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRef: ${{ parameters.checkoutRef }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
dependencyList: ${{ parameters.rocmDependencies }} parameters:
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -80,55 +67,4 @@ jobs:
-DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules -DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/include;$(Agent.BuildDirectory)/rocm/include/hsa -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/rocm/include;$(Agent.BuildDirectory)/rocm/include/hsa
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# extraEnvVars:
# - ROCR_INC_DIR:::/home/user/workspace/rocm
# - ROCR_LIB_DIR:::/home/user/workspace/rocm
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocm_bandwidth_test_test_${{ job.target }}
dependsOn: rocm_bandwidth_test_build
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm_bandwidth_test
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/rocm-bandwidth-test'
testParameters: ''
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,145 +1,28 @@
parameters: parameters:
- name: componentName
type: string
default: rocm_smi_lib
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-dev
- ninja-build
- pkg-config
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocm_smi_lib
- job: rocm_smi_lib_build_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
${{ if eq(job.os, 'ubuntu2404') }}: clean: all
vmImage: 'ubuntu-24.04' steps:
${{ else }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
vmImage: 'ubuntu-22.04' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: checkoutRepo: ${{ parameters.checkoutRepo }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
endpoint: ContainerService3 parameters:
variables: extraBuildFlags: >-
- group: common -DBUILD_TESTS=ON
- template: /.azuredevops/variables-global.yml -DROCM_DEP_ROCMCORE=ON
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DBUILD_TESTS=ON
-DROCM_DEP_ROCMCORE=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocm_smi_lib_test_${{ job.os }}_${{ job.target }}
dependsOn: rocm_smi_lib_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)'
testExecutable: 'sudo ./rocm/share/rocm_smi/rsmitst_tests/rsmitst'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,175 +1,45 @@
parameters: parameters:
- name: componentName
type: string
default: rocminfo
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdrm-amdgpu-dev
- libdrm-dev
- ninja-build
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- ROCR-Runtime
- rocprofiler-register
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocminfo
- job: ${{ parameters.componentName }}_build_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
vmImage: 'ubuntu-22.04' clean: all
${{ if eq(job.os, 'almalinux8') }}: steps:
container: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
endpoint: ContainerService3 parameters:
variables: checkoutRepo: ${{ parameters.checkoutRepo }}
- group: common # CI case: download latest default branch build
- template: /.azuredevops/variables-global.yml - ${{ if eq(parameters.checkoutRef, '') }}:
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
${{ if parameters.triggerDownstreamJobs }}: - ${{ if ne(parameters.checkoutRef, '') }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
componentName: ${{ parameters.componentName }} dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }} dependencySource: tag-builds
useAmdclang: false - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
extraBuildFlags: >- parameters:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-DROCRTST_BLD_TYPE=release -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -DROCRTST_BLD_TYPE=release
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocminfo_test_${{ job.target }}
dependsOn: rocminfo_build_${{ job.os }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/rocminfo'
testParameters: ''
testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm_agent_enumerator
testDir: '$(Agent.BuildDirectory)'
testExecutable: './rocm/bin/rocm_agent_enumerator'
testParameters: ''
testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
registerROCmPackages: true
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,221 +0,0 @@
parameters:
- name: componentName
type: string
default: rocprofiler-compute
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- libdw-dev
- libtbb-dev
- locales
- ninja-build
- python3-pip
- name: pipModules
type: object
default:
- astunparse==1.6.2
- colorlover
- dash-bootstrap-components
- dash-svg
- "dash>=3.0.0"
- kaleido==0.2.1
- matplotlib
- "numpy>=1.17.5"
- "pandas>=1.4.3"
- plotext
- plotille
- pymongo
- pyyaml
- setuptools
- sqlalchemy
- tabulate
- textual
- textual_plotext
- textual-fspicker
- tqdm
- mock
- pytest
- pytest-cov
- pytest-xdist
- name: rocmTestDependencies
type: object
default:
- amdsmi
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- rocm_smi_lib
- ROCR-Runtime
- rocprofiler
- rocprofiler-sdk
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_compute_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_compute_test_${{ job.target }}
timeoutInMinutes: 120
dependsOn: rocprofiler_compute_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTHON_VERSION
value: 3.10
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add en_US.UTF-8 locale
inputs:
targetType: inline
script: |
sudo locale-gen en_US.UTF-8
sudo update-locale
locale -a
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_HIP_ARCHITECTURES=${{ job.target }}
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DENABLE_TESTS=ON
-DINSTALL_TESTS=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Build.BinariesDirectory)/libexec/rocprofiler-compute
testExecutable: ROCM_PATH=$(Agent.BuildDirectory)/rocm ctest
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,104 +1,37 @@
parameters: parameters:
- name: componentName
type: string
default: rocprofiler-register
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- cmake
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: ubuntu2404, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocprofiler_register
- job: rocprofiler_register_${{ job.os }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: pool:
- ${{ build }}_${{ job.os }} vmImage: ${{ variables.BASE_BUILD_POOL }}
pool: workspace:
${{ if eq(job.os, 'ubuntu2404') }}: clean: all
vmImage: 'ubuntu-24.04' steps:
${{ else }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
vmImage: 'ubuntu-22.04' - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
${{ if eq(job.os, 'almalinux8') }}: parameters:
container: checkoutRepo: ${{ parameters.checkoutRepo }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
endpoint: ContainerService3 parameters:
variables: componentName: rocprofiler-register
- group: common - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: /.azuredevops/variables-global.yml parameters:
workspace: componentName: rocprofiler-register-tests
clean: all extraBuildFlags: >-
steps: -DCMAKE_PREFIX_PATH=$(Build.BinariesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml cmakeBuildDir: 'tests/build'
parameters: installEnabled: false
aptPackages: ${{ parameters.aptPackages }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
packageManager: ${{ job.packageManager }} parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml componentName: rocprofiler-register
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml testDir: 'tests/build'
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Build.BinariesDirectory)
-DROCPROFILER_REGISTER_BUILD_TESTS=ON
-DROCPROFILER_REGISTER_BUILD_SAMPLES=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/s/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# environment: combined

View File

@@ -1,231 +0,0 @@
parameters:
- name: componentName
type: string
default: rocprofiler-sdk
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- build-essential
- cmake
- libdrm-amdgpu-dev
- libdrm-dev
- libdw-dev
- libelf-dev
- libsqlite3-dev
- libva-dev
- ninja-build
- pkg-config
- python3-pip
- name: pipModules
type: object
default:
- black
- clang-format
- clang-tidy
- cmake
- cmake-format
- dataclasses
- numpy
- otf2
- pandas
- perfetto
- pycobertura
- "pytest>=6.2.5"
- pyyaml
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rccl
- rocDecode
- rocJPEG
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_sdk_build_${{ job.os }}_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os}}_${{ job.target }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFILER_BUILD_TESTS=ON
-DROCPROFILER_BUILD_SAMPLES=ON
-DROCPROFILER_BUILD_RELEASE=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_sdk_test_${{ job.os }}_${{ job.target }}
dependsOn: rocprofiler_sdk_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add Python and ROCm binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFILER_BUILD_TESTS=ON
-DROCPROFILER_BUILD_SAMPLES=ON
-DROCPROFILER_BUILD_RELEASE=ON
-DGPU_TARGETS=${{ job.target }}
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH}}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
testDir: $(Agent.BuildDirectory)/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -1,271 +0,0 @@
# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
# monorepo related parameters
- name: componentName
type: string
default: rocprofiler-systems
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages
type: object
default:
- autoconf
- autotools-dev
- bison
- build-essential
- bzip2
- clang
- cmake
- chrpath
- environment-modules
- ffmpeg
- g++-12
- libavcodec-dev
- libavformat-dev
- libavutil-dev
- libdrm-amdgpu-dev
- libdrm-dev
- libdw-dev
- libfabric-dev
- libiberty-dev
- libpapi-dev
- libpfm4-dev
- libtool
- libopenmpi-dev
- libsqlite3-dev
- m4
- ninja-build
- openmpi-bin
- pkg-config
- python3-pip
- software-properties-common
- texinfo
- zlib1g-dev
- name: pipModules
type: object
default:
- numpy
- perfetto
- dataclasses
- name: rocmDependencies
type: object
default:
- amdsmi
- aomp
- clr
- llvm-project
- rccl
- rocDecode
- rocJPEG
- rocm-core
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- rocprofiler-sdk
- ROCR-Runtime
- roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: rocprofiler_systems_build_${{ job.target }}
${{ if parameters.buildDependsOn }}:
dependsOn:
- ${{ each build in parameters.buildDependsOn }}:
- ${{ build }}_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
aggregatePipeline: ${{ parameters.aggregatePipeline }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DROCPROFSYS_BUILD_TESTING=ON
-DROCPROFSYS_BUILD_DYNINST=ON
-DROCPROFSYS_BUILD_LIBUNWIND=ON
-DROCPROFSYS_DISABLE_EXAMPLES="openmp-target"
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DROCPROFSYS_USE_PAPI=ON
-DROCPROFSYS_USE_MPI=ON
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/include/rocjpeg
-DGPU_TARGETS=${{ job.target }}
-GNinja
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
componentName: ${{ parameters.componentName }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
registerROCmPackages: true
extraPaths: /home/user/workspace/rocm/bin:/home/user/workspace/rocm/llvm/bin
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocprofiler_systems_test_${{ job.target }}
dependsOn: rocprofiler_systems_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
timeoutInMinutes: 180
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool:
name: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: |
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
cmakeSourceDir: $(Agent.BuildDirectory)/s/projects/rocprofiler-systems
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocprofiler-systems
-DROCPROFSYS_USE_PYTHON=ON
-DROCPROFSYS_BUILD_TESTING=ON
-DROCPROFSYS_BUILD_DYNINST=ON
-DROCPROFSYS_BUILD_LIBUNWIND=ON
-DROCPROFSYS_DISABLE_EXAMPLES="openmp-target"
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DROCPROFSYS_USE_PAPI=ON
-DROCPROFSYS_USE_MPI=ON
-DCMAKE_CXX_FLAGS=-I$(Agent.BuildDirectory)/rocm/include/rocjpeg
-DGPU_TARGETS=${{ job.target }}
-GNinja
- task: Bash@3
displayName: Set up rocprofiler-systems env
inputs:
targetType: inline
script: source $(Agent.BuildDirectory)/rocprofiler-systems/share/rocprofiler-systems/setup-env.sh
workingDirectory: $(Agent.BuildDirectory)/rocprofiler-systems/share/rocprofiler-systems
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testDir: $(Agent.BuildDirectory)/s/build/tests/
testParameters: '--output-on-failure'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
registerROCmPackages: true
gpuTarget: ${{ job.target }}
extraPaths: /home/user/workspace/rocm/bin:/home/user/workspace/rocm/llvm/bin

View File

@@ -1,39 +1,15 @@
parameters: parameters:
- name: componentName
type: string
default: rocprofiler
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- libgtest-dev
- libdrm-dev - libdrm-dev
- libdw-dev - libdw-dev
- libsystemd-dev - libsystemd-dev
@@ -44,13 +20,13 @@ parameters:
- name: pipModules - name: pipModules
type: object type: object
default: default:
- barectf
- Cppheaderparser
- lxml
- matplotlib
- pandas
- pyyaml==5.3.1 - pyyaml==5.3.1
- Cppheaderparser
- websockets - websockets
- matplotlib
- lxml
- barectf
- pandas
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -59,175 +35,75 @@ parameters:
- ROCdbgapi - ROCdbgapi
- rocm-cmake - rocm-cmake
- rocm-core - rocm-core
- rocminfo
- rocm_smi_lib - rocm_smi_lib
- rocprofiler-register - rocminfo
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register
- ROCT-Thunk-Interface
- roctracer - roctracer
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rocprofiler
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Agent.BuildDirectory)/rocm
variables: - name: ROCM_PATH
- group: common value: $(Agent.BuildDirectory)/rocm
- template: /.azuredevops/variables-global.yml pool: ${{ variables.MEDIUM_BUILD_POOL }}
- name: HIP_ROCCLR_HOME workspace:
value: $(Agent.BuildDirectory)/rocm clean: all
- name: ROCM_PATH steps:
value: $(Agent.BuildDirectory)/rocm - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }} parameters:
${{ if eq(job.os, 'almalinux8') }}: aptPackages: ${{ parameters.aptPackages }}
container: pipModules: ${{ parameters.pipModules }}
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
endpoint: ContainerService3 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
workspace: parameters:
clean: all checkoutRepo: ${{ parameters.checkoutRepo }}
steps: # Manually download aqlprofile, hard-coded 6.1.0 version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - task: Bash@3
parameters: displayName: 'Download aqlprofile'
aptPackages: ${{ parameters.aptPackages }} inputs:
pipModules: ${{ parameters.pipModules }} targetType: inline
packageManager: ${{ job.packageManager }} script: wget -nv https://repo.radeon.com/rocm/misc/aqlprofile/ubuntu-22.04/hsa-amd-aqlprofile_1.0.0.60200.60200-crdnnh.14213~22.04_amd64.deb
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml workingDirectory: '$(Pipeline.Workspace)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - task: Bash@3
parameters: displayName: 'Extract aqlprofile'
checkoutRepo: ${{ parameters.checkoutRepo }} inputs:
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }} targetType: inline
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml script: |
parameters: mkdir hsa-amd-aqlprofile
dependencyList: dpkg-deb -R hsa-amd-aqlprofile_1.0.0.60200.60200-crdnnh.14213~22.04_amd64.deb hsa-amd-aqlprofile
- gtest workingDirectory: '$(Pipeline.Workspace)'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml - task: Bash@3
parameters: displayName: 'Move aqlprofile'
os: ${{ job.os }} inputs:
targetType: inline
script: |
mkdir -p $(Agent.BuildDirectory)/rocm
cp -R hsa-amd-aqlprofile/opt/rocm-6.2.0-14213/* $(Agent.BuildDirectory)/rocm
workingDirectory: '$(Pipeline.Workspace)'
# CI case: download latest default branch build
- ${{ if eq(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
os: ${{ job.os }} # manual build case: triggered by ROCm/ROCm repo
aggregatePipeline: ${{ parameters.aggregatePipeline }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
useAmdclang: false dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules;$(Agent.BuildDirectory)/rocm/lib/cmake;$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/lib64/cmake;$(Agent.BuildDirectory)/rocm/lib64/cmake/hip parameters:
-DCMAKE_PREFIX_PATH="$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/vendor" extraBuildFlags: >-
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_MODULE_PATH=$(Build.SourcesDirectory)/cmake_modules;$(Agent.BuildDirectory)/rocm/lib/cmake;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DENABLE_LDCONFIG=OFF -DENABLE_LDCONFIG=OFF
-DUSE_PROF_API=1 -DUSE_PROF_API=1
-DGPU_TARGETS=${{ job.target }} -DGPU_TARGETS=gfx1030;gfx1100
multithreadFlag: -- -j32 - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.os, 'ubuntu2204') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
gpuTarget: ${{ job.target }}
extraEnvVars:
- HIP_ROCCLR_HOME:::/home/user/workspace/rocm
- ROCM_PATH:::/home/user/workspace/rocm
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
- name: LD_LIBRARY_PATH
value: $(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1/test:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocprofilerV1
testDir: $(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1
testExecutable: ./run.sh
testParameters: ''
testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocprofilerV2
testDir: $(Agent.BuildDirectory)/rocm
testExecutable: share/rocprofiler/tests/runUnitTests
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -5,11 +5,6 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
@@ -17,9 +12,6 @@ parameters:
- ninja-build - ninja-build
- libelf-dev - libelf-dev
- libdw-dev - libdw-dev
- libstdc++-12-dev
- python-is-python3
- python3-pip
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
@@ -30,27 +22,9 @@ parameters:
- rocminfo - rocminfo
- ROCR-Runtime - ROCR-Runtime
- rocprofiler-register - rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- ROCdbgapi
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- job: rocr_debug_agent_build - job: rocr_debug_agent
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
@@ -66,11 +40,18 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters: parameters:
checkoutRepo: ${{ parameters.checkoutRepo }} checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRef: ${{ parameters.checkoutRef }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
dependencyList: ${{ parameters.rocmDependencies }} parameters:
aggregatePipeline: ${{ parameters.aggregatePipeline }} dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
- ${{ if ne(parameters.checkoutRef, '') }}:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
extraBuildFlags: >- extraBuildFlags: >-
@@ -79,57 +60,4 @@ jobs:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rocr_debug_agent_test_${{ job.target }}
dependsOn: rocr_debug_agent_build
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: rocr_debug_agent-tests
extraBuildFlags: >-
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake;$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_PATH=$(Agent.BuildDirectory)/rocm
cmakeBuildDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test'
cmakeSourceDir: '.'
installEnabled: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocr_debug_agent
testDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,43 +1,17 @@
parameters: parameters:
- name: componentName
type: string
default: roctracer
- name: checkoutRepo - name: checkoutRepo
type: string type: string
default: 'self' default: 'self'
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# monorepo related parameters
- name: sparseCheckoutDir
type: string
default: ''
- name: triggerDownstreamJobs
type: boolean
default: false
- name: downstreamAggregateNames
type: string
default: ''
- name: buildDependsOn
type: object
default: null
- name: unifiedBuild
type: boolean
default: false
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake - cmake
- ninja-build
- doxygen - doxygen
- graphviz - graphviz
- libdrm-amdgpu-dev
- ninja-build
- zlib1g-dev
- name: pipModules - name: pipModules
type: object type: object
default: default:
@@ -51,155 +25,46 @@ parameters:
- rocprofiler-register - rocprofiler-register
- clr - clr
- rocminfo - rocminfo
- name: rocmTestDependencies
type: object
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
testJobs:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: roctracer
- job: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }} variables:
${{ if parameters.buildDependsOn }}: - group: common
dependsOn: - template: /.azuredevops/variables-global.yml
- ${{ each build in parameters.buildDependsOn }}: - name: HIP_ROCCLR_HOME
- ${{ build }}_${{ job.os }}_${{ job.target }} value: $(Build.BinariesDirectory)/rocm
variables: pool:
- group: common vmImage: ${{ variables.BASE_BUILD_POOL }}
- template: /.azuredevops/variables-global.yml workspace:
pool: clean: all
vmImage: ${{ variables.BASE_BUILD_POOL }} steps:
${{ if eq(job.os, 'almalinux8') }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
container: parameters:
image: rocmexternalcicd.azurecr.io/manylinux228:latest aptPackages: ${{ parameters.aptPackages }}
endpoint: ContainerService3 pipModules: ${{ parameters.pipModules }}
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
steps: parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml checkoutRepo: ${{ parameters.checkoutRepo }}
parameters: # CI case: download latest default branch build
aptPackages: ${{ parameters.aptPackages }} - ${{ if eq(parameters.checkoutRef, '') }}:
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
os: ${{ job.os }} - ${{ if ne(parameters.checkoutRef, '') }}:
${{ if parameters.triggerDownstreamJobs }}: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
# the linker flags will not affect ubuntu2204 builds as the paths do not exist
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
os: ${{ job.os }} dependencyList: ${{ parameters.rocmDependencies }}
useAmdclang: false dependencySource: tag-builds
extraBuildFlags: >- - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
-DCMAKE_BUILD_TYPE=release parameters:
-DROCM_PATH=$(Agent.BuildDirectory)/rocm extraBuildFlags: >-
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip;$(Agent.BuildDirectory)/rocm/lib64/cmake/hip -DCMAKE_BUILD_TYPE=release
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DROCM_PATH=$(Agent.BuildDirectory)/rocm
-DGPU_TARGETS=${{ job.target }} -DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-GNinja -DGPU_TARGETS=gfx1030;gfx1100
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml -GNinja
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
componentName: ${{ parameters.componentName }}
sparseCheckoutDir: ${{ parameters.sparseCheckoutDir }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
componentName: ${{ parameters.componentName }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
# registerROCmPackages: true
- ${{ if eq(parameters.unifiedBuild, False) }}:
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: ${{ parameters.componentName }}_test_${{ job.os }}_${{ job.target }}
dependsOn: ${{ parameters.componentName }}_build_${{ job.os }}_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), '${{ parameters.componentName }}')),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
packageManager: ${{ job.packageManager }}
registerROCmPackages: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
preTargetFilter: ${{ parameters.componentName }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: ${{ job.target }}
os: ${{ job.os }}
${{ if parameters.triggerDownstreamJobs }}:
downstreamAggregateNames: ${{ parameters.downstreamAggregateNames }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ${{ parameters.componentName }}
testExecutable: $(Agent.BuildDirectory)/rocm/share/roctracer/run_tests.sh
testParameters: ''
testDir: $(Agent.BuildDirectory)
testPublishResults: false
os: ${{ job.os }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}
registerROCmPackages: true

View File

@@ -5,196 +5,61 @@ parameters:
- name: checkoutRef - name: checkoutRef
type: string type: string
default: '' default: ''
# set to true if doing full build of ROCm stack
# and dependencies are pulled from same pipeline
- name: aggregatePipeline
type: boolean
default: false
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- clang
- cmake - cmake
- imagemagick - libomp-dev # needed to pass flag step
- libopencv-dev
- libsndfile1-dev
- libstdc++-12-dev
- ninja-build - ninja-build
- python3-pip - clang
- name: pipModules
type: object
default:
- openpyxl
- pandas
- sphinx
- name: rocmDependencies - name: rocmDependencies
type: object type: object
default: default:
- aomp - aomp # needed to pass build step
- clr - clr
- half - half
- llvm-project - llvm-project
- rocm-cmake
- rocminfo - rocminfo
- ROCR-Runtime - ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- half
- hipTensor
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- name: jobMatrix
type: object
default:
buildJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
testJobs:
- gfx942:
target: gfx942
- gfx90a:
target: gfx90a
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: rpp
- job: rpp_build_${{ job.target }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
- name: ROCM_PATH vmImage: ${{ variables.BASE_BUILD_POOL }}
value: $(Agent.BuildDirectory)/rocm workspace:
pool: clean: all
vmImage: ${{ variables.BASE_BUILD_POOL }} steps:
workspace: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
clean: all parameters:
steps: aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
parameters: - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
aptPackages: ${{ parameters.aptPackages }} parameters:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml # CI case: download latest default branch build
parameters: - ${{ if eq(parameters.checkoutRef, '') }}:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }}
dependencyList: ${{ parameters.rocmDependencies }} dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: ${{ job.target }} dependencySource: staging
aggregatePipeline: ${{ parameters.aggregatePipeline }} # manual build case: triggered by ROCm/ROCm repo
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - ${{ if ne(parameters.checkoutRef, '') }}:
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DHALF_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=${{ job.target }}
-DROCM_PLATFORM_VERSION=$(NEXT_RELEASE_VERSION)
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
# - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
# parameters:
# aptPackages: ${{ parameters.aptPackages }}
# pipModules: ${{ parameters.pipModules }}
# gpuTarget: ${{ job.target }}
- ${{ each job in parameters.jobMatrix.testJobs }}:
- job: rpp_test_${{ job.target }}
dependsOn: rpp_build_${{ job.target }}
condition:
and(succeeded(),
eq(variables['ENABLE_${{ upper(job.target) }}_TESTS'], 'true'),
not(containsValue(split(variables['DISABLED_${{ upper(job.target) }}_TESTS'], ','), variables['Build.DefinitionName'])),
eq(${{ parameters.aggregatePipeline }}, False)
)
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
pool: ${{ job.target }}_test_pool
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: ${{ job.target }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters: parameters:
checkoutRef: ${{ parameters.checkoutRef }} dependencyList: ${{ parameters.rocmDependencies }}
dependencyList: ${{ parameters.rocmTestDependencies }} dependencySource: tag-builds
gpuTarget: ${{ job.target }} - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
# Dependencies from: https://github.com/ROCm/rpp/blob/develop/utilities/test_suite/README.md parameters:
- task: Bash@3 extraBuildFlags: >-
displayName: Build and install Turbo JPEG -DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
inputs: -DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
targetType: 'inline' -DROCM_PATH=$(Agent.BuildDirectory)/rocm
script: | -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
sudo apt-get install nasm -DHALF_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include
sudo apt-get install wget -DCMAKE_BUILD_TYPE=Release
git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git -DAMDGPU_TARGETS=gfx1030;gfx1100
cd libjpeg-turbo -GNinja
mkdir build - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_BUILD_TYPE=RELEASE \
-DENABLE_STATIC=FALSE \
-DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \
-DWITH_JPEG8=TRUE \
..
make -j$nproc
sudo make install
- task: Bash@3
displayName: Build and install Nifti
inputs:
targetType: 'inline'
script: |
git clone -b v3.0.1 https://github.com/NIFTI-Imaging/nifti_clib.git
cd nifti_clib
mkdir build
cd build
cmake ..
sudo make -j$nproc install
- task: Bash@3
displayName: Build rpp tests
inputs:
targetType: 'inline'
script: |
mkdir rpp-tests
cd rpp-tests
cmake $(Agent.BuildDirectory)/rocm/share/rpp/test \
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ \
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rpp
testExecutable: 'CMAKE_VERBOSE_MAKEFILE=ON VERBOSE=1 ctest'
testDir: 'rpp-tests'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/docker-container.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
environment: test
gpuTarget: ${{ job.target }}

View File

@@ -1,64 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: boostVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- git
- name: rocmDependencies
type: object
default:
- llvm-project
jobs:
- job: boost
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: 'git clone boost'
inputs:
targetType: inline
script: git clone -b ${{ parameters.boostVersion }} https://github.com/boostorg/boost --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- task: Bash@3
displayName: 'Build Boost with clang'
inputs:
targetType: inline
script: |
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
./bootstrap.sh --with-toolset=clang --prefix=$(Build.BinariesDirectory)
./b2 --toolset=clang threading=multi link=shared --prefix=$(Build.BinariesDirectory) cxxflags="-std=c++20"
./b2 install
workingDirectory: $(Build.SourcesDirectory)/boost
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

View File

@@ -1,63 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: catch2Version
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: catch2_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone catch2 ${{ parameters.catch2Version }}
inputs:
targetType: inline
script: git clone https://github.com/catchorg/Catch2.git -b ${{ parameters.catch2Version }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/Catch2/build
cmakeSourceDir: $(Agent.BuildDirectory)/Catch2
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,63 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: cli11Version
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: cli11_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone cli11 ${{ parameters.cli11Version }}
inputs:
targetType: inline
script: git clone https://github.com/CLIUtils/CLI11.git -b ${{ parameters.cli11Version }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/CLI11/build
cmakeSourceDir: $(Agent.BuildDirectory)/CLI11
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,67 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: fmtlibVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- libfmt-dev
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: fmtlib_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone fmtlib ${{ parameters.fmtlibVersion }}
inputs:
targetType: inline
script: git clone https://github.com/fmtlib/fmt.git -b ${{ parameters.fmtlibVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/fmt/build
cmakeSourceDir: $(Agent.BuildDirectory)/fmt
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DFMT_SYSTEM_HEADERS=ON
-DFMT_INSTALL=ON
-DFMT_TEST=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -20,7 +20,7 @@ jobs:
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all
@@ -38,7 +38,6 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters: parameters:
cmakeBuildDir: $(Agent.BuildDirectory)/grpc/build cmakeBuildDir: $(Agent.BuildDirectory)/grpc/build
cmakeSourceDir: $(Agent.BuildDirectory)/grpc
extraBuildFlags: >- extraBuildFlags: >-
-DgRPC_INSTALL=ON -DgRPC_INSTALL=ON
-DgRPC_BUILD_TESTS=OFF -DgRPC_BUILD_TESTS=OFF

View File

@@ -11,54 +11,35 @@ parameters:
- name: aptPackages - name: aptPackages
type: object type: object
default: default:
- cmake
- git - git
- cmake
- ninja-build - ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt }
- { os: almalinux8, packageManager: dnf }
jobs: jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}: - job: gtest
- job: gtest_${{ job.os }} variables:
variables: - group: common
- group: common - template: /.azuredevops/variables-global.yml
- template: /.azuredevops/variables-global.yml pool:
pool: vmImage: ${{ variables.BASE_BUILD_POOL }}
vmImage: 'ubuntu-22.04' workspace:
${{ if eq(job.os, 'almalinux8') }}: clean: all
container: steps:
image: rocmexternalcicd.azurecr.io/manylinux228:latest - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
endpoint: ContainerService3 parameters:
workspace: aptPackages: ${{ parameters.aptPackages }}
clean: all - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
steps: - task: Bash@3
- checkout: none displayName: 'git clone gtest'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml inputs:
parameters: targetType: inline
aptPackages: ${{ parameters.aptPackages }} script: git clone -b ${{ parameters.gtestVersion }} https://github.com/google/googletest --depth=1 --shallow-submodules --recurse-submodules
packageManager: ${{ job.packageManager }} workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- task: Bash@3 parameters:
displayName: Clone GTest ${{ parameters.gtestVersion }} cmakeBuildDir: $(Agent.BuildDirectory)/googletest/build
inputs: extraBuildFlags: >-
targetType: inline -DGTEST_FORCE_SHARED_CRT=ON
script: git clone https://github.com/google/googletest -b ${{ parameters.gtestVersion }} --depth=1 --shallow-submodules --recurse-submodules -DCMAKE_DEBUG_POSTFIX=d
workingDirectory: $(Agent.BuildDirectory) -GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml - template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/googletest/build
cmakeSourceDir: $(Agent.BuildDirectory)/googletest
useAmdclang: false
extraBuildFlags: >-
-DGTEST_FORCE_SHARED_CRT=ON
-DCMAKE_DEBUG_POSTFIX=d
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -19,7 +19,7 @@ jobs:
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
container: container:
image: rocm/dev-ubuntu-22.04:${{ variables.LATEST_RELEASE_VERSION }} image: ${{ variables.DOCKER_IMAGE_NAME }}:${{ variables.LATEST_DOCKER_VERSION }}
workspace: workspace:
clean: all clean: all
steps: steps:

View File

@@ -21,7 +21,7 @@ jobs:
variables: variables:
- group: common - group: common
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
pool: pool:
vmImage: ${{ variables.BASE_BUILD_POOL }} vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace: workspace:
clean: all clean: all

View File

@@ -1,64 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: libdivideVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: libdivide_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone libdivide ${{ parameters.libdivideVersion }}
inputs:
targetType: inline
script: git clone https://github.com/ridiculousfish/libdivide.git -b ${{ parameters.libdivideVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/libdivide/build
cmakeSourceDir: $(Agent.BuildDirectory)/libdivide
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DLIBDIVIDE_BUILD_TESTS=OFF
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,71 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: spdlogVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: spdlog_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-vendor.yml
parameters:
dependencyList:
- fmtlib
- task: Bash@3
displayName: Clone spdlog ${{ parameters.spdlogVersion }}
inputs:
targetType: inline
script: git clone https://github.com/gabime/spdlog.git -b ${{ parameters.spdlogVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/spdlog/build
cmakeSourceDir: $(Agent.BuildDirectory)/spdlog
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/vendor
-DCMAKE_BUILD_TYPE=Release
-DSPDLOG_USE_STD_FORMAT=OFF
-DSPDLOG_FMT_EXTERNAL_HO=ON
-DSPDLOG_INSTALL=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,66 +0,0 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: yamlcppVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- git
- ninja-build
- name: jobMatrix
type: object
default:
buildJobs:
- { os: ubuntu2204, packageManager: apt}
- { os: almalinux8, packageManager: dnf}
jobs:
- ${{ each job in parameters.jobMatrix.buildJobs }}:
- job: yamlcpp_${{ job.os }}
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: 'ubuntu-22.04'
${{ if eq(job.os, 'almalinux8') }}:
container:
image: rocmexternalcicd.azurecr.io/manylinux228:latest
endpoint: ContainerService3
workspace:
clean: all
steps:
- checkout: none
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
packageManager: ${{ job.packageManager }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: Clone yaml-cpp ${{ parameters.yamlcppVersion }}
inputs:
targetType: inline
script: git clone https://github.com/jbeder/yaml-cpp.git -b ${{ parameters.yamlcppVersion }}
workingDirectory: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
os: ${{ job.os }}
cmakeBuildDir: $(Agent.BuildDirectory)/yaml-cpp/build
cmakeSourceDir: $(Agent.BuildDirectory)/yaml-cpp
useAmdclang: false
extraBuildFlags: >-
-DCMAKE_BUILD_TYPE=Release
-DYAML_CPP_BUILD_TOOLS=OFF
-DYAML_BUILD_SHARED_LIBS=OFF
-DYAML_CPP_INSTALL=ON
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
os: ${{ job.os }}

View File

@@ -1,551 +0,0 @@
parameters:
# ubuntu near-equivalent list of yum installs in https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfile-centos-7-complete
# plus additional packages found through iterative testing of pipeline
- name: aptPackages
type: object
default:
- autoconf
- bc
- bridge-utils
- build-essential
- ca-certificates
- ccache
- devscripts
- dkms
- doxygen
- fakeroot
- ffmpeg
- gfortran
- git
- gnutls-bin
- libamd2
- libavformat-dev
- libblas3
- libcamd2
- libccolamd2
- libcholmod3
- libcolamd2
- libdpkg-dev
- libdpkg-perl
- libdrm-amdgpu1
- libdrm-dev
- libelf-dev
- libfreetype-dev
- libgfortran5
- libgomp1
- libjpeg-dev
- libjpeg-turbo-official
- liblapack-dev
- liblapack3
- libmetis5
- libncurses-dev
- libnuma-dev
- libopenblas-dev
- libpth-dev
- libquadmath0
- libssh-dev
- libstdc++-12-dev
- libsuitesparseconfig5
- libswscale-dev
- libtinfo-dev
- libunwind-dev
- libwebp-dev
- llvm-dev
- ncurses-base
- ninja-build
- numactl
- openjdk-8-jdk
- python-is-python3
- python3-dev
- python3-pip
- python3-venv
- qemu-kvm
- re2c
- subversion
- wget
- zip
- name: pipModules
type: object
default:
- astunparse
- "expecttest>=0.3.0"
- hypothesis
- numpy
- psutil
- pyyaml
- requests
- setuptools==75.8.0
- types-dataclasses
- "typing-extensions>=4.10.0"
- "sympy>=1.13.3"
- filelock
- networkx
- jinja2
- fsspec
- lintrunner
- ninja
- packaging
- "optree>=0.13.0"
- "click>=8.0.3"
# list for vision
- auditwheel
- future
- pytest
- pytest-azurepipelines
- pillow
# list from https://github.com/pytorch/pytorch/blob/main/.ci/manywheel/build_rocm.sh
- name: rocmDependencies
type: object
default:
- clr
- hipBLAS
- hipBLASLt
- hipFFT
- hipRAND
- hipSOLVER
- hipSPARSE
- hipSPARSELt
- llvm-project
- MIOpen
- rccl
- rocBLAS
- rocFFT
- rocm-core
- rocminfo
- rocm_smi_lib
- rocPRIM
- rocprofiler-register
- rocRAND
- ROCR-Runtime
- rocSOLVER
- rocSPARSE
- roctracer
# below are additional dependencies not called out by build script, but throw errors during cmake
- composable_kernel
- hipBLAS-common
- hipCUB
- rocThrust
- name: rocmTestDependencies
type: object
default:
# rocroller.so needed and is not included in the wheel
- hipBLASLt
- rocminfo
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_pytorch_test1.sh#L54
# Will iterate through this list using pytest
- name: torchTestList
type: object
default:
- nn
- torch
# - cuda seg faults and might need cuda installed on test system
# - ops takes too long
- unary_ufuncs
- binary_ufuncs
- autograd
# - inductor/torchinductor takes too long
# set to false to disable torchvision build and test
- name: includeVision
type: boolean
default: false
trigger: none
pr: none
schedules:
- cron: '30 7 * * *'
displayName: nightly pytorch
branches:
include:
- develop
always: true
jobs:
- job: pytorch
timeoutInMinutes: 120
strategy:
matrix:
amd-staging-gfx942:
ROCM_BRANCH: amd-staging
JOB_GPU_TARGET: gfx942
variables:
- group: common
- template: /.azuredevops/variables-global.yml
# various flags/parameters expected by bash scripts in pytorch repo's .ci directory
- name: ROCM_VERSION
value: 6.5.0
- name: ROCM_PATH
value: /opt/rocm
- name: DESIRED_CUDA
value: 6.5.0
- name: MAGMA_ROCM
value: 6.3
- name: DESIRED_PYTHON
value: 3.10
- name: PYTORCH_ROOT
value: $(Build.SourcesDirectory)/pytorch
- name: DESIRED_DEVTOOLSET
value: cxx11-abi
pool: ${{ variables.ULTRA_BUILD_POOL }}
workspace:
clean: all
steps:
# copy environment setup from https://github.com/pytorch/pytorch/blob/main/.ci/docker/manywheel/Dockerfile
# but instead of centos, use ubuntu environment
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
# wheel install location different on azure agent compared to where wheel is assumed to be installed on upstream script
- task: Bash@3
displayName: wheel install path symlink
inputs:
targetType: inline
script: |
sudo mkdir -p /opt/python/cp310-cp310/lib/python3.10
sudo ln -s /usr/local/lib/python3.10/dist-packages /opt/python/cp310-cp310/lib/python3.10/site-packages
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
setupHIPLibrarySymlinks: true
- task: Bash@3
displayName: ROCm symbolic link
inputs:
targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- checkout: self
- task: Bash@3
displayName: git clone upstream pytorch
inputs:
targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
# builder clone still needed due to run_tests.sh at end of build_common.sh call
- task: Bash@3
displayName: git clone pytorch builder
inputs:
targetType: inline
script: |
git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules
sudo ln -s $(Build.SourcesDirectory)/builder /builder
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install patchelf
inputs:
targetType: inline
script: |
sudo bash pytorch/.ci/docker/common/install_patchelf.sh
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install rocm drm
inputs:
targetType: inline
script: |
sudo bash pytorch/.ci/docker/common/install_rocm_drm.sh
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install rocm magma
inputs:
targetType: inline
script: |
sudo bash pytorch/.ci/docker/common/install_rocm_magma.sh $(MAGMA_ROCM)
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install targeted typing_extensions for build
inputs:
targetType: inline
script: pip install --target=$(Build.SourcesDirectory)/pytorch/torch/.. typing_extensions
- task: Bash@3
displayName: Run ROCm Build Script
inputs:
targetType: inline
script: >-
sudo
PYTHONPATH=/home/AzDevOps/.local/lib/python3.10/site-packages:/usr/lib/python3/dist-packages
DESIRED_CUDA=$(DESIRED_CUDA)
PYTORCH_ROCM_ARCH=$(JOB_GPU_TARGET)
GPU_TARGET=$(JOB_GPU_TARGET)
DESIRED_PYTHON=$(DESIRED_PYTHON)
PYTORCH_ROOT=$(PYTORCH_ROOT)
CMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
DESIRED_DEVTOOLSET=$(DESIRED_DEVTOOLSET)
PYTORCH_BUILD_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)
PYTORCH_BUILD_NUMBER=$(date -u +%Y%m%d)
SKIP_ALL_TESTS=1
bash ./.ci/manywheel/build_rocm.sh
workingDirectory: $(Build.SourcesDirectory)/pytorch
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: /remote/wheelhouserocm$(ROCM_VERSION)
contentsString: '*.whl'
# common helper source for pytorch vision and audio
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3
displayName: git clone pytorch test-infra
inputs:
targetType: inline
script: git clone https://github.com/pytorch/test-infra.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: install package helper
inputs:
targetType: inline
script: python3 -m pip install test-infra/tools/pkg-helpers
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: pytorch pkg helpers
inputs:
targetType: inline
script: CU_VERSION=${CU_VERSION} CHANNEL=${CHANNEL} python -m pytorch_pkg_helpers
# get torch vision source and build
- task: Bash@3
displayName: git clone pytorch vision
inputs:
targetType: inline
script: git clone https://github.com/pytorch/vision.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Build vision
inputs:
targetType: inline
script: >-
PYTORCH_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)post$(date -u +%Y%m%d)
BUILD_VERSION=$(cat $(Build.SourcesDirectory)/vision/version.txt | cut -da -f1)post$(date -u +%Y%m%d)
python3 setup.py bdist_wheel
workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3
displayName: Relocate vision
inputs:
targetType: inline
script: python3 packaging/wheel/relocate.py
workingDirectory: $(Build.SourcesDirectory)/vision
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Build.SourcesDirectory)/vision/dist
contentsString: '*.whl'
clean: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.BinariesDirectory)
- task: Bash@3
displayName: Save pipeline artifact file name
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: |
whlFile=$(find "$(Build.BinariesDirectory)" -type f -name "*.whl" | head -n 1)
if [ -n "$whlFile" ]; then
echo $(basename "$whlFile") >> pipelineArtifacts.txt
fi
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- job: pytorch_testing
dependsOn: pytorch
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTORCH_TEST_WITH_ROCM
value: 1
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
# pytorch tests require an updated version of click, even if requirements is not called outright
- task: Bash@3
displayName: 'pip update click'
inputs:
targetType: inline
script: pip install --upgrade click
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
retryCountOnTaskFailure: 3
inputs:
itemPattern: '**/*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# get sources to run test scripts
- task: Bash@3
displayName: git clone upstream pytorch
inputs:
targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3
displayName: git clone pytorch vision
inputs:
targetType: inline
script: git clone https://github.com/pytorch/vision.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip install --no-index --find-links=. --no-dependencies -v {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Show Updated pip List
inputs:
targetType: inline
script: pip list -v
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- task: Bash@3
displayName: Add torch libs to ldconfig
inputs:
targetType: inline
script: |
echo $(python3 -m site --user-site)/torch/lib | sudo tee /etc/ld.so.conf.d/torch.conf
sudo ldconfig -v
ldconfig -p
# https://pytorch.org/get-started/locally/#linux-verification
# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#testing-the-pytorch-installation
- task: Bash@3
displayName: Simple Import Torch Tests
inputs:
targetType: inline
script: |
python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'
python3 -c 'import torch; x = torch.rand(5, 3); print(x)'
# Test artifact build script has too many if statements for different environments
# Based off the snippet of interest for this environment, with some adjustments
# https://github.com/pytorch/pytorch/blob/main/.ci/pytorch/build.sh#L330-L371
# Removing in-line comments since it does not fit with the yaml markup
- task: Bash@3
displayName: Build Pytorch Test Artifacts
continueOnError: true
inputs:
targetType: inline
script: |
CUSTOM_TEST_ARTIFACT_BUILD_DIR="build/custom_test_artifacts"
CUSTOM_TEST_USE_ROCM=ON
CUSTOM_TEST_MODULE_PATH="${PWD}/cmake/public"
mkdir -pv "${CUSTOM_TEST_ARTIFACT_BUILD_DIR}"
CUSTOM_OP_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-op-build"
CUSTOM_OP_TEST="${PWD}/test/custom_operator"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$CUSTOM_OP_BUILD"
pushd "$CUSTOM_OP_BUILD"
cmake "$CUSTOM_OP_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
JIT_HOOK_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/jit-hook-build"
JIT_HOOK_TEST="$PWD/test/jit_hooks"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$JIT_HOOK_BUILD"
pushd "$JIT_HOOK_BUILD"
cmake "$JIT_HOOK_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
CUSTOM_BACKEND_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-backend-build"
CUSTOM_BACKEND_TEST="${PWD}/test/custom_backend"
python --version
mkdir -p "$CUSTOM_BACKEND_BUILD"
pushd "$CUSTOM_BACKEND_BUILD"
cmake "$CUSTOM_BACKEND_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
workingDirectory: $(Build.SourcesDirectory)/pytorch
- ${{ each torchTest in parameters.torchTestList }}:
- task: Bash@3
displayName: Test ${{ torchTest }}
continueOnError: true
inputs:
targetType: inline
workingDirectory: $(Build.SourcesDirectory)/pytorch
${{ if contains(torchTest, '/') }}:
script: pytest test/${{ split(torchTest, '/')[0] }}/test_${{ split(torchTest, '/')[1] }}.py
${{ else }}:
script: pytest test/test_${{ torchTest }}.py
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_torchvision.sh#L51
- ${{ if eq(parameters.includeVision, true) }}:
- task: Bash@3
displayName: Test vision/transforms
continueOnError: true
inputs:
targetType: inline
script: pytest test/test_transforms.py
workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3
displayName: Uninstall Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip uninstall -y {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Remove Python site-packages binaries from path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"

View File

@@ -1,239 +0,0 @@
parameters:
# using a parameter instead of a strategy matrix allows job variables to be eval'd at compile-time rather than runtime
- name: jobList
type: object
default:
- { os: ubuntu2204, packageManager: apt, target: gfx942 }
- { os: ubuntu2204, packageManager: apt, target: gfx90a }
- { os: ubuntu2204, packageManager: apt, target: gfx1201 }
- { os: ubuntu2204, packageManager: apt, target: gfx1100 }
- { os: ubuntu2204, packageManager: apt, target: gfx1030 }
- { os: ubuntu2404, packageManager: apt, target: gfx942 }
- { os: ubuntu2404, packageManager: apt, target: gfx90a }
- { os: ubuntu2404, packageManager: apt, target: gfx1201 }
- { os: ubuntu2404, packageManager: apt, target: gfx1100 }
- { os: ubuntu2404, packageManager: apt, target: gfx1030 }
- { os: almalinux8, packageManager: dnf, target: gfx942 }
- { os: almalinux8, packageManager: dnf, target: gfx90a }
- { os: almalinux8, packageManager: dnf, target: gfx1201 }
- { os: almalinux8, packageManager: dnf, target: gfx1100 }
- { os: almalinux8, packageManager: dnf, target: gfx1030 }
- name: rocmDependencies
type: object
default:
- AMDMIGraphX
- amdsmi
- aomp-extras
- aomp
- clr
- composable_kernel
- half
- hip-tests
- hipBLAS
- hipBLAS-common
- hipBLASLt
- hipCUB
- hipFFT
- hipfort
- HIPIFY
- hipRAND
- hipSOLVER
- hipSPARSE
- hipSPARSELt
- hipTensor
- llvm-project
- MIOpen
- MIVisionX
- rccl
- rdc
- rocAL
- rocALUTION
- rocBLAS
- ROCdbgapi
- rocDecode
- rocFFT
- ROCgdb
- rocJPEG
- rocm-cmake
- rocm-core
- rocm-examples
- rocminfo
- rocMLIR
- ROCmValidationSuite
- rocm_bandwidth_test
- rocm_smi_lib
- rocPRIM
- rocprofiler-compute
- rocprofiler-register
- rocprofiler-sdk
- rocprofiler-systems
- rocprofiler
- rocPyDecode
- ROCR-Runtime
- rocRAND
- rocr_debug_agent
- rocSOLVER
- rocSPARSE
- rocThrust
- roctracer
- rocWMMA
- rpp
- TransferBench
trigger: none
pr: none
schedules:
- cron: '30 7 * * *'
displayName: Nightly build
branches:
include:
- develop
always: true
jobs:
- ${{ each job in parameters.jobList }}:
- job: nightly_${{ job.os }}_${{ job.target }}
timeoutInMinutes: 120
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
steps:
- task: DeleteFiles@1
displayName: 'Cleanup checkout space'
inputs:
SourceFolder: '$(Agent.BuildDirectory)/s'
Contents: '**/*'
- task: DeleteFiles@1
displayName: 'Cleanup Staging Area'
inputs:
SourceFolder: '$(Build.ArtifactStagingDirectory)'
Contents: '/**/*'
RemoveDotFiles: true
- script: df -h
displayName: System disk space before ROCm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
os: ${{ job.os }}
gpuTarget: ${{ job.target }}
- script: df -h
displayName: System disk space after ROCm
- script: du -sh $(Agent.BuildDirectory)/rocm
displayName: Uncompressed ROCm size
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/manifest.yml
- task: ArchiveFiles@2
displayName: Compress rocm-nightly
inputs:
rootFolderOrFile: $(Agent.BuildDirectory)/rocm
includeRootFolder: false
archiveType: tar
tarCompression: gz
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.DefinitionName)_$(Build.BuildNumber)_${{ job.os }}_${{ job.target }}.tar.gz
- script: du -sh $(Build.ArtifactStagingDirectory)
displayName: Compressed ROCm size
- task: PublishPipelineArtifact@1
displayName: 'Publish ROCm Nightly Artifact'
retryCountOnTaskFailure: 3
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
- task: Bash@3
displayName: Save pipeline artifact file name
inputs:
workingDirectory: $(Pipeline.Workspace)
targetType: inline
script: echo "$(Build.DefinitionName)_$(Build.BuildNumber)_${{ job.os }}_${{ job.target }}.tar.gz" >> pipelineArtifacts.txt
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-links.yml
- ${{ if eq(job.packageManager, 'apt') }}:
- task: Bash@3
displayName: Create Dockerfile
inputs:
workingDirectory: $(Agent.BuildDirectory)
targetType: inline
script: |
cat <<'EOF' > Dockerfile
${{ iif(eq(job.os, 'ubuntu2204'), 'FROM ubuntu:22.04', '') }}
${{ iif(eq(job.os, 'ubuntu2404'), 'FROM ubuntu:24.04', '') }}
WORKDIR /root
RUN mkdir rocm
RUN apt update \
&& apt upgrade -y \
&& apt install -y cmake curl git gcc g++ gpg lsb-release lsof ninja-build pkg-config python3 python3-pip wget zip libdrm-dev libelf-dev libgtest-dev libhsakmt-dev libhwloc-dev libnuma-dev libstdc++-12-dev libtbb-dev jq \
&& apt clean all
RUN PACKAGE_NAME=$(curl -s https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/ | grep -oP "href=\"\K[^\"]*$(lsb_release -rs)[^\"]*\.deb") \
&& wget -nv --retry-connrefused https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/$PACKAGE_NAME \
&& mkdir hsa-amd-aqlprofile \
&& dpkg-deb -R $PACKAGE_NAME hsa-amd-aqlprofile \
&& cp -R hsa-amd-aqlprofile/opt/rocm-*/* rocm
RUN ARTIFACT_URL="https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/$(Build.BuildId)/artifacts?artifactName=nightly${{ job.os }}${{ job.target }}&api-version=7.1" \
&& DOWNLOAD_URL=$(curl -s $ARTIFACT_URL | jq ".resource.downloadUrl" | tr -d '"') \
&& wget -nv --retry-connrefused $DOWNLOAD_URL -O nightly.zip \
&& unzip nightly.zip \
&& tar -xf nightly${{ job.os }}${{ job.target }}/rocm-nightly*${{ job.os }}*${{ job.target }}*.tar.gz -C rocm
RUN echo /root/rocm/lib | tee /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN ldconfig -v
ENV PATH="$PATH:/root/rocm/bin"
ENTRYPOINT ["/bin/bash"]
EOF
cat Dockerfile
- ${{ elseif eq(job.packageManager, 'dnf') }}:
- task: Bash@3
displayName: Create Dockerfile
inputs:
workingDirectory: $(Agent.BuildDirectory)
targetType: inline
script: |
cat <<'EOF' > Dockerfile
${{ iif(eq(job.os, 'almalinux8'), 'FROM almalinux:8', '') }}
WORKDIR /root
RUN mkdir rocm
RUN dnf install -y cmake curl git gcc gcc-c++ gnupg2 redhat-lsb-core lsof pkgconf python3 python3-pip wget zip libdrm-devel elfutils-libelf-devel numactl-devel libstdc++-devel tbb-devel jq \
&& dnf clean all
RUN PACKAGE_NAME=$(curl -s https://repo.radeon.com/rocm/rhel8/$(REPO_RADEON_VERSION)/main/ | grep -oP "hsa-amd-aqlprofile-[^\"]+\.rpm" | head -n1) \
&& wget -nv --retry-connrefused https://repo.radeon.com/rocm/rhel8/$(REPO_RADEON_VERSION)/main/$PACKAGE_NAME \
&& mkdir hsa-amd-aqlprofile \
&& dnf -y install rpm-build cpio \
&& rpm2cpio $PACKAGE_NAME | (cd hsa-amd-aqlprofile && cpio -idmv) \
&& cp -R hsa-amd-aqlprofile/opt/rocm-*/* rocm
RUN ARTIFACT_URL="https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/$(Build.BuildId)/artifacts?artifactName=nightly${{ job.os }}${{ job.target }}&api-version=7.1" \
&& DOWNLOAD_URL=$(curl -s $ARTIFACT_URL | jq ".resource.downloadUrl" | tr -d '"') \
&& wget -nv --retry-connrefused $DOWNLOAD_URL -O nightly.zip \
&& UNZIP_DISABLE_ZIPBOMB_DETECTION=TRUE unzip nightly.zip \
&& tar -xf nightly${{ job.os }}${{ job.target }}/rocm-nightly*${{ job.os }}*${{ job.target }}*.tar.gz -C rocm
RUN echo /root/rocm/lib | tee /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN echo /root/rocm/llvm/lib64 | tee -a /etc/ld.so.conf.d/rocm-ci.conf
RUN ldconfig -v
ENV PATH="$PATH:/root/rocm/bin"
ENTRYPOINT ["/bin/bash"]
EOF
cat Dockerfile
- task: Docker@2
displayName: Build and upload Docker image
retryCountOnTaskFailure: 3
inputs:
containerRegistry: ContainerService3
repository: 'nightly-${{ job.os }}-${{ job.target }}'
Dockerfile: '$(Agent.BuildDirectory)/Dockerfile'
buildContext: '$(Agent.BuildDirectory)'
- task: Bash@3
displayName: '!! Docker Run Command !!'
inputs:
targetType: inline
script: echo "docker run -it --network=host --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined rocmexternalcicd.azurecr.io/nightly-${{ job.os }}-${{ job.target }}:$(Build.BuildId)" | tr '[:upper:]' '[:lower:]'

View File

@@ -1,68 +0,0 @@
#!/usr/bin/env python3
import json
import requests
import argparse
from pathlib import Path
def get_builds(entries, gpu_target, output):
already_downloaded = {}
for entry in entries:
already_downloaded = _get_builds(entry, gpu_target, already_downloaded, output)
def _get_builds(entry, gpu_target, already_downloaded, output):
print()
print(f"{entry['buildNumber']} - {entry['buildId']} - {entry['repoName']}")
if already_downloaded.get(entry['buildId']):
print('Skipping, already downloaded from build ' + entry['buildId'])
return already_downloaded
artifacts_url = f"https://dev.azure.com/ROCm-CI/ROCm-CI/_apis/build/builds/{entry['buildId']}/artifacts?api-version=7.1"
artifacts = requests.get(artifacts_url).json()
for artifact in artifacts['value']:
if 'gfx' in artifact['name'] and gpu_target not in artifact['name']:
continue
print('Artifact name: ' + artifact['name'])
print('File size: ~' +
str(round(int(artifact['resource']['properties']['artifactsize'])/1000000, 2)) + ' MB')
download_url = f"{artifact['resource']['downloadUrl']}"
download = requests.get(download_url)
zip_file = Path(output) / f"{artifact['name']}.zip"
with open(zip_file, 'wb') as f:
f.write(download.content)
already_downloaded[entry['buildId']] = True
return already_downloaded
def main():
parser = argparse.ArgumentParser(description="Command line tool for downloading external ci artifacts")
parser.add_argument('--target', type=str, dest="target", choices=["gfx90a", "gfx942"], help="Target gfx")
parser.add_argument('--manifest', type=str, dest="manifest", help='JSON manifest url or path to local manifest')
parser.add_argument('--output_dir', type=str, dest="output", help='Path to download directory')
args = parser.parse_args()
manifest = args.manifest
gpu_target = args.target
if not gpu_target:
print("Enter the GPU target (gfx942, gfx90a)")
gpu_target = input()
if not manifest:
print("Enter the manifest file (URL or local path)")
manifest = input()
if 'http' in manifest:
data = requests.get(manifest).json()
else:
with open(manifest, 'r') as f:
data = json.load(f)
entries = [e for e in data['current']]
entries.extend([e for e in data['dependencies']])
get_builds(entries, gpu_target, args.output)
if __name__ == "__main__":
main()

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/Tensile
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/Tensile.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/TransferBench
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/TransferBench.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -3,41 +3,22 @@ variables:
- template: /.azuredevops/variables-global.yml - template: /.azuredevops/variables-global.yml
parameters: parameters:
- name: aompRef - name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: extrasRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: flangRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
- name: llvmRef
type: string type: string
default: refs/tags/$(LATEST_RELEASE_TAG) default: refs/tags/$(LATEST_RELEASE_TAG)
resources: resources:
repositories: repositories:
- repository: aomp_repo - repository: release_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/aomp name: ROCm/aomp
ref: ${{ parameters.aompRef }} ref: ${{ parameters.checkoutRef }}
- repository: aomp-extras_repo
type: github
endpoint: ROCm
name: ROCm/aomp-extras
ref: ${{ parameters.extrasRef }}
- repository: flang_repo
type: github
endpoint: ROCm
name: ROCm/flang
ref: ${{ parameters.flangRef }}
- repository: llvm-project_repo - repository: llvm-project_repo
type: github type: github
endpoint: ROCm endpoint: ROCm
name: ROCm/llvm-project name: ROCm/llvm-project
ref: ${{ parameters.llvmRef }} ref: ${{ parameters.checkoutRef }}
trigger: none trigger: none
pr: none pr: none
@@ -45,5 +26,5 @@ pr: none
jobs: jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml - template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml
parameters: parameters:
checkoutRepo: aomp_repo checkoutRepo: release_repo
checkoutRef: ${{ parameters.aompRef }} checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: boostVersion
type: string
default: 'master'
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/boost.yml
parameters:
boostVersion: ${{ parameters.boostVersion }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: catch2Version
type: string
default: "v3.7.0"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/catch2.yml
parameters:
catch2Version: ${{ parameters.catch2Version }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: cli11Version
type: string
default: "main"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/cli11.yml
parameters:
cli11Version: ${{ parameters.cli11Version }}

View File

@@ -28,22 +28,12 @@ resources:
endpoint: ROCm endpoint: ROCm
name: ROCm/hipother name: ROCm/hipother
ref: ${{ parameters.checkoutRef }} ref: ${{ parameters.checkoutRef }}
pipelines:
- pipeline: hip_pipeline
source: \experimental\HIP
trigger: true
- pipeline: hipother_pipeline
source: \experimental\hipother
trigger: true
trigger: none trigger: none
pr: none pr: none
jobs: jobs:
- ${{ if eq(variables['Build.Reason'], 'ResourceTrigger') }}: - template: ${{ variables.CI_COMPONENT_PATH }}/HIP.yml
- template: ${{ variables.CI_COMPONENT_PATH }}/copyHIP.yml@pipelines_repo parameters:
- ${{ if ne(variables['Build.Reason'], 'ResourceTrigger') }}: checkoutRepo: release_repo
- template: ${{ variables.CI_COMPONENT_PATH }}/HIP.yml@pipelines_repo checkoutRef: ${{ parameters.checkoutRef }}
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: fmtlibVersion
type: string
default: "11.1.3"
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/fmtlib.yml
parameters:
fmtlibVersion: ${{ parameters.fmtlibVersion }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/hip-tests
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/hip-tests.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: develop
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/hipBLAS-common
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/hipBLAS-common.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,23 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: libdivideVersion
type: string
default: master
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/libdivide.yml
parameters:
libdivideVersion: ${{ parameters.libdivideVersion }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/omnitrace
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/omnitrace.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocAL
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocAL.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocJPEG
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocJPEG.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocPyDecode
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocPyDecode.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-compute
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-compute.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

View File

@@ -1,29 +0,0 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-sdk
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-sdk.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

Some files were not shown because too many files have changed in this diff Show More