github/ROCm
1
1
Fork 0
mirror of https://github.com/ROCm/ROCm.git synced 2026-01-09 14:48:06 -05:00
Code Issues Packages Projects Releases Wiki Activity

Sync change from ROCm to ROCm-internal (#194)

Browse Source 
* Fix Radeon link and point at R6.1.3 as absolute link (#3757)

* Update ROCm manifest to 6.2.1

* Update ROCm branch name

* Add 6.2.1 to version list (#3770)

* Add links to GH issues in 6.2.1 release notes (#3769)

* add MAD page

* link to GitHub issues in release notes known issues

* update templates for 6.2.1

* Revert "add MAD page"

This reverts commit 9cce72bba3.

* update wordlist for spellcheck linter

* add rccl note

* update rocal version change heading to be more obvious

* make rocal note more specific

* fix missing space

* fix capitalization

* Update RCCL known issue wording (#3775)

* add MAD page

* fix wording in RCCL known issue

* Revert "add MAD page"

This reverts commit c81d0f3b0a.

* update llvm version for 6.2.1 (#3779)

* Fix broken links in 6.2.1 release notes (#3782)

* External CI: Replace libomp dependencies with aomp (#3781)

Add roctracer dependency for hipBLAS and rocWMMA testing

* External CI: Add rocprofiler v1 and v2 smoke tests (#3784)

* External CI: ROCgdb smoke tests (#3785)

- Since this is an autotools project and not cmake, build and test on gfx942 system instead of separating into two jobs. Pipeline time is short anyway.
- Follow build instructions to update build flags and to incorporate the ROCdbgapi.
- Results are not parsed and graphed, but the log contents are printed at the end. This was helpful for debugging and will be kept in the pipeline, as the make check-gdb command's output was not helpful on its own.

* External CI: rocPyDecode Smoke Test (#3786)

* External CI: omniperf pipeline (#3788)

- Referred to public documentation, source, and iterative attempts to create and improve build and test pipeline.
- ctest failures are due to the test node not having expected marketing name string and override not working.
- The fix should be on the omniperf repo side of things, so this pull request should be fine as is.

* External CI: create omniperf pipeline IDs, update nightly build (#3790)

* Fixed greater than to be less than in rocFFT changes

* fix footnote for 6.1.0 (#3791)

* fix footnote for 6.1.0

* fix empty columns in historical KFD title

* External CI: Publish wheel as artifact for rocPyDecode (#3796)

* External CI: fix hip-tests symlink creation (#3799)

* Docs: Add Ubuntu 24.04.1 (#3801)

* add ubuntu 24.04.1

* add 24.04.1 to bottom os section

* fix heading and template

* Update compatibility-matrix.rst for OpenMP version

* Update compatibility-matrix-historical-6.0.csv for OpenMP version

* rm ubuntu 24.04.1 from 6.2.0

* Update docs/compatibility/compatibility-matrix.rst

Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>

* rm duplicate ubuntu in historical

---------

Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>

* External CI: fixes for rocMLIR and nightly build (#3800)

* External CI: fix symlinks for rocMLIR and nightly build

* add pipeline IDs for hip-tests

* fix hip-test ID typo

* remove llvm-alt license (#3727)

* remove llvm-alt license

* fix linting error

* External CI: enable ROCR-Runtime tests (#3809)

* External CI: default branches for hip-tests, omniperf (#3811)

* External CI: torch and torchvision smoke tests (#3810)

* External CI: torch and torchvision smoke tests

- Fixed issues with package name and version for the vision wheel that prevented it from installing. A patch is used until my pull request in vision repo is merged.
- Referred to rocAutomation scripts to pick which test scripts to run out of the many in the torch and vision repo, and iteratively tested suggested scripts to see which ones completed in a timely manner.
- Leveraging pytest-azurepipelines module to automatically parse and graph results from these tests.

* External CI: omnitrace build pipeline (#3812)

* External CI: omnitrace build pipeline starter

- Adding initial set of dependencies and build flags.

* External CI: omnitrace build pipeline

- Add bison, rccl, texinfo dependencies based on build failures.
- Add AMDGPU_TARGETS flag
- Add ROCm binaries to PATH for clang-format and other tools used.

* Fix indentation

---------

Co-authored-by: Daniel Su <danielsu@amd.com>

* External CI: AMDMIGraphX Build Fix (#3814)

- Swap to default gcc on OS to resolve build errors from recent commits.
- Added libdnnl-dev dependency from iterative attempts with compiler change.
- Referred to the passing GitHub checks to observe the compilers that was used.
- Build CK jit lib and include in AMDMIGraphX build.

* External CI: test fixes w/ roctracer, list omniperf as partially succeeding (#3815)

* External CI: rpp tests (#3816)

* External CI: Build pipeline for rocprofiler-sdk (#3819)

* External CI: Pipeline for rocprofiler-sdk

* Add rocprofiler dependency

* External CI: rocprofiler-sdk build pipeline

---------

Co-authored-by: Daniel Su <danielsu@amd.com>

* External CI: Fix/add missing pipeline IDs (#3818)

* External CI: omnitrace tests (#3822)

* Update tags to 6.2.2 (#3827)

* External CI: add roctracer to roc/hipSOLVER test deps (#3825)

* External CI: add rocprofiler-sdk pipeline IDs (#3824)

* External CI: AMDMIGraphX Smoke Tests (#3830)

Co-authored-by: Daniel Su <danielsu@amd.com>

* External CI: MIOpen tests (#3837)

* Point to release history instead of deprecated changelog (#3836)

* External CI: filter out hipTensor extended tests (#3838)

* added revised note re. radeon gpus (#3839)

* Restructured the contributions section. (#3715)

* testing if this file is editable

* changed 'kebob-case' to 'dash-case'

* Restructured the page to be more straightforward and provide additional repo information

* forgot to save

* Moved the topic sentence

* Wrong accent on the a in diataxis

* Removed the feedback info from contributing and moved it to Feedback

* fixed spelling errors

* fixed some wording and removed second person text

* consolidated Build and Structure into Contribute; edited toolchai to (hopefully) conform to style guide; updated toc

* updated the titles in the toc

* made changes based on feedback

* it's better when you save

* removed structure and build; fixed something for the linter

* added rst to wordlist

* added customizations to wordlist

* Add links to gpu cluster network guides (#3763)

* Add links to gpu cluster network guides

* Add newline character to eof

* Make link absolute

* add dynamic branch in toc

* remove unnecessary page

clean up

* clean up index/toc

* make multi-node topics adjacent

---------

Co-authored-by: Peter Park <peter.park@amd.com>

* updated the radeon note (#3850)

* External CI: Fix rocPyDecode wheel creation (#3852)

- Set values for expected environment variables.
- Accompanying changes required in rocPyDecode repo. Pull request will be made.

* External CI: pytorch vision patch removal (#3855)

My pull request applying this patch was merged upstream, so this is no longer needed and will break the pipeline since it can no longer be applied.

* Build(deps): Bump rocm-docs-core from 1.8.1 to 1.8.2 in /docs/sphinx (#3807)

Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/v1.8.2/CHANGELOG.md)
- [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* updated the radeon note, as it were (#3857)

* updated the radeon note, as it were

* updated the note again

* Set devops team as codeowners for rocm-build (#3860)

* Set ext CI as codeowners for rocm-build

* Update CODEOWNERS to rocm-devops

* External CI: Add option to pull mainline branch for dependencies (#3689)

* External CI: Add option to pull mainline branch for dependencies

* Missing parameter for mainline branch dependencies.

* External CI: mainline branch definitions

* Removed MIGraphX optimization page (#3848)

* External CI: add a global variable to control gfx942 tests (#3864)

* External CI: update component default/mainline branches (#3871)

* External CI: Stop building gfx90a (#3872)

Save on VM resources until infrastructure has test targets.

* External CI: add libstdc++-12 to rocMLIR (#3874)

* Add building doc section (#3873)

* External CI: programmatically get latest aqlprofile (#3876)

* External CI: use ctest for rocm-examples (#3877)

* External CI: Tensile pipeline (#3884)

* add oversubscription conceptual doc (#3885)

add mitigiation steps

add to toc

move page for build

move doc

fix spelling

update doc

update oversubscription

update order

fix spelling

add oversubscription to wordlist

move oversubscription topic to bottom of toc and index

* add oversubscription conceptual doc (#3885)

(cherry picked from commit d0ecf51b0c)

* External CI: Add pipeline to build upstream boost (#3896)

* Update bitsandbytes branch in docs (#3898)

* Documentation: Add reference to precision-support floating-point types (#3899)

* External CI: use Boost template for MIOpen (#3903)

* External CI: create rocprofiler-systems pipeline (#3906)

* External CI: omnitrace/rocprof-sys pipeline IDs (#3908)

* External CI: MIOpen parse test results (#3913)

* External CI: Use pip to install latest cmake on test system (#3915)

* added a link to the compatibility matrix (#3904)

* added a link to the compatibility matrix

* removed quotes

* docs: Remove invalid amd_iommu=on parameter

Per kernel-parameters.txt, there is no "on" option for amd_iommu. While
intel_iommu has it, amd_iommu is automatically on unless specified
otherwise. For more info, see these 2 links:

https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt
75aa74d52f/drivers/iommu/amd/init.c (L3481)

Signed-off-by: Kent Russell <kent.russell@amd.com>

* External CI: hipBLASLt build now requires python packaging module (#3926)

https://github.com/ROCm/hipBLASLt/pull/1250/files#diff-fee2e6f068b33fca3a1dc49392de8848dbf05c3f4632b680abb1052523e5a30fR35

* External CI: Moved location of upstream pytorch build scripts (#3930)

https://github.com/pytorch/pytorch/pull/138103

* External CI: disable rocMLIR tests (#3931)

* External CI: disable rocMLIR tests

* roctracer AMDGPU_TARGETS flag

* External CI: create a GPU diagnostics template (#3932)

* External CI: Add CK into pytorch build environment (#3934)

* External CI: add support to disable individual component tests (#3938)

* External CI: AMDMIGraphX greater-equal pip dependencies (#3939)

* Build(deps): Bump rocm-docs-core from 1.8.2 to 1.8.3 in /docs/sphinx (#3933)

Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.8.2...v1.8.3)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* External CI: rocDecode add libva-amdgpu-dev dependency (#3940)

* External CI: enumerate GPUs in gpu-diagnostics (#3942)

* External CI: move gpu-diag directly before tests (#3943)

* External CI: fix HIP_PIPELINE_ID (#3944)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kent Russell <kent.russell@amd.com>
Co-authored-by: Jeffrey Novotny <jnovotny@amd.com>
Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
Co-authored-by: Wang, Yanyao <yanyao.wang@amd.com>
Co-authored-by: Yanyao Wang <yanywang@amd.com>
Co-authored-by: Peter Park <peter.park@amd.com>
Co-authored-by: Young Hui - AMD <145490163+yhuiYH@users.noreply.github.com>
Co-authored-by: Joseph Macaranas <145489236+amd-jmacaran@users.noreply.github.com>
Co-authored-by: Daniel Su <danielsu@amd.com>
Co-authored-by: Sandra Polifroni <sandra.polifroni@amd.com>
Co-authored-by: randyh62 <42045079+randyh62@users.noreply.github.com>
Co-authored-by: Michael Benavidez <michael.benavidez@amd.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: MKKnorr <MKKnorr@web.de>
Co-authored-by: Kent Russell <kent.russell@amd.com>
Co-authored-by: Joseph Greathouse <jlgreathouse@users.noreply.github.com>
This commit is contained in:
alexxu-amd
2024-10-25 14:41:40 -04:00
committed by GitHub
parent b9ec9507db
commit 8e3d51c31d
101 changed files with 2480 additions and 930 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
.azuredevops/ci-builds/aomp-mainline.yml Normal file
View File

@@ -0,0 +1,42 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
resources:
repositories:
- repository: aomp_repo
type: github
endpoint: ROCm
name: ROCm/aomp
ref: amd-mainline-open
- repository: aomp-extras_repo
type: github
endpoint: ROCm
name: ROCm/aomp-extras
ref: amd-mainline-open
- repository: flang_repo
type: github
endpoint: ROCm
name: ROCm/flang
ref: amd-mainline-open
- repository: llvm-project_repo
type: github
endpoint: ROCm
name: ROCm/llvm-project
ref: amd-mainline-open
pipelines:
- pipeline: rocr-runtime_pipeline
source: \ROCR-Runtime
trigger:
branches:
include:
- amd-master
# this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/aomp.yml
parameters:
checkoutRepo: aomp_repo

5
.azuredevops/ci-builds/aomp.yml → .azuredevops/ci-builds/aomp-staging.yml
View File

@@ -27,7 +27,10 @@ resources:
pipelines:
- pipeline: rocr-runtime_pipeline
source: \ROCR-Runtime
trigger: true
trigger:
branches:
include:
- amd-staging
# this job will only be triggered after successful build sequence of llvm-project and ROCR-Runtime
trigger: none

126
.azuredevops/components/AMDMIGraphX.yml
View File

@@ -8,30 +8,31 @@ parameters:
- name: aptPackages
type: object
default:
- git
- cmake
- ninja-build
- git
- libdnnl-dev
- libdrm-dev
- libnuma-dev
- python3-pip
- python3-venv
- libtbb-dev
- nlohmann-json3-dev
- libmsgpack-dev
- libsqlite3-dev
- libnuma-dev
- libprotobuf-dev
- libsqlite3-dev
- libtbb-dev
- ninja-build
- nlohmann-json3-dev
- protobuf-compiler
- python3-pip
- python3-pybind11
- python3-venv
- name: pipModules
type: object
default:
- https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
- onnx==1.14.1
- numpy==1.21.6
- typing==3.7.4
- pytest==6.0.1
- packaging==23.0
- protobuf==3.20.2
- onnx>=1.14.1
- numpy>=1.21.6
- typing>=3.7.4
- pytest>=6.0.1
- packaging>=23.0
- protobuf>=3.20.2
- name: rocmDependencies
type: object
default:
@@ -50,6 +51,26 @@ parameters:
- rocminfo
- rocMLIR
- rocprofiler-register
- roctracer
- name: rocmTestDependencies
type: object
default:
- aomp
- aomp-extras
- clr
- composable_kernel
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- MIOpen
- rocm-cmake
- ROCR-Runtime
- rocBLAS
- rocminfo
- rocMLIR
- rocprofiler-register
- roctracer
jobs:
- job: AMDMIGraphX
@@ -63,8 +84,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -95,17 +114,84 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DMIGRAPHX_USE_COMPOSABLEKERNEL=OFF
-DBUILD_TESTING=ON
-GNinja
# REFERENCE: https://github.com/ROCm/composable_kernel/issues/782
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: AMDMIGraphX_testing
dependsOn: AMDMIGraphX
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: MIGRAPHX_TRACE_BENCHMARKING
value: 1
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# half version should be fixed to 5.6.0
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencySource: fixed
fixedComponentName: half
fixedPipelineIdentifier: ${{ variables.HALF560_PIPELINE_ID }}
skipLibraryLinking: true
skipLlvmSymlink: true
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: CMake@1
displayName: MIGraphXTest CMake Flags
inputs:
cmakeArgs: >-
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm/llvm;$(Agent.BuildDirectory)/rocm
-DHALF_INCLUDE_DIR=$(Agent.BuildDirectory)/rocm/include
-DBUILD_TESTING=ON
-DMIGRAPHX_ENABLE_C_API_TEST=ON
..
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Build and run MIGraphX tests
inputs:
targetType: inline
workingDirectory: build
script: make -j$(nproc) check

159
.azuredevops/components/MIOpen.yml
View File

@@ -8,15 +8,24 @@ parameters:
- name: aptPackages
type: object
default:
- software-properties-common
- python3-pip
- cmake
- ninja-build
- libsqlite3-dev
- libbz2-dev
- nlohmann-json3-dev
- libgtest-dev
- libdrm-dev
- libeigen3-dev
- libgmock-dev
- libgtest-dev
- libsqlite3-dev
- libstdc++-12-dev
- libzstd-dev
- ninja-build
- nlohmann-json3-dev
- python3-pip
- software-properties-common
- zstd
- name: pipModules
type: object
default:
- cget
- name: rocmDependencies
type: object
default:
@@ -35,6 +44,24 @@ parameters:
- clr
- rocminfo
- roctracer
- name: rocmTestDependencies
type: object
default:
- clr
- composable_kernel
- half
- hipBLAS
- hipBLAS-common
- hipBLASLt
- llvm-project
- rocBLAS
- rocm-cmake
- rocminfo
- rocMLIR
- ROCR-Runtime
- rocprofiler-register
- rocRAND
- roctracer
jobs:
- job: MIOpen
@@ -48,28 +75,17 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
# The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+)
# Upgrade boost from apt by following https://launchpad.net/~mhier/+archive/ubuntu/libboost-latest
- task: Bash@3
displayName: 'update boost version'
inputs:
targetType: inline
script: sudo add-apt-repository ppa:mhier/libboost-latest -y
- task: Bash@3
displayName: 'install boost'
inputs:
targetType: inline
script: sudo apt-get --yes install libboost1.83-dev libboost-system1.83-dev libboost-filesystem1.83-dev
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-boost.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
@@ -85,7 +101,8 @@ jobs:
extraBuildFlags: >-
-DMIOPEN_BACKEND=HIP
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Agent.BuildDirectory)/boost
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DMIOPEN_ENABLE_AI_KERNEL_TUNING=OFF
-DMIOPEN_ENABLE_AI_IMMED_MODE_FALLBACK=OFF
-DCMAKE_BUILD_TYPE=Release
@@ -94,3 +111,103 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: MIOpen_testing
dependsOn: MIOpen
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
# The default boost library from apt is 1.74, which does not satisfy MIOpen's build requirement (1.79+)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-boost.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# MIOpen depends on a specific version of frugally-deep which is forked here: https://github.com/ROCm/frugally-deep
# https://github.com/ROCm/frugally-deep/blob/master/INSTALL.md
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- task: Bash@3
displayName: Install FunctionalPlus
inputs:
targetType: inline
script: cget install Dobiasd/FunctionalPlus
- task: Bash@3
displayName: Remove Python site-packages binaries from path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: git clone frugally-deep
inputs:
targetType: inline
script: git clone https://github.com/ROCm/frugally-deep --depth=1 --shallow-submodules
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
componentName: frugally-deep
cmakeBuildDir: $(Build.SourcesDirectory)/frugally-deep/build
installDir: $(Build.SourcesDirectory)/bin
extraBuildFlags: -DCMAKE_PREFIX_PATH=$(Build.SourcesDirectory)/cget/cget/pkg/Dobiasd__FunctionalPlus/install
- task: CMake@1
displayName: 'MIOpen Test CMake Flags'
inputs:
cmakeArgs: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm;$(Build.SourcesDirectory)/bin;$(Build.SourcesDirectory)/cget/cget/pkg/Dobiasd__FunctionalPlus/install;$(Agent.BuildDirectory)/boost
-DCMAKE_INSTALL_PREFIX=$(Agent.BuildDirectory)/rocm
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DMIOPEN_BACKEND=HIP
-DMIOPEN_TEST_FLAGS=" --disable-verification-cache"
-DCMAKE_BUILD_TYPE=release
-DBUILD_DEV=OFF
-DMIOPEN_USE_MLIR=ON
-DMIOPEN_GPU_SYNC=OFF
..
- task: Bash@3
displayName: 'MIOpen Test Build'
inputs:
targetType: inline
script: |
cmake --build . --target tests -- -j$(nproc)
workingDirectory: $(Build.SourcesDirectory)/build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: MIOpen
reloadAMDGPU: true

5
.azuredevops/components/MIVisionX.yml
View File

@@ -86,8 +86,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -120,7 +118,7 @@ jobs:
- job: MIVisionX_testing
dependsOn: MIVisionX
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -162,6 +160,7 @@ jobs:
mkdir mivisionx-tests
cd mivisionx-tests
cmake /opt/rocm/share/mivisionx/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: MIVisionX

128
.azuredevops/components/ROCR-Runtime.yml
View File

@@ -8,15 +8,21 @@ parameters:
- name: aptPackages
type: object
default:
- libelf-dev
- g++
- libdrm-dev
- libelf-dev
- libnuma-dev
- name: rocmDependencies
type: object
default:
- llvm-project
- rocprofiler-register
- name: rocmTestDependencies
type: object
default:
- llvm-project
- rocm_smi_lib
- rocprofiler-register
jobs:
- job: ROCR_Runtime
@@ -50,3 +56,123 @@ jobs:
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DBUILD_SHARED_LIBS=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- job: ROCR_Runtime_testing
dependsOn: ROCR_Runtime
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- task: Bash@3
displayName: Install libhwloc5
inputs:
targetType: 'inline'
script: |
wget http://ftp.us.debian.org/debian/pool/main/h/hwloc/libhwloc5_1.11.12-3_amd64.deb
wget http://ftp.us.debian.org/debian/pool/main/h/hwloc/libhwloc-dev_1.11.12-3_amd64.deb
sudo apt install -y --allow-downgrades ./libhwloc5_1.11.12-3_amd64.deb ./libhwloc-dev_1.11.12-3_amd64.deb
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- task: Bash@3
displayName: Build kfdtest
inputs:
targetType: 'inline'
workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/kfdtest
script: |
mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm ..
make
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: kfdtest
testExecutable: BIN_DIR=$(Build.SourcesDirectory)/libhsakmt/tests/kfdtest/build ./run_kfdtest.sh
testParameters: '-p core --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Build.SourcesDirectory)/libhsakmt/tests/kfdtest/scripts
- task: Bash@3
displayName: Build rdmatest app
inputs:
targetType: 'inline'
workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/app
script: |
cmake -DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm -DLIBHSAKMT_PATH=$(Agent.BuildDirectory)/rocm -DDRM_AMDGPU_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include .
cmake --build .
- task: Bash@3
displayName: Build rdmatest driver
inputs:
targetType: 'inline'
workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/drv
script: |
sed -i 's/HSAKMT_PAGE_SHIFT/PAGE_SHIFT/g' amdp2ptest.c
sed -i 's/"MIT"/"GPL"/' amdp2ptest.c
RDMA_HEADER_DIR=/usr/src/amdgpu-*/include make all
- task: Bash@3
displayName: Install rdmatest driver
inputs:
targetType: 'inline'
workingDirectory: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/drv
script: |
sudo rmmod amdp2ptest.ko
sudo insmod amdp2ptest.ko
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rdmatest
testExecutable: yes | ./rdma_test
testParameters: ''
testDir: $(Build.SourcesDirectory)/libhsakmt/tests/rdma/simple/app
testPublishResults: false
- task: Bash@3
displayName: Build rocrtst
inputs:
targetType: 'inline'
workingDirectory: $(Build.SourcesDirectory)/rocrtst/suites/test_common
script: |
sudo rm -rf $(Agent.BuildDirectory)/external/llvm-project
mkdir -p $(Agent.BuildDirectory)/external/llvm-project/clang/lib
sudo ln -s $(Agent.BuildDirectory)/rocm/llvm/lib/clang/20/include $(Agent.BuildDirectory)/external/llvm-project/clang/lib/Headers
mkdir build && cd build
cmake .. \
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm \
-DTARGET_DEVICES=$(JOB_GPU_TARGET) \
-DROCM_DIR=$(Agent.BuildDirectory)/rocm \
-DLLVM_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin \
-DOPENCL_DIR=$(Agent.BuildDirectory)/rocm/llvm/bin
make
make rocrtst_kernels
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocrtst
testExecutable: ./rocrtst64
testParameters: '--gtest_filter="-rocrtstNeg.Memory_Negative_Tests" --gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Build.SourcesDirectory)/rocrtst/suites/test_common/build/$(JOB_GPU_TARGET)

94
.azuredevops/components/ROCgdb.yml
View File

@@ -5,24 +5,47 @@ parameters:
- name: checkoutRef
type: string
default: ''
# reference: https://github.com/ROCm/ROCgdb/blob/amd-staging/README-ROCM.md
- name: aptPackages
type: object
default:
- libgmp-dev
- libmpfr-dev
- texinfo
- bison
- dejagnu
- flex
- libbabeltrace-dev
- libexpat-dev
- libgmp-dev
- liblzma-dev
- libmpfr-dev
- ncurses-dev
- texinfo
- zlib1g-dev
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- ROCdbgapi
- rocminfo
- rocprofiler-register
- ROCR-Runtime
jobs:
- job: rocgdb
- job: ROCgdb
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
- name: PKG_CONFIG_PATH
value: $(Agent.BuildDirectory)/rocm/share/pkgconfig
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -31,5 +54,64 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-autotools.yml
parameters:
configureFlags: >-
--program-prefix=roc
--enable-64-bit-bfd
--enable-targets="x86_64-linux-gnu,amdgcn-amd-amdhsa"
--disable-ld
--disable-gas
--disable-gdbserver
--disable-sim
--enable-tui
--disable-gdbtk
--disable-shared
--disable-gprofng
--with-expat
--with-system-zlib
--without-guile
--with-babeltrace
--with-lzma
--with-python=python3
--with-rocm-dbgapi=$(Agent.BuildDirectory)/rocm
LDFLAGS="-Wl,--enable-new-dtags,-rpath=$(Agent.BuildDirectory)/rocm/lib"
makeCallPrefix: LD_RUN_PATH='${ORIGIN}/../lib'
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
echo "##vso[task.prependpath]/opt/rocm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: check-gdb
continueOnError: true
inputs:
targetType: inline
script: make check-gdb TESTS=gdb.rocm/simple.exp
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: print gdb log
inputs:
targetType: inline
script: find -name gdb.log -exec cat {} \;
workingDirectory: $(Build.SourcesDirectory)

7
.azuredevops/components/ROCmValidationSuite.yml
View File

@@ -40,9 +40,11 @@ parameters:
- llvm-project
- rocBLAS
- rocm_smi_lib
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rocRAND
- roctracer
jobs:
- job: ROCmValidationSuite
@@ -63,8 +65,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -97,7 +97,7 @@ jobs:
- job: ROCmValidationSuite_testing
dependsOn: ROCmValidationSuite
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -131,6 +131,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: ROCmValidationSuite

169
.azuredevops/components/Tensile.yml Normal file
View File

@@ -0,0 +1,169 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- python3-pip
- cmake
- libmsgpack-dev
- libboost-program-options-dev
- name: pipModules
type: object
default:
- tox
- pytest
- name: rocmDependencies
type: object
default:
- aomp
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- rocm_smi_lib
- rocprofiler-register
- ROCR-Runtime
jobs:
- job: Tensile
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: python3 setup.py bdist_wheel
workingDirectory: $(Build.SourcesDirectory)
env:
ROCM_PATH: $(Agent.BuildDirectory)/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Build.SourcesDirectory)/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- job: Tensile_testing
dependsOn: Tensile
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
inputs:
itemPattern: '**/*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: pip install
inputs:
targetType: inline
script: find -name *.whl -exec pip install {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: tox test
inputs:
targetType: inline
script: tox run -v -e ci -- -m pre_checkin
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Remove Python site-packages binaries from path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm compilers from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')"

5
.azuredevops/components/amdsmi.yml
View File

@@ -35,7 +35,7 @@ jobs:
- job: amdsmi_testing
dependsOn: amdsmi
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -53,6 +53,9 @@ jobs:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: amdsmi

3
.azuredevops/components/aomp.yml
View File

@@ -413,7 +413,7 @@ jobs:
- job: aomp_testing
dependsOn: aomp
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -472,6 +472,7 @@ jobs:
Contents: FileCheck
TargetFolder: $(Agent.BuildDirectory)/rocm/share/openmp-extras/tests/bin
retryCount: 3
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Test AOMP
continueOnError: true

7
.azuredevops/components/composable_kernel.yml
View File

@@ -48,8 +48,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -90,6 +88,8 @@ jobs:
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_HIP_FLAGS="-Wno-missing-include-dirs"
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCK_BUILD_JIT_LIB=ON
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DCMAKE_BUILD_TYPE=Release
-DGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja
@@ -99,7 +99,7 @@ jobs:
- job: composable_kernel_testing
dependsOn: composable_kernel
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -135,6 +135,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Iterate through test scripts
inputs:

2
.azuredevops/components/copyHIP.yml
View File

@@ -25,7 +25,7 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-download.yml
parameters:
componentName: HIP
pipelineId: $(hip-pipeline-id)
pipelineId: $(HIP_PIPELINE_ID)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Agent.BuildDirectory)/rocm

8
.azuredevops/components/hip-tests.yml
View File

@@ -52,8 +52,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -89,9 +87,9 @@ jobs:
gpuTarget: $(JOB_GPU_TARGET)
- job: hip_tests_testing
timeoutInMinutes: 180
timeoutInMinutes: 240
dependsOn: hip_tests
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -130,8 +128,10 @@ jobs:
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo mkdir -p /opt/rocm/bin
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator /opt/rocm/bin/rocm_agent_enumerator
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hip_tests

2
.azuredevops/components/hipBLAS-common.yml
View File

@@ -29,7 +29,7 @@ jobs:
- name: ROCM_PATH
value: $(Agent.BuildDirectory)/rocm
- template: /.azuredevops/variables-global.yml
pool:
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all

7
.azuredevops/components/hipBLAS.yml
View File

@@ -16,7 +16,6 @@ parameters:
- libgtest-dev
- wget
- python3-pip
- libomp-dev
- name: rocmDependencies
type: object
default:
@@ -46,6 +45,7 @@ parameters:
- rocBLAS
- rocSPARSE
- rocSOLVER
- roctracer
jobs:
- job: hipBLAS
@@ -59,8 +59,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -98,7 +96,7 @@ jobs:
- job: hipBLAS_testing
dependsOn: hipBLAS
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -132,6 +130,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipBLAS

10
.azuredevops/components/hipBLASLt.yml
View File

@@ -16,13 +16,13 @@ parameters:
- python3-pip
- python3-venv
- gfortran
- libomp-dev
- libblas-dev
- ccache
- name: pipModules
type: object
default:
- joblib
- packaging
- name: rocmDependencies
type: object
default:
@@ -66,6 +66,8 @@ jobs:
value: $(Agent.BuildDirectory)/rocm/bin/hipcc
- name: PATH
value: $(Agent.BuildDirectory)/rocm/llvm/bin:$(Agent.BuildDirectory)/rocm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
- name: DAY_STRING
value: $[format('{0:ddMMyyyy}', pipeline.startTime)]
pool: ${{ variables.ULTRA_BUILD_POOL }}
workspace:
clean: all
@@ -73,8 +75,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -149,7 +149,7 @@ jobs:
- job: hipBLASLt_testing
dependsOn: hipBLASLt
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -165,6 +165,7 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
@@ -183,6 +184,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipBLASLt

5
.azuredevops/components/hipCUB.yml
View File

@@ -43,8 +43,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -78,7 +76,7 @@ jobs:
- job: hipCUB_testing
dependsOn: hipCUB
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -112,6 +110,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipCUB

7
.azuredevops/components/hipFFT.yml
View File

@@ -14,11 +14,11 @@ parameters:
- libgtest-dev
- libfftw3-dev
- python3-pip
- libomp-14-dev
# rocm dependencies should match dependencies-rocm.yml
- name: rocmDependencies
type: object
default:
- aomp
- rocRAND
- hipRAND
- llvm-project
@@ -55,8 +55,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -96,7 +94,7 @@ jobs:
- job: hipFFT_testing
dependsOn: hipFFT
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -130,6 +128,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipFFT

5
.azuredevops/components/hipRAND.yml
View File

@@ -45,8 +45,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -82,7 +80,7 @@ jobs:
- job: hipRAND_testing
dependsOn: hipRAND
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -116,6 +114,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipRAND

6
.azuredevops/components/hipSOLVER.yml
View File

@@ -42,6 +42,7 @@ parameters:
- ROCR-Runtime
- rocSOLVER
- rocSPARSE
- roctracer
jobs:
- job: hipSOLVER
@@ -56,8 +57,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -102,7 +101,7 @@ jobs:
- job: hipSOLVER_testing
dependsOn: hipSOLVER
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -136,6 +135,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipSOLVER

6
.azuredevops/components/hipSPARSE.yml
View File

@@ -37,6 +37,7 @@ parameters:
- rocprofiler-register
- ROCR-Runtime
- rocSPARSE
- roctracer
jobs:
- job: hipSPARSE
@@ -51,8 +52,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -96,7 +95,7 @@ jobs:
- job: hipSPARSE_testing
dependsOn: hipSPARSE
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -130,6 +129,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipSPARSE

4
.azuredevops/components/hipSPARSELt.yml
View File

@@ -15,7 +15,6 @@ parameters:
- python3-pip
- gfortran
- libgfortran5
- libomp-dev
- libopenblas-dev
- name: pipModules
type: object
@@ -124,7 +123,7 @@ jobs:
- job: hipSPARSELt_testing
dependsOn: hipSPARSELt
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -152,6 +151,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipSPARSELt

7
.azuredevops/components/hipTensor.yml
View File

@@ -42,8 +42,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -79,7 +77,7 @@ jobs:
- job: hipTensor_testing
timeoutInMinutes: 90
dependsOn: hipTensor
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -113,8 +111,9 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: hipTensor
testDir: '$(Agent.BuildDirectory)/rocm/bin/hiptensor'
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
testParameters: '-E ".*-extended" -VV --output-on-failure --force-new-ctest-process --output-junit test_output.xml'

5
.azuredevops/components/hipfort.yml
View File

@@ -51,8 +51,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -94,7 +92,7 @@ jobs:
- job: hipfort_testing
dependsOn: hipfort
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -139,6 +137,7 @@ jobs:
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: 'Test hipfort'
inputs:

166
.azuredevops/components/omniperf.yml Normal file
View File

@@ -0,0 +1,166 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- python3-pip
- name: pipModules
type: object
default:
- astunparse==1.6.2
- colorlover
- dash>=1.12.0
- matplotlib
- numpy>=1.17.5
- pandas>=1.4.3
- pymongo
- pyyaml
- tabulate
- tqdm
- dash-svg
- dash-bootstrap-components
- kaleido
- setuptools
- plotille
- mock
- pytest
- pytest-cov
- pytest-xdist
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler
- rocprofiler-register
- roctracer
jobs:
- job: omniperf
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: omniperf_testing
dependsOn: omniperf
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTHON_VERSION
value: 3.10
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET)
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DCMAKE_BUILD_TYPE=Release
-DENABLE_TESTS=ON
-DINSTALL_TESTS=ON
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: omniperf
testDir: $(Build.BinariesDirectory)/libexec/omniperf
testExecutable: export OMNIPERF_ARCH_OVERRIDE="MI300X"; ctest
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm compilers from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')"

139
.azuredevops/components/omnitrace.yml Normal file
View File

@@ -0,0 +1,139 @@
# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- autoconf
- autotools-dev
- bison
- build-essential
- bzip2
- clang
- cmake
- environment-modules
- g++-12
- libdrm-dev
- libfabric-dev
- libiberty-dev
- libpapi-dev
- libpfm4-dev
- libtool
- libopenmpi-dev
- m4
- openmpi-bin
- software-properties-common
- python3-pip
- texinfo
- zlib1g-dev
- name: pipModules
type: object
default:
- numpy
- perfetto
- dataclasses
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rccl
- rocm-core
- rocm_smi_lib
- rocminfo
- ROCR-Runtime
- rocprofiler
- rocprofiler-register
- roctracer
jobs:
- job: omnitrace
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: ROCm symbolic link
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DOMNITRACE_BUILD_TESTING=ON
-DOMNITRACE_BUILD_DYNINST=ON
-DOMNITRACE_BUILD_LIBUNWIND=ON
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DOMNITRACE_USE_PAPI=ON
-DOMNITRACE_USE_MPI=ON
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
multithreadFlag: -- -j32
- task: Bash@3
displayName: Set up omnitrace env
inputs:
targetType: inline
script: source share/omnitrace/setup-env.sh
workingDirectory: build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: omnitrace
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm compilers from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)

5
.azuredevops/components/rccl.yml
View File

@@ -60,8 +60,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -102,7 +100,7 @@ jobs:
- job: rccl_testing
timeoutInMinutes: 120
dependsOn: rccl
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -136,6 +134,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rccl

7
.azuredevops/components/rdc.yml
View File

@@ -60,8 +60,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -113,7 +111,7 @@ jobs:
- job: rdc_testing
dependsOn: rdc
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -131,6 +129,8 @@ jobs:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
@@ -156,6 +156,7 @@ jobs:
sudo ln -s $(Agent.BuildDirectory)/rocm/bin/rdcd /usr/sbin/rdcd
echo $(Agent.BuildDirectory)/rocm/lib/rdc/grpc/lib | sudo tee /etc/ld.so.conf.d/grpc.conf
sudo ldconfig -v
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- task: Bash@3
displayName: Test rdc
inputs:

6
.azuredevops/components/rocAL.yml
View File

@@ -53,6 +53,7 @@ parameters:
- half
- llvm-project
- MIVisionX
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- rpp
@@ -70,8 +71,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
@@ -157,7 +156,7 @@ jobs:
- job: rocAL_testing
dependsOn: rocAL
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -220,6 +219,7 @@ jobs:
mkdir rocAL-tests
cd rocAL-tests
cmake $(Agent.BuildDirectory)/rocm/share/rocal/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocAL

6
.azuredevops/components/rocALUTION.yml
View File

@@ -43,6 +43,7 @@ parameters:
- ROCR-Runtime
- rocRAND
- rocSPARSE
- roctracer
jobs:
- job: rocALUTION
@@ -59,8 +60,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -97,7 +96,7 @@ jobs:
- job: rocALUTION_testing
dependsOn: rocALUTION
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -131,6 +130,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocALUTION

6
.azuredevops/components/rocBLAS.yml
View File

@@ -52,6 +52,7 @@ parameters:
- rocprofiler-register
- rocm_smi_lib
- ROCR-Runtime
- roctracer
jobs:
- job: rocBLAS
@@ -73,8 +74,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -120,7 +119,7 @@ jobs:
- job: rocBLAS_testing
dependsOn: rocBLAS
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -155,6 +154,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocBLAS

6
.azuredevops/components/rocDecode.yml
View File

@@ -17,7 +17,7 @@ parameters:
- libavformat-dev
- libavutil-dev
- libstdc++-12-dev
- libva-dev
- libva-amdgpu-dev
- mesa-amdgpu-va-drivers
- libdrm-dev
- name: rocmDependencies
@@ -35,6 +35,7 @@ parameters:
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
@@ -88,7 +89,7 @@ jobs:
- job: rocDecode_testing
dependsOn: rocDecode
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -128,6 +129,7 @@ jobs:
mkdir rocDecode-tests
cd rocDecode-tests
cmake /opt/rocm/share/rocdecode/test
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocDecode

5
.azuredevops/components/rocFFT.yml
View File

@@ -55,8 +55,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -95,7 +93,7 @@ jobs:
- job: rocFFT_testing
dependsOn: rocFFT
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -129,6 +127,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocFFT

13
.azuredevops/components/rocMLIR.yml
View File

@@ -13,6 +13,7 @@ parameters:
- git
- python3-pip
- libdrm-dev
- libstdc++-12-dev
- name: pipModules
type: object
default:
@@ -28,7 +29,7 @@ parameters:
- ROCR-Runtime
jobs:
- job: rocMLIR_library
- job: rocMLIR
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -66,16 +67,19 @@ jobs:
# compiling and running test on the test system together
- job: rocMLIR_testing
dependsOn: rocMLIR
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.GFX942_TEST_POOL }}
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -105,7 +109,9 @@ jobs:
displayName: ROCm symbolic link
inputs:
targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
installEnabled: false
@@ -117,6 +123,7 @@ jobs:
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DROCM_TEST_CHIPSET=$(JOB_GPU_TARGET)
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocMLIR

5
.azuredevops/components/rocPRIM.yml
View File

@@ -42,8 +42,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -77,7 +75,7 @@ jobs:
- job: rocPRIM_testing
dependsOn: rocPRIM
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -111,6 +109,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocPRIM

92
.azuredevops/components/rocPyDecode.yml
View File

@@ -52,8 +52,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -73,6 +71,13 @@ jobs:
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: 'ROCm symbolic link'
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
@@ -85,3 +90,86 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
publish: false
- task: Bash@3
displayName: Create wheel file
inputs:
targetType: inline
script: |
export ROCM_PATH=$(Agent.BuildDirectory)/rocm
export HIP_INCLUDE_DIRS=$(Agent.BuildDirectory)/rocm/include/hip
python3 setup.py bdist_wheel
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: $(Build.SourcesDirectory)/dist
contentsString: '*.whl'
targetDir: $(Build.ArtifactStagingDirectory)
clean: false
- task: PublishPipelineArtifact@1
displayName: 'wheel file Publish'
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- job: rocPyDecode_testing
dependsOn: rocPyDecode
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
cd $(Build.SourcesDirectory)
sudo pip install .
cmake -DAMDGPU_TARGETS=$(JOB_GPU_TARGET) .
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocPyDecode
testDir: $(Build.SourcesDirectory)
# sudo required for pip install but screws up permissions for next pipeline run
- task: Bash@3
displayName: Clean up test environment
inputs:
targetType: inline
script: |
sudo rm -rf $(Build.SourcesDirectory)/*

5
.azuredevops/components/rocRAND.yml
View File

@@ -45,8 +45,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -79,7 +77,7 @@ jobs:
- job: rocRAND_testing
dependsOn: rocRAND
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -113,6 +111,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocRAND

6
.azuredevops/components/rocSOLVER.yml
View File

@@ -44,6 +44,7 @@ parameters:
- rocprofiler-register
- ROCR-Runtime
- rocSPARSE
- roctracer
jobs:
- job: rocSOLVER
@@ -57,8 +58,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -112,7 +111,7 @@ jobs:
- job: rocSOLVER_testing
dependsOn: rocSOLVER
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -146,6 +145,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocSOLVER

6
.azuredevops/components/rocSPARSE.yml
View File

@@ -40,6 +40,7 @@ parameters:
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
jobs:
- job: rocSPARSE
@@ -55,8 +56,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -106,7 +105,7 @@ jobs:
- job: rocSPARSE_testing
timeoutInMinutes: 90
dependsOn: rocSPARSE
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -140,6 +139,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocSPARSE

5
.azuredevops/components/rocThrust.yml
View File

@@ -47,8 +47,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -82,7 +80,7 @@ jobs:
- job: rocThrust_testing
dependsOn: rocThrust
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -116,6 +114,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocThrust

8
.azuredevops/components/rocWMMA.yml
View File

@@ -16,10 +16,10 @@ parameters:
- googletest
- libfftw3-dev
- git
- libomp-dev
- name: rocmDependencies
type: object
default:
- aomp
- rocm-cmake
- llvm-project
- ROCR-Runtime
@@ -42,6 +42,7 @@ parameters:
- rocminfo
- rocprofiler-register
- ROCR-Runtime
- roctracer
jobs:
- job: rocWMMA
@@ -55,8 +56,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -93,7 +92,7 @@ jobs:
- job: rocWMMA_testing
timeoutInMinutes: 90
dependsOn: rocWMMA
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -127,6 +126,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocWMMA

42
.azuredevops/components/rocm-examples.yml
View File

@@ -60,6 +60,7 @@ parameters:
- rocSOLVER
- rocSPARSE
- rocThrust
- roctracer
jobs:
- job: rocm_examples
@@ -73,8 +74,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -116,7 +115,7 @@ jobs:
- job: rocm_examples_testing
dependsOn: rocm_examples
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -135,9 +134,9 @@ jobs:
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
@@ -152,20 +151,19 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: Unload and reload AMDGPU
inputs:
targetType: inline
script: |
sudo modprobe -r amdgpu
sudo modprobe amdgpu
- task: Bash@3
displayName: Iterate through examples
inputs:
targetType: inline
script: |
for file in *; do
echo Now running: $file
./$file | tee -a $(TEST_LOG_FILE)
done
workingDirectory: $(Agent.BuildDirectory)/rocm/examples
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
# https://github.com/ROCm/HIP/issues/2203
extraBuildFlags: >-
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCM_ROOT=$(Agent.BuildDirectory)/rocm
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-DCMAKE_HIP_ARCHITECTURES=$(JOB_GPU_TARGET)
-DCMAKE_EXE_LINKER_FLAGS=-fgpu-rdc
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm-examples
testDir: $(Build.SourcesDirectory)/build
reloadAMDGPU: true

4
.azuredevops/components/rocm_bandwidth_test.yml
View File

@@ -27,6 +27,7 @@ parameters:
- name: rocmTestDependencies
type: object
default:
- rocminfo
- rocprofiler-register
- ROCR-Runtime
@@ -72,7 +73,7 @@ jobs:
- job: rocm_bandwidth_test_testing
dependsOn: rocm_bandwidth_test
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -99,6 +100,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm_bandwidth_test

5
.azuredevops/components/rocm_smi_lib.yml
View File

@@ -29,7 +29,7 @@ jobs:
- job: rocm_smi_lib_testing
dependsOn: rocm_smi_lib
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -44,6 +44,9 @@ jobs:
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocm_smi_lib

5
.azuredevops/components/rocminfo.yml
View File

@@ -49,7 +49,7 @@ jobs:
- job: rocminfo_testing
dependsOn: rocminfo
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -72,6 +72,9 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
parameters:
runRocminfo: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocminfo

93
.azuredevops/components/rocprofiler-sdk.yml Normal file
View File

@@ -0,0 +1,93 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- cmake
- python3-pip
- libdrm-dev
- libdw-dev
- libelf-dev
- name: pipModules
type: object
default:
- black
- clang-format
- clang-tidy
- cmake
- cmake-format
- dataclasses
- numpy
- otf2
- pandas
- perfetto
- pycobertura
- pytest
- pyyaml
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rccl
- rocm-cmake
- rocm-core
- rocminfo
- ROCR-Runtime
- rocprofiler-register
- roctracer
jobs:
- job: rocprofilersdk
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFILER_BUILD_TESTS=ON
-DROCPROFILER_BUILD_SAMPLES=OFF
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
multithreadFlag: -- -j2
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)

139
.azuredevops/components/rocprofiler-systems.yml Normal file
View File

@@ -0,0 +1,139 @@
# largely referenced from: https://github.com/ROCm/omnitrace/blob/main/.github/workflows/ubuntu-jammy.yml
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: aptPackages
type: object
default:
- autoconf
- autotools-dev
- bison
- build-essential
- bzip2
- clang
- cmake
- environment-modules
- g++-12
- libdrm-dev
- libfabric-dev
- libiberty-dev
- libpapi-dev
- libpfm4-dev
- libtool
- libopenmpi-dev
- m4
- openmpi-bin
- software-properties-common
- python3-pip
- texinfo
- zlib1g-dev
- name: pipModules
type: object
default:
- numpy
- perfetto
- dataclasses
- name: rocmDependencies
type: object
default:
- clr
- llvm-project
- rccl
- rocm-core
- rocm_smi_lib
- rocminfo
- ROCR-Runtime
- rocprofiler
- rocprofiler-register
- roctracer
jobs:
- job: rocprofiler_systems
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: ${{ variables.MEDIUM_BUILD_POOL }}
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/checkout.yml
parameters:
checkoutRepo: ${{ parameters.checkoutRepo }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
# CI case: download latest default branch build
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
# manual build case: triggered by ROCm/ROCm repo
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: ROCm symbolic link
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/build-cmake.yml
parameters:
# build flags reference: https://rocm.docs.amd.com/projects/omnitrace/en/latest/install/install.html
extraBuildFlags: >-
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DROCPROFSYS_BUILD_TESTING=ON
-DROCPROFSYS_BUILD_DYNINST=ON
-DROCPROFSYS_BUILD_LIBUNWIND=ON
-DDYNINST_BUILD_TBB=ON
-DDYNINST_BUILD_ELFUTILS=ON
-DDYNINST_BUILD_LIBIBERTY=ON
-DDYNINST_BUILD_BOOST=ON
-DROCPROFSYS_USE_PAPI=ON
-DROCPROFSYS_USE_MPI=ON
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
multithreadFlag: -- -j32
- task: Bash@3
displayName: Set up rocprofiler-systems env
inputs:
targetType: inline
script: source share/rocprofiler-systems/setup-env.sh
workingDirectory: build
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocprofiler-systems
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm compilers from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/llvm/bin;;' -e 's;^/;;' -e 's;/$;;')"
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)

62
.azuredevops/components/rocprofiler.yml
View File

@@ -57,8 +57,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -92,6 +90,66 @@ jobs:
-DENABLE_LDCONFIG=OFF
-DUSE_PROF_API=1
-DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
multithreadFlag: -- -j32
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: rocprofiler_testing
dependsOn: rocprofiler
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- task: Bash@3
displayName: Setup test environment
inputs:
targetType: inline
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocprofilerV1
testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1/test" ./run.sh
testDir: $(Agent.BuildDirectory)/rocm/share/rocprofiler/tests-v1
testParameters: ''
testPublishResults: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocprofilerV2
testExecutable: LD_LIBRARY_PATH="$(Agent.BuildDirectory)/rocm/lib/rocprofiler:$(Agent.BuildDirectory)/rocm/share/rocprofiler/tests" share/rocprofiler/tests/runUnitTests
testParameters: '--gtest_output=xml:./test_output.xml --gtest_color=yes'
testDir: $(Agent.BuildDirectory)/rocm

3
.azuredevops/components/rocr_debug_agent.yml
View File

@@ -72,7 +72,7 @@ jobs:
- job: rocr_debug_agent_testing
dependsOn: rocr_debug_agent
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -114,6 +114,7 @@ jobs:
cmakeBuildDir: '$(Agent.BuildDirectory)/rocm/src/rocm-debug-agent-test'
cmakeSourceDir: '.'
installEnabled: false
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rocr_debug_agent

7
.azuredevops/components/roctracer.yml
View File

@@ -30,6 +30,7 @@ parameters:
default:
- clr
- llvm-project
- rocminfo
- rocprofiler-register
- ROCR-Runtime
@@ -48,8 +49,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -77,6 +76,7 @@ jobs:
-DCMAKE_MODULE_PATH=$(Agent.BuildDirectory)/rocm/lib/cmake/hip
-DCMAKE_PREFIX_PATH=$(Agent.BuildDirectory)/rocm
-DGPU_TARGETS=$(JOB_GPU_TARGET)
-DAMDGPU_TARGETS=$(JOB_GPU_TARGET)
-GNinja
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
@@ -84,7 +84,7 @@ jobs:
- job: roctracer_testing
dependsOn: roctracer
condition: succeeded()
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
@@ -118,6 +118,7 @@ jobs:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: roctracer

117
.azuredevops/components/rpp.yml
View File

@@ -9,18 +9,39 @@ parameters:
type: object
default:
- cmake
- libomp-dev # needed to pass flag step
- libopencv-dev
- libsndfile1-dev
- libstdc++-12-dev
- imagemagick
- ninja-build
- clang
- name: pipModules
type: object
default:
- openpyxl
- pandas
- sphinx
- name: rocmDependencies
type: object
default:
- aomp # needed to pass build step
- aomp
- clr
- half
- llvm-project
- rocminfo
- ROCR-Runtime
- name: rocmTestDependencies
type: object
default:
- aomp
- clr
- half
- hipTensor
- llvm-project
- rocm-cmake
- rocminfo
- rocprofiler-register
- ROCR-Runtime
jobs:
- job: rpp
@@ -35,8 +56,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
@@ -69,3 +88,93 @@ jobs:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- job: rpp_testing
dependsOn: rpp
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: LD_LIBRARY_PATH
value: $(Agent.BuildDirectory)/rocm/lib;$(Agent.BuildDirectory)/rocm/llvm/lib
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/local-artifact-download.yml
parameters:
gpuTarget: $(JOB_GPU_TARGET)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
${{ if eq(parameters.checkoutRef, '') }}:
dependencySource: staging
${{ elseif ne(parameters.checkoutRef, '') }}:
dependencySource: tag-builds
# Dependencies from: https://github.com/ROCm/rpp/blob/develop/utilities/test_suite/README.md
- task: Bash@3
displayName: Build and install Turbo JPEG
inputs:
targetType: 'inline'
script: |
sudo apt-get install nasm
sudo apt-get install wget
git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git
cd libjpeg-turbo
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_BUILD_TYPE=RELEASE \
-DENABLE_STATIC=FALSE \
-DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \
-DWITH_JPEG8=TRUE \
..
make -j$nproc
sudo make install
- task: Bash@3
displayName: Build and install Nifti
inputs:
targetType: 'inline'
script: |
git clone https://github.com/NIFTI-Imaging/nifti_clib.git
cd nifti_clib
mkdir build
cd build
cmake ..
sudo make -j$nproc install
- task: Bash@3
displayName: Build rpp tests
inputs:
targetType: 'inline'
script: |
sudo rm -rf /opt/rocm
sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
mkdir rpp-tests
cd rpp-tests
cmake /opt/rocm/share/rpp/test \
-DCMAKE_CXX_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang++ \
-DCMAKE_C_COMPILER=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/gpu-diagnostics.yml
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/test.yml
parameters:
componentName: rpp
testExecutable: 'export PATH=$(Agent.BuildDirectory)/rocm/llvm/bin:$PATH; CC=$(Agent.BuildDirectory)/rocm/llvm/bin/amdclang CMAKE_VERBOSE_MAKEFILE=ON VERBOSE=1 ctest'
testDir: 'rpp-tests'
- script: sudo rm /opt/rocm

65
.azuredevops/dependencies/boost.yml Normal file
View File

@@ -0,0 +1,65 @@
parameters:
- name: checkoutRepo
type: string
default: 'self'
- name: checkoutRef
type: string
default: ''
- name: boostVersion
type: string
default: ''
- name: aptPackages
type: object
default:
- git
- name: rocmDependencies
type: object
default:
- llvm-project
jobs:
- job: boost
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all
steps:
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: Bash@3
displayName: 'git clone boost'
inputs:
targetType: inline
script: git clone -b ${{ parameters.boostVersion }} https://github.com/boostorg/boost --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add ROCm compilers to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/llvm/bin"
- task: Bash@3
displayName: 'Build Boost with clang'
inputs:
targetType: inline
script: |
export CC=$(Agent.BuildDirectory)/rocm/llvm/bin/clang
export CXX=$(Agent.BuildDirectory)/rocm/llvm/bin/clang++
./bootstrap.sh --with-toolset=clang --prefix=$(Build.BinariesDirectory)
./b2 --toolset=clang threading=multi link=shared --prefix=$(Build.BinariesDirectory) cxxflags="-std=c++20"
./b2 install
workingDirectory: $(Build.SourcesDirectory)/boost
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-upload.yml

2
.azuredevops/dependencies/grpc.yml
View File

@@ -20,7 +20,7 @@ jobs:
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all

2
.azuredevops/dependencies/gtest.yml
View File

@@ -20,7 +20,7 @@ jobs:
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all

2
.azuredevops/dependencies/lapack.yml
View File

@@ -21,7 +21,7 @@ jobs:
variables:
- group: common
- template: /.azuredevops/variables-global.yml
pool:
pool:
vmImage: ${{ variables.BASE_BUILD_POOL }}
workspace:
clean: all

245
.azuredevops/nightly/pytorch.yml
View File

@@ -11,7 +11,6 @@ parameters:
- ca-certificates
- bc
- bridge-utils
- cmake
- devscripts
- dkms
- doxygen
@@ -62,9 +61,12 @@ parameters:
- ffmpeg
- libopenblas-dev
- liblapack-dev
- libswscale-dev
- libavformat-dev
- name: pipModules
type: object
default:
- cmake
- astunparse
- expecttest!=0.2.0
- hypothesis
@@ -87,7 +89,10 @@ parameters:
# list for vision
- auditwheel
- future
# list from https://github.com/pytorch/builder/blob/main/manywheel/build_rocm.sh
- pytest
- pytest-azurepipelines
- pillow
# list from https://github.com/pytorch/pytorch/blob/main/.ci/manywheel/build_rocm.sh
- name: rocmDependencies
type: object
default:
@@ -117,6 +122,25 @@ parameters:
- hipCUB
- rocThrust
- hipBLAS-common
- composable_kernel
- name: rocmTestDependencies
type: object
default:
- rocminfo
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_pytorch_test1.sh#L54
# Will iterate through this list using pytest
- name: torchTestList
type: object
default:
- nn
- torch
# - cuda seg faults and might need cuda installed on test system
# - ops takes too long
- unary_ufuncs
- binary_ufuncs
- autograd
# - inductor/torchinductor takes too long
trigger: none
pr: none
@@ -136,13 +160,10 @@ jobs:
amd-staging-gfx942:
ROCM_BRANCH: amd-staging
JOB_GPU_TARGET: gfx942
amd-staging-gfx90a:
ROCM_BRANCH: amd-staging
JOB_GPU_TARGET: gfx90a
variables:
- group: common
- template: /.azuredevops/variables-global.yml
# various flags/parameters expected by bash scripts in pytorch builder repo
# various flags/parameters expected by bash scripts in pytorch repo's .ci directory
- name: ROCM_VERSION
value: 6.3.0
- name: ROCM_PATH
@@ -163,7 +184,7 @@ jobs:
workspace:
clean: all
steps:
# copy environment setup from https://github.com/pytorch/builder/blob/main/manywheel/Dockerfile
# copy environment setup from https://github.com/pytorch/pytorch/blob/main/.ci/docker/manywheel/Dockerfile
# but instead of centos, use ubuntu environment
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-cmake-latest.yml
- task: Bash@3
@@ -200,18 +221,19 @@ jobs:
targetType: inline
script: sudo ln -s $(Agent.BuildDirectory)/rocm /opt/rocm
- checkout: self
- task: Bash@3
displayName: git clone pytorch builder
inputs:
targetType: inline
script: git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: git clone upstream pytorch
inputs:
targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
# builder clone still needed due to run_tests.sh at end of build_common.sh call
- task: Bash@3
displayName: git clone pytorch builder
inputs:
targetType: inline
script: git clone https://github.com/pytorch/builder.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)/pytorch
- task: Bash@3
displayName: Install patchelf
inputs:
@@ -264,8 +286,8 @@ jobs:
PYTORCH_BUILD_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)
PYTORCH_BUILD_NUMBER=$(date -u +%Y%m%d)
SKIP_ALL_TESTS=1
bash ./manywheel/build_rocm.sh
workingDirectory: $(Build.SourcesDirectory)/builder
bash ./.ci/manywheel/build_rocm.sh
workingDirectory: $(Build.SourcesDirectory)/pytorch
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/artifact-prepare-package.yml
parameters:
sourceDir: /remote/wheelhouserocm$(ROCM_VERSION)
@@ -300,7 +322,10 @@ jobs:
inputs:
targetType: inline
script: >-
BUILD_VERSION=$(ROCM_BRANCH).$(JOB_GPU_TARGET)-$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)$(date -u +%Y%m%d)
TORCH_PACKAGE_NAME=torch.$(ROCM_BRANCH).$(JOB_GPU_TARGET)
TORCHVISION_PACKAGE_NAME=torchvision.$(ROCM_BRANCH).$(JOB_GPU_TARGET)
PYTORCH_VERSION=$(cat $(Build.SourcesDirectory)/pytorch/version.txt | cut -da -f1)post$(date -u +%Y%m%d)
BUILD_VERSION=$(cat $(Build.SourcesDirectory)/vision/version.txt | cut -da -f1)post$(date -u +%Y%m%d)
python3 setup.py bdist_wheel
workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3
@@ -319,3 +344,191 @@ jobs:
retryCountOnTaskFailure: 3
inputs:
targetPath: $(Build.BinariesDirectory)
- job: torchvision_testing
dependsOn: pytorch
condition: and(succeeded(), eq(variables.ENABLE_GFX942_TESTS, 'true'), not(containsValue(split(variables.DISABLED_GFX942_TESTS, ','), variables['Build.DefinitionName'])))
variables:
- group: common
- template: /.azuredevops/variables-global.yml
- name: PYTORCH_TEST_WITH_ROCM
value: 1
pool: $(JOB_TEST_POOL)
workspace:
clean: all
strategy:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
JOB_TEST_POOL: ${{ variables.GFX942_TEST_POOL }}
steps:
- task: Bash@3
displayName: 'Register libjpeg-turbo packages'
inputs:
targetType: inline
script: |
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget -q -O- https://packagecloud.io/dcommander/libjpeg-turbo/gpgkey | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/libjpeg-turbo.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/libjpeg-turbo.gpg] https://packagecloud.io/dcommander/libjpeg-turbo/any/ any main" | sudo tee /etc/apt/sources.list.d/libjpeg-turbo.list
sudo apt update
apt-cache show libjpeg-turbo-official | grep Version
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-other.yml
parameters:
aptPackages: ${{ parameters.aptPackages }}
pipModules: ${{ parameters.pipModules }}
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/preamble.yml
- task: DownloadPipelineArtifact@2
displayName: 'Download Pipeline Wheel Files'
inputs:
itemPattern: '**/*$(JOB_GPU_TARGET)*.whl'
targetPath: $(Agent.BuildDirectory)
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-aqlprofile.yml
parameters:
dependencySource: staging
- template: ${{ variables.CI_TEMPLATE_PATH }}/steps/dependencies-rocm.yml
parameters:
dependencyList: ${{ parameters.rocmTestDependencies }}
gpuTarget: $(JOB_GPU_TARGET)
dependencySource: staging
skipLlvmSymlink: true
# get sources to run test scripts
- task: Bash@3
displayName: git clone upstream pytorch
inputs:
targetType: inline
script: git clone https://github.com/pytorch/pytorch.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: git clone pytorch vision
inputs:
targetType: inline
script: git clone https://github.com/pytorch/vision.git --depth=1 --recurse-submodules
workingDirectory: $(Build.SourcesDirectory)
- task: Bash@3
displayName: Install Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip install --no-index --find-links=. --no-dependencies -v {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Show Updated pip List
inputs:
targetType: inline
script: pip list -v
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Add ROCm binaries to PATH
inputs:
targetType: inline
script: echo "##vso[task.prependpath]$(Agent.BuildDirectory)/rocm/bin"
- task: Bash@3
displayName: Add Python site-packages binaries to path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.prependpath]$USER_BASE/bin"
- task: Bash@3
displayName: Add torch libs to ldconfig
inputs:
targetType: inline
script: |
echo $(python3 -m site --user-site)/torch/lib | sudo tee /etc/ld.so.conf.d/torch.conf
sudo ldconfig -v
ldconfig -p
# https://pytorch.org/get-started/locally/#linux-verification
# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#testing-the-pytorch-installation
- task: Bash@3
displayName: Simple Import Torch Tests
inputs:
targetType: inline
script: |
python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'
python3 -c 'import torch; x = torch.rand(5, 3); print(x)'
# Test artifact build script has too many if statements for different environments
# Based off the snippet of interest for this environment, with some adjustments
# https://github.com/pytorch/pytorch/blob/main/.ci/pytorch/build.sh#L335-L375
# Removing in-line comments since it does not fit with the yaml markup
- task: Bash@3
displayName: Build Pytorch Test Artifacts
continueOnError: true
inputs:
targetType: inline
script: |
CUSTOM_TEST_ARTIFACT_BUILD_DIR="build/custom_test_artifacts"
CUSTOM_TEST_USE_ROCM=ON
CUSTOM_TEST_MODULE_PATH="${PWD}/cmake/public"
mkdir -pv "${CUSTOM_TEST_ARTIFACT_BUILD_DIR}"
CUSTOM_OP_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-op-build"
CUSTOM_OP_TEST="${PWD}/test/custom_operator"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$CUSTOM_OP_BUILD"
pushd "$CUSTOM_OP_BUILD"
cmake "$CUSTOM_OP_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
JIT_HOOK_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/jit-hook-build"
JIT_HOOK_TEST="$PWD/test/jit_hooks"
python --version
SITE_PACKAGES="$(python -c 'import site; print(";".join([x for x in site.getsitepackages()] + [x + "/torch" for x in site.getsitepackages()]))')"
mkdir -p "$JIT_HOOK_BUILD"
pushd "$JIT_HOOK_BUILD"
cmake "$JIT_HOOK_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
CUSTOM_BACKEND_BUILD="${CUSTOM_TEST_ARTIFACT_BUILD_DIR}/custom-backend-build"
CUSTOM_BACKEND_TEST="${PWD}/test/custom_backend"
python --version
mkdir -p "$CUSTOM_BACKEND_BUILD"
pushd "$CUSTOM_BACKEND_BUILD"
cmake "$CUSTOM_BACKEND_TEST" -DCMAKE_PREFIX_PATH="$SITE_PACKAGES" -DPython_EXECUTABLE="$(which python)" \
-DCMAKE_MODULE_PATH="$CUSTOM_TEST_MODULE_PATH" -DUSE_ROCM="$CUSTOM_TEST_USE_ROCM"
make VERBOSE=1
popd
workingDirectory: $(Build.SourcesDirectory)/pytorch
- ${{ each torchTest in parameters.torchTestList }}:
- task: Bash@3
displayName: Test ${{ torchTest }}
continueOnError: true
inputs:
targetType: inline
workingDirectory: $(Build.SourcesDirectory)/pytorch
${{ if contains(torchTest, '/') }}:
script: pytest test/${{ split(torchTest, '/')[0] }}/test_${{ split(torchTest, '/')[1] }}.py
${{ else }}:
script: pytest test/test_${{ torchTest }}.py
# Reference on what tests to run for torchvision found in private repo:
# https://github.com/ROCm/rocAutomation/blob/jenkins-pipelines/pytorch/pytorch_ci/test_torchvision.sh#L51
- task: Bash@3
displayName: Test vision/transforms
continueOnError: true
inputs:
targetType: inline
script: pytest test/test_transforms.py
workingDirectory: $(Build.SourcesDirectory)/vision
- task: Bash@3
displayName: Uninstall Wheel Files
inputs:
targetType: inline
script: find . -name "*.whl" -exec pip uninstall -y {} \;
workingDirectory: $(Agent.BuildDirectory)
- task: Bash@3
displayName: Remove Python site-packages binaries from path
inputs:
targetType: inline
script: |
USER_BASE=$(python3 -m site --user-base)
echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$USER_BASE/bin;;' -e 's;^/;;' -e 's;/$;;')"
- task: Bash@3
displayName: Remove ROCm binaries from PATH
inputs:
targetType: inline
script: echo "##vso[task.setvariable variable=PATH]$(echo $PATH | sed -e 's;:$(Agent.BuildDirectory)/rocm/bin;;' -e 's;^/;;' -e 's;/$;;')"

13
.azuredevops/nightly/rocm-nightly.yml
View File

@@ -1,5 +1,5 @@
parameters:
# currently excludes clr and rocm-examples
# currently excludes clr
- name: rocmDependencies
type: object
default:
@@ -10,6 +10,7 @@ parameters:
- composable_kernel
- half
- HIP
- hip-tests
- hipBLAS
- hipBLAS-common
- hipBLASLt
@@ -25,6 +26,7 @@ parameters:
- llvm-project
- MIOpen
- MIVisionX
- omniperf
- rccl
- rdc
- rocAL
@@ -36,6 +38,7 @@ parameters:
- ROCgdb
- rocm-cmake
- rocm-core
- rocm-examples
- rocminfo
- rocMLIR
- ROCmValidationSuite
@@ -43,7 +46,10 @@ parameters:
- rocm_smi_lib
- rocPRIM
- rocprofiler-register
- rocprofiler-sdk
- rocprofiler-systems
- rocprofiler
- rocPyDecode
- ROCR-Runtime
- rocRAND
- rocr_debug_agent
@@ -76,8 +82,6 @@ jobs:
matrix:
gfx942:
JOB_GPU_TARGET: gfx942
gfx90a:
JOB_GPU_TARGET: gfx90a
steps:
- task: DeleteFiles@1
displayName: 'Cleanup checkout space'
@@ -97,6 +101,7 @@ jobs:
dependencyList: ${{ parameters.rocmDependencies }}
dependencySource: staging
skipLibraryLinking: true
skipLlvmSymlink: true
gpuTarget: $(JOB_GPU_TARGET)
- script: df -h
displayName: System disk space after ROCm
@@ -113,7 +118,7 @@ jobs:
- script: du -sh $(Build.ArtifactStagingDirectory)
displayName: Compressed ROCm size
- task: PublishPipelineArtifact@1
displayName: 'Public ROCm Nightly Artifact'
displayName: 'Publish ROCm Nightly Artifact'
retryCountOnTaskFailure: 3
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'

29
.azuredevops/tag-builds/Tensile.yml Normal file
View File

@@ -0,0 +1,29 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/Tensile
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/Tensile.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

23
.azuredevops/tag-builds/boost.yml Normal file
View File

@@ -0,0 +1,23 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: boostVersion
type: string
default: 'master'
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_DEPENDENCIES_PATH }}/boost.yml
parameters:
boostVersion: ${{ parameters.boostVersion }}

29
.azuredevops/tag-builds/omniperf.yml Normal file
View File

@@ -0,0 +1,29 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/omniperf
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/omniperf.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

29
.azuredevops/tag-builds/omnitrace.yml Normal file
View File

@@ -0,0 +1,29 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/omnitrace
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/omnitrace.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

29
.azuredevops/tag-builds/rocprofiler-sdk.yml Normal file
View File

@@ -0,0 +1,29 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-sdk
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-sdk.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

29
.azuredevops/tag-builds/rocprofiler-systems.yml Normal file
View File

@@ -0,0 +1,29 @@
variables:
- group: common
- template: /.azuredevops/variables-global.yml
parameters:
- name: checkoutRef
type: string
default: refs/tags/$(LATEST_RELEASE_TAG)
resources:
repositories:
- repository: pipelines_repo
type: github
endpoint: ROCm
name: ROCm/ROCm
- repository: release_repo
type: github
endpoint: ROCm
name: ROCm/rocprofiler-systems
ref: ${{ parameters.checkoutRef }}
trigger: none
pr: none
jobs:
- template: ${{ variables.CI_COMPONENT_PATH }}/rocprofiler-systems.yml
parameters:
checkoutRepo: release_repo
checkoutRef: ${{ parameters.checkoutRef }}

87
.azuredevops/templates/steps/artifact-download.yml
View File

@@ -9,6 +9,10 @@ parameters:
- name: useDefaultBranch
type: boolean
default: true
# useMainlineBranch only processed if useDefaultBranch is false
- name: useMainlineBranch
type: boolean
default: false
- name: latestFromBranch
type: boolean
default: true
@@ -22,13 +26,14 @@ parameters:
type: object
default:
AMDMIGraphX: develop
amdsmi: develop
amdsmi: amd-staging
aomp-extras: aomp-dev
aomp: aomp-dev
clr: amd-staging
composable_kernel: develop
half: rocm
HIP: amd-staging
hip-tests: amd-staging
hipBLAS: develop
hipBLASLt: develop
hipBLAS-common: develop
@@ -44,8 +49,10 @@ parameters:
llvm-project: amd-staging
MIOpen: develop
MIVisionX: develop
omniperf: amd-staging
omnitrace: amd-staging
rccl: develop
rdc: develop
rdc: amd-staging
rocAL: develop
rocALUTION: develop
rocBLAS: develop
@@ -54,16 +61,19 @@ parameters:
rocFFT: develop
ROCgdb: amd-staging
rocm-cmake: develop
rocm-core: master
rocm-core: amd-staging
rocm-examples: develop
rocminfo: amd-staging
rocMLIR: develop
ROCmValidationSuite: master
rocm_bandwidth_test: master
rocm_smi_lib: develop
rocm_smi_lib: amd-staging
rocPRIM: develop
rocprofiler-register: amd-mainline
rocprofiler: amd-staging
rocprofiler-register: amd-staging
rocprofiler-sdk: amd-staging
rocprofiler-systems: amd-staging
rocPyDecode: develop
ROCR-Runtime: amd-staging
rocRAND: develop
rocr_debug_agent: amd-staging
@@ -73,18 +83,81 @@ parameters:
roctracer: amd-staging
rocWMMA: develop
rpp: develop
- name: mainlineBranchList
type: object
default:
AMDMIGraphX: mainline
amdsmi: amd-mainline
aomp-extras: amd-mainline-open
aomp: amd-mainline-open
clr: amd-mainline
composable_kernel: mainline
half: rocm
HIP: amd-mainline
hip-tests: amd-mainline
hipBLAS: mainline
hipBLASLt: mainline
hipBLAS-common: mainline
hipCUB: mainline
hipFFT: mainline
hipfort: mainline
HIPIFY: amd-mainline
hipRAND: mainline
hipSOLVER: mainline
hipSPARSE: mainline
hipSPARSELt: mainline
hipTensor: mainline
llvm-project: amd-mainline-open
MIOpen: mainline
MIVisionX: mainline
omniperf: amd-mainline
omnitrace: amd-mainline
rccl: mainline
rdc: amd-mainline
rocAL: master # needs the yaml file
rocALUTION: mainline
rocBLAS: mainline
ROCdbgapi : amd-mainline
rocDecode: mainline
rocFFT: mainline
ROCgdb: amd-mainline-rocgdb-15 #
rocm-cmake: mainline
rocm-core: amd-master
rocm-examples: develop # no mainline
rocminfo: amd-master
rocMLIR: mainline # needs the yaml file
ROCmValidationSuite: mainline
rocm_bandwidth_test: master
rocm_smi_lib: amd-mainline
rocPRIM: mainline
rocprofiler: amd-master
rocprofiler-register: amd-mainline
rocprofiler-sdk: amd-mainline
rocprofiler-systems: amd-mainline
rocPyDecode: mainline
ROCR-Runtime: amd-master
rocRAND: mainline
rocr_debug_agent: amd-mainline
rocSOLVER: mainline
rocSPARSE: mainline
rocThrust: mainline
roctracer: amd-master
rocWMMA: mainline
rpp: mainline
- name: allowPartiallySucceededBuilds
type: object
default:
- amdsmi
- aomp
- HIPIFY
- MIVisionX
- omniperf
- rccl
- rdc
- rocm-cmake
- rocm_smi_lib
- rocFFT
- MIVisionX
- rpp
# BELOW REQUIRED IF useDefaultBranch false
- name: branchName
type: string
@@ -104,6 +177,8 @@ steps:
buildVersionToDownload: latestFromBranch # default is 'latest'
${{ if eq(parameters.useDefaultBranch, true) }}:
branchName: refs/heads/${{ parameters.defaultBranchList[parameters.componentName] }}
${{ elseif eq(parameters.useMainlineBranch, true) }}:
branchName: refs/heads/${{ parameters.mainlineBranchList[parameters.componentName] }}
${{ else }}:
branchName: ${{ parameters.branchName }}
${{ if containsValue(parameters.allowPartiallySucceededBuilds, parameters.componentName) }}:

5
.azuredevops/templates/steps/build-autotools.yml
View File

@@ -11,6 +11,9 @@ parameters:
- name: installDir
type: string
default: '$(Build.BinariesDirectory)'
- name: makeCallPrefix
type: string
default: ''
steps:
- task: Bash@3
@@ -23,7 +26,7 @@ steps:
displayName: '${{ parameters.componentName }} make'
inputs:
targetType: inline
script: make -j$(nproc)
script: ${{ parameters.makeCallPrefix }} make -j$(nproc)
workingDirectory: ${{ parameters.buildDir }}
- task: Bash@3
displayName: '${{ parameters.componentName }} make install'

22
.azuredevops/templates/steps/dependencies-aqlprofile.yml
View File

@@ -8,20 +8,22 @@ parameters:
- name: repositoryUrl
type: object
default:
staging: https://repo.radeon.com/rocm/apt/6.2/pool/main/h/hsa-amd-aqlprofile
tag-builds: https://repo.radeon.com/rocm/apt/6.2/pool/main/h/hsa-amd-aqlprofile
- name: packageName
type: object
default:
staging: hsa-amd-aqlprofile_1.0.0.60200.60200-66~22.04_amd64.deb
tag-builds: hsa-amd-aqlprofile_1.0.0.60200.60200-66~22.04_amd64.deb
staging: https://repo.radeon.com/rocm/apt/latest/pool/main/h/hsa-amd-aqlprofile/ # end slash is important for curl!
tag-builds: https://repo.radeon.com/rocm/apt/$(TAGGED_RELEASE)/pool/main/h/hsa-amd-aqlprofile/
steps:
- task: Bash@3
displayName: Get aqlprofile package name
inputs:
targetType: inline
script: |
export packageName=$(curl -s ${{ parameters.repositoryUrl[parameters.dependencySource] }} | grep -oP "href=\"\K[^\"]*$(lsb_release -rs)[^\"]*\.deb")
echo "##vso[task.setvariable variable=packageName;isreadonly=true]$packageName"
- task: Bash@3
displayName: 'Download aqlprofile'
inputs:
targetType: inline
script: wget -nv ${{ parameters.repositoryUrl[parameters.dependencySource] }}/${{ parameters.packageName[parameters.dependencySource] }}
script: wget -nv ${{ parameters.repositoryUrl[parameters.dependencySource] }}$(packageName)
workingDirectory: '$(Pipeline.Workspace)'
- task: Bash@3
displayName: 'Extract aqlprofile'
@@ -29,7 +31,7 @@ steps:
targetType: inline
script: |
mkdir hsa-amd-aqlprofile
dpkg-deb -R ${{ parameters.packageName[parameters.dependencySource] }} hsa-amd-aqlprofile
dpkg-deb -R $(packageName) hsa-amd-aqlprofile
workingDirectory: '$(Pipeline.Workspace)'
- task: Bash@3
displayName: 'Copy aqlprofile files'
@@ -43,5 +45,5 @@ steps:
displayName: 'Clean up aqlprofile'
inputs:
targetType: inline
script: rm -rf hsa-amd-aqlprofile ${{ parameters.packageName[parameters.dependencySource] }}
script: rm -rf hsa-amd-aqlprofile $(packageName)
workingDirectory: '$(Pipeline.Workspace)'

35
.azuredevops/templates/steps/dependencies-boost.yml Normal file
View File

@@ -0,0 +1,35 @@
steps:
- task: DownloadPipelineArtifact@2
displayName: Download Boost
inputs:
buildType: specific
project: ROCm-CI
definition: $(BOOST_DEPENDENCY_PIPELINE_ID)
targetPath: $(Pipeline.Workspace)/d
- task: ExtractFiles@1
displayName: Extract Boost
inputs:
archiveFilePatterns: '$(Pipeline.Workspace)/d/**/*.tar.gz'
destinationFolder: $(Agent.BuildDirectory)/boost
cleanDestinationFolder: true
overwriteExistingFiles: true
- task: DeleteFiles@1
displayName: Cleanup Compressed Boost
inputs:
SourceFolder: $(Pipeline.Workspace)/d
Contents: '**/*.tar.gz'
RemoveDotFiles: true
- task: Bash@3
displayName: 'List Boost files'
inputs:
targetType: inline
script: ls -1R $(Agent.BuildDirectory)/boost
- task: Bash@3
displayName: 'Link Boost shared libraries'
inputs:
targetType: inline
script: |
echo $(Agent.BuildDirectory)/boost/lib | sudo tee /etc/ld.so.conf.d/boost.conf
sudo cat /etc/ld.so.conf.d/boost.conf
sudo ldconfig -v
ldconfig -p

2
.azuredevops/templates/steps/dependencies-other.yml
View File

@@ -34,4 +34,4 @@ steps:
displayName: 'pip install ...'
inputs:
targetType: inline
script: pip install ${{ join(' ', parameters.pipModules) }}
script: pip install -v ${{ join(' ', parameters.pipModules) }}

26
.azuredevops/templates/steps/dependencies-rocm.yml
View File

@@ -9,6 +9,7 @@ parameters:
default: staging
values:
- staging
- mainline
- tag-builds
- fixed
- name: extractToMnt
@@ -37,6 +38,7 @@ parameters:
composable_kernel: $(COMPOSABLE_KERNEL_PIPELINE_ID)
half: $(HALF_PIPELINE_ID)
HIP: $(HIP_PIPELINE_ID)
hip-tests: $(HIP_TESTS_PIPELINE_ID)
hipBLAS: $(HIPBLAS_PIPELINE_ID)
hipBLAS-common: $(HIPBLAS_COMMON_PIPELINE_ID)
hipBLASLt: $(HIPBLASLT_PIPELINE_ID)
@@ -52,6 +54,8 @@ parameters:
llvm-project: $(LLVM_PROJECT_PIPELINE_ID)
MIOpen: $(MIOpen_PIPELINE_ID)
MIVisionX: $(MIVISIONX_PIPELINE_ID)
omniperf: $(OMNIPERF_PIPELINE_ID)
omnitrace: $(OMNITRACE_PIPELINE_ID)
rccl: $(RCCL_PIPELINE_ID)
rdc: $(RDC_PIPELINE_ID)
rocAL: $(ROCAL_PIPELINE_ID)
@@ -71,7 +75,10 @@ parameters:
rocm_smi_lib: $(ROCM_SMI_LIB_PIPELINE_ID)
rocPRIM: $(ROCPRIM_PIPELINE_ID)
rocprofiler-register: $(ROCPROFILER_REGISTER_PIPELINE_ID)
rocprofiler-sdk: $(ROCPROFILER_SDK_PIPELINE_ID)
rocprofiler-systems: $(ROCPROFILER_SYSTEMS_PIPELINE_ID)
rocprofiler: $(ROCPROFILER_PIPELINE_ID)
rocPyDecode: $(ROCPYDECODE_PIPELINE_ID)
ROCR-Runtime: $(ROCR_RUNTIME_PIPELINE_ID)
rocRAND: $(ROCRAND_PIPELINE_ID)
rocr_debug_agent: $(ROCR_DEBUG_AGENT_PIPELINE_ID)
@@ -93,6 +100,7 @@ parameters:
composable_kernel: $(COMPOSABLE_KERNEL_TAGGED_PIPELINE_ID)
half: $(HALF_TAGGED_PIPELINE_ID)
HIP: $(HIP_TAGGED_PIPELINE_ID)
hip-tests: $(HIP_TESTS_TAGGED_PIPELINE_ID)
hipBLAS: $(HIPBLAS_TAGGED_PIPELINE_ID)
hipBLAS-common: $(HIPBLAS_COMMON_TAGGED_PIPELINE_ID)
hipBLASLt: $(HIPBLASLT_TAGGED_PIPELINE_ID)
@@ -108,6 +116,8 @@ parameters:
llvm-project: $(LLVM_PROJECT_TAGGED_PIPELINE_ID)
MIOpen: $(MIOpen_TAGGED_PIPELINE_ID)
MIVisionX: $(MIVISIONX_TAGGED_PIPELINE_ID)
omniperf: $(OMNIPERF_TAGGED_PIPELINE_ID)
omnitrace: $(OMNITRACE_TAGGED_PIPELINE_ID)
rccl: $(RCCL_TAGGED_PIPELINE_ID)
rdc: $(RDC_TAGGED_PIPELINE_ID)
rocAL: $(ROCAL_TAGGED_PIPELINE_ID)
@@ -127,7 +137,10 @@ parameters:
rocm_smi_lib: $(ROCM_SMI_LIB_TAGGED_PIPELINE_ID)
rocPRIM: $(ROCPRIM_TAGGED_PIPELINE_ID)
rocprofiler-register: $(ROCPROFILER_REGISTER_TAGGED_PIPELINE_ID)
rocprofiler-sdk: $(ROCPROFILER_SDK_TAGGED_PIPELINE_ID)
rocprofiler-systems: $(ROCPROFILER_SYSTEMS_PIPELINE_ID)
rocprofiler: $(ROCPROFILER_TAGGED_PIPELINE_ID)
rocPyDecode: $(ROCPYDECODE_TAGGED_PIPELINE_ID)
ROCR-Runtime: $(ROCR_RUNTIME_TAGGED_PIPELINE_ID)
rocRAND: $(ROCRAND_TAGGED_PIPELINE_ID)
rocr_debug_agent: $(ROCR_DEBUG_AGENT_TAGGED_PIPELINE_ID)
@@ -166,13 +179,17 @@ parameters:
- hipRAND
- hipSPARSELt
- hipTensor
- omnitrace
- rccl
- rocALUTION
- rocBLAS
- rocFFT
- rocm-examples
- rocPRIM
- rocprofiler-sdk
- rocprofiler-systems
- rocprofiler
- rocPyDecode
- rocRAND
- rocSOLVER
- rocSPARSE
@@ -186,6 +203,7 @@ parameters:
- hipSPARSE
- MIOpen
- MIVision
- omniperf
- rocAL
- ROCmValidationSuite
@@ -205,6 +223,10 @@ steps:
${{ if eq(parameters.dependencySource, 'staging') }}:
pipelineId: ${{ parameters.stagingPipelineIdentifiers[ split(dependency, ':')[0] ] }}
latestFromBranch: ${{ parameters.latestFromBranch }}
${{ elseif eq(parameters.dependencySource, 'mainline') }}:
pipelineId: ${{ parameters.stagingPipelineIdentifiers[ split(dependency, ':')[0] ] }}
useMainlineBranch: true
latestFromBranch: ${{ parameters.latestFromBranch }}
${{ elseif eq(parameters.dependencySource, 'tag-builds') }}:
pipelineId: ${{ parameters.taggedPipelineIdentifiers[ split(dependency, ':')[0] ] }}
latestFromBranch: false
@@ -221,6 +243,10 @@ steps:
${{ if eq(parameters.dependencySource, 'staging') }}:
pipelineId: ${{ parameters.stagingPipelineIdentifiers[dependency] }}
latestFromBranch: ${{ parameters.latestFromBranch }}
${{ elseif eq(parameters.dependencySource, 'mainline') }}:
pipelineId: ${{ parameters.stagingPipelineIdentifiers[dependency] }}
useMainlineBranch: true
latestFromBranch: ${{ parameters.latestFromBranch }}
${{ elseif eq(parameters.dependencySource, 'tag-builds') }}:
pipelineId: ${{ parameters.taggedPipelineIdentifiers[dependency] }}
latestFromBranch: false

54
.azuredevops/templates/steps/gpu-diagnostics.yml Normal file
View File

@@ -0,0 +1,54 @@
# Diagnostics for GPU-enabled systems
parameters:
- name: runRocminfo
type: boolean
default: true
steps:
- ${{ if eq(parameters.runRocminfo, true) }}:
- task: Bash@3
displayName: 'rocminfo'
continueOnError: true
inputs:
targetType: inline
script: $(Agent.BuildDirectory)/rocm/bin/rocminfo
- task: Bash@3
displayName: 'rocm_agent_enumerator'
continueOnError: true
inputs:
targetType: inline
script: $(Agent.BuildDirectory)/rocm/bin/rocm_agent_enumerator
- task: Bash@3
displayName: 'List DRI devices'
continueOnError: true
inputs:
targetType: inline
script: ls -la /dev/dri/by-path/
- task: Bash@3
displayName: 'List amdgpu/rocm/mesa packages'
continueOnError: true
inputs:
targetType: inline
script: apt list --installed | grep -E 'amdgpu|rocm|mesa'
- task: Bash@3
displayName: 'List GPU processes'
continueOnError: true
inputs:
targetType: inline
script: |
ls /sys/class/kfd/kfd/proc/
sudo lsof | grep amdgpu
- task: Bash@3
displayName: 'System snapshot'
continueOnError: true
inputs:
targetType: inline
script: top -bn1
- task: Bash@3
displayName: 'List dmesg'
continueOnError: true
inputs:
targetType: inline
script: |
echo 'rocm-ci: $(Build.DefinitionName) $(System.DefinitionId)' | sudo tee /dev/kmsg
sudo dmesg

34
.azuredevops/variables-global.yml
View File

@@ -79,6 +79,10 @@ variables:
value: 93
- name: HIP_TAGGED_PIPELINE_ID
value: 31
- name: HIP_TESTS_PIPELINE_ID
value: 233
- name: HIP_TESTS_TAGGED_PIPELINE_ID
value: 220
- name: HIPBLAS_COMMON_PIPELINE_ID
value: 223
- name: HIPBLAS_COMMON_TAGGED_PIPELINE_ID
@@ -159,6 +163,14 @@ variables:
value: 80
- name: MIVISIONX_TAGGED_PIPELINE_ID
value: 18
- name: OMNIPERF_PIPELINE_ID
value: 241
- name: OMNIPERF_TAGGED_PIPELINE_ID
value: 242
- name: OMNITRACE_PIPELINE_ID
value: 253
- name: OMNITRACE_TAGGED_PIPELINE_ID
value: 252
- name: RCCL_GFX942_TEST_PIPELINE_ID
value: 184
- name: RCCL_PIPELINE_ID
@@ -219,6 +231,10 @@ variables:
value: 22
- name: ROCM_EXAMPLES_GFX942_TEST_PIPELINE_ID
value: 204
- name: ROCM_EXAMPLES_PIPELINE_ID
value: 216
- name: ROCM_EXAMPLES_TAGGED_PIPELINE_ID
value: 245
- name: ROCM_SMI_LIB_PIPELINE_ID
value: 96
- name: ROCM_SMI_LIB_TAGGED_PIPELINE_ID
@@ -243,14 +259,26 @@ variables:
value: 20
- name: ROCPROFILER_GFX942_TEST_PIPELINE_ID
value: 190
- name: ROCPROFILER_PIPELINE_ID
value: 143
- name: ROCPROFILER_REGISTER_PIPELINE_ID
value: 1
- name: ROCPROFILER_REGISTER_TAGGED_PIPELINE_ID
value: 25
- name: ROCPROFILER_SDK_PIPELINE_ID
value: 246
- name: ROCPROFILER_SDK_TAGGED_PIPELINE_ID
value: 234
- name: ROCPROFILER_SYSTEMS_PIPELINE_ID
value: 255
- name: ROCPROFILER_SYSTEMS_TAGGED_PIPELINE_ID
value: 254
- name: ROCPROFILER_PIPELINE_ID
value: 143
- name: ROCPROFILER_TAGGED_PIPELINE_ID
value: 28
- name: ROCPYDECODE_PIPELINE_ID
value: 239
- name: ROCPYDECODE_TAGGED_PIPELINE_ID
value: 232
- name: ROCR_DEBUG_AGENT_PIPELINE_ID
value: 136
- name: ROCR_DEBUG_AGENT_TAGGED_PIPELINE_ID
@@ -305,3 +333,5 @@ variables:
value: 78
- name: RPP_TAGGED_PIPELINE_ID
value: 39
- name: BOOST_DEPENDENCY_PIPELINE_ID
value: 250

1
.github/CODEOWNERS vendored
View File

@@ -5,3 +5,4 @@ docs/ @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
*.rst @amd-aakash @jlgreathouse @samjwu @yhuiYH @ROCm/rocm-documentation
# External CI
/.azuredevops/ @ROCm/external-ci
tools/rocm-build/ @ROCm/rocm-devops

4
.wordlist.txt
View File

@@ -272,6 +272,7 @@ OpenMPI
OpenSSL
OpenVX
OpenXLA
Oversubscription
PCC
PCI
PCIe
@@ -490,6 +491,7 @@ cuLIB
cuRAND
cuSOLVER
cuSPARSE
customizations
cTDP
dataset
datasets
@@ -619,6 +621,7 @@ openmp
openssl
optimizers
os
oversubscription
pageable
parallelization
parameterization
@@ -698,6 +701,7 @@ rocsolver
rocsparse
rocthrust
roctracer
rst
runtime
runtimes
sL

6
README.md
View File

@@ -50,7 +50,7 @@ The following example shows how to use the repo tool to download the ROCm source
```bash
mkdir -p ~/ROCm/
cd ~/ROCm/
~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.0.x
~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.2.x
~/bin/repo sync
```
@@ -76,7 +76,7 @@ The Build time will reduce significantly if we limit the GPU Architecture/s agai
mkdir -p ~/WORKSPACE/ # Or any folder name other than WORKSPACE
cd ~/WORKSPACE/
export ROCM_VERSION=6.2.0
export ROCM_VERSION=6.2.2 # Or 6.2.0 or 6.2.1
~/bin/repo init -u http://github.com/ROCm/ROCm.git -b roc-6.2.x -m tools/rocm-build/rocm-${ROCM_VERSION}.xml
~/bin/repo sync
@@ -199,4 +199,4 @@ cmake --build build --target=doc
## Older ROCm releases
For release information for older ROCm releases, refer to the
[CHANGELOG](./CHANGELOG.md).
[ROCm release history](https://rocm.docs.amd.com/en/latest/release/versions.html).

2
default.xml
View File

@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="rocm-org" fetch="https://github.com/ROCm/" />
<default revision="refs/tags/rocm-6.2.0"
<default revision="refs/tags/rocm-6.2.2"
remote="rocm-org"
sync-c="true"
sync-j="4" />

3
docs/about/license.md
View File

@@ -91,8 +91,7 @@ additional licenses. Please review individual repositories for more information.
Open sourced ROCm components are released via public GitHub
repositories, packages on [https://repo.radeon.com](https://repo.radeon.com) and other distribution channels.
Proprietary products are only available on [https://repo.radeon.com](https://repo.radeon.com). Currently, only
one component of ROCm, `rocm-llvm-alt` is governed by a proprietary license.
Proprietary products are only available on [https://repo.radeon.com](https://repo.radeon.com).
Proprietary components are organized in a proprietary subdirectory in the package
repositories to distinguish from open sourced packages.

2
docs/compatibility/compatibility-matrix.rst
View File

@@ -10,6 +10,8 @@ Use this matrix to view the ROCm compatibility and system requirements across su
You can also refer to the :ref:`past versions of ROCm compatibility matrix<past-rocm-compatibility-matrix>`.
Accelerators and GPUs listed in the following table support compute workloads (no display information or graphics). If you’re using ROCm with AMD Radeon or Radeon Pro GPUs for graphics workloads, see the `Use ROCm on Radeon GPU documentation <https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility.html>`_ to verify compatibility and system requirements.
.. |br| raw:: html
<br/>

333
docs/conceptual/ai-migraphx-optimization.md
View File

@@ -1,333 +0,0 @@
<head>
<meta charset="UTF-8">
<meta name="description" content="Inference optimization with MIGraphX">
<meta name="keywords" content="Inference optimization, MIGraphX, deep-learning, MIGraphX
installation, AMD, ROCm">
</head>
# Inference optimization with MIGraphX
The following sections cover inferencing and introduces [MIGraphX](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/).
## Inference
The inference is where capabilities learned during deep-learning training are put to work. It refers to using a fully trained neural network to make conclusions (predictions) on unseen data that the model has never interacted with before. Deep-learning inferencing is achieved by feeding new data, such as new images, to the network, giving the Deep Neural Network a chance to classify the image.
Taking our previous example of MNIST, the DNN can be fed new images of handwritten digit images, allowing the neural network to classify digits. A fully trained DNN should make accurate predictions about what an image represents, and inference cannot happen without training.
## MIGraphX introduction
MIGraphX is a graph compiler focused on accelerating the machine-learning inference that can target AMD GPUs and CPUs. MIGraphX accelerates the machine-learning models by leveraging several graph-level transformations and optimizations. These optimizations include:
* Operator fusion
* Arithmetic simplifications
* Dead-code elimination
* Common subexpression elimination (CSE)
* Constant propagation
After doing all these transformations, MIGraphX emits code for the AMD GPU by calling to MIOpen or rocBLAS or creating HIP kernels for a particular operator. MIGraphX can also target CPUs using DNNL or ZenDNN libraries.
MIGraphX provides easy-to-use APIs in C++ and Python to import machine models in ONNX or TensorFlow. Users can compile, save, load, and run these models using the MIGraphX C++ and Python APIs. Internally, MIGraphX parses ONNX or TensorFlow models into internal graph representation where each operator in the model gets mapped to an operator within MIGraphX. Each of these operators defines various attributes such as:
* Number of arguments
* Type of arguments
* Shape of arguments
After optimization passes, all these operators get mapped to different kernels on GPUs or CPUs.
After importing a model into MIGraphX, the model is represented as `migraphx::program`. `migraphx::program` is made up of `migraphx::module`. The program can consist of several modules, but it always has one main_module. Modules are made up of `migraphx::instruction_ref`. Instructions contain the `migraphx::op` and arguments to the operator.  
## Installing MIGraphX
There are three options to get started with MIGraphX installation. MIGraphX depends on ROCm libraries; assume that the machine has ROCm installed.
### Option 1: installing binaries
To install MIGraphX on Debian-based systems like Ubuntu, use the following command:
```bash
sudo apt update && sudo apt install -y migraphx
```
The header files and libraries are installed under `/opt/rocm-\<version\>`, where \<version\> is the ROCm version.
### Option 2: building from source
There are two ways to build the MIGraphX sources.
* [Use the ROCm build tool](https://github.com/ROCm/AMDMIGraphX#use-the-rocm-build-tool-rbuild) - This approach uses `[rbuild](https://github.com/ROCm/rbuild)` to install the prerequisites and build the libraries with just one command.
or
* [Use CMake](https://github.com/ROCm/AMDMIGraphX#use-cmake-to-build-migraphx) - This approach uses a script to install the prerequisites, then uses CMake to build the source.
For detailed steps on building from source and installing dependencies, refer to the following `README` file:
[https://github.com/ROCm/AMDMIGraphX#building-from-source](https://github.com/ROCm/AMDMIGraphX#building-from-source)
### Option 3: use docker
To use Docker, follow these steps:
1. The easiest way to set up the development environment is to use Docker. To build Docker from scratch, first clone the MIGraphX repository by running:
```bash
git clone --recursive https://github.com/ROCm/AMDMIGraphX
```
2. The repository contains a Dockerfile from which you can build a Docker image as:
```bash
docker build -t migraphx .
```
3. Then to enter the development environment, use Docker run:
```bash
docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX --group-add video -it migraphx
```
The Docker image contains all the prerequisites required for the installation, so users can go to the folder `/code/AMDMIGraphX` and follow the steps mentioned in [Option 2: Building from Source](#option-2-building-from-source).
## MIGraphX example
MIGraphX provides both C++ and Python APIs. The following sections show examples of both using the Inception v3 model. To walk through the examples, fetch the Inception v3 ONNX model by running the following:
```py
import torch
import torchvision.models as models
inception = models.inception_v3(pretrained=True)
torch.onnx.export(inception,torch.randn(1,3,299,299), "inceptioni1.onnx")
```
This will create `inceptioni1.onnx`, which can be imported in MIGraphX using C++ or Python API.
### MIGraphX Python API
Follow these steps:
1. To import the MIGraphX module in Python script, set `PYTHONPATH` to the MIGraphX libraries installation. If binaries are installed using steps mentioned in [Option 1: Installing Binaries](#option-1-installing-binaries), perform the following action:
```bash
export PYTHONPATH=$PYTHONPATH:/opt/rocm/
```
2. The following script shows the usage of Python API to import the ONNX model, compile it, and run inference on it. Set `LD_LIBRARY_PATH` to `/opt/rocm/` if required.
```py
# import migraphx and numpy
import migraphx
import numpy as np
# import and parse inception model
model = migraphx.parse_onnx("inceptioni1.onnx")
# compile model for the GPU target
model.compile(migraphx.get_target("gpu"))
# optionally print compiled model
model.print()
# create random input image
input_image = np.random.rand(1, 3, 299, 299).astype('float32')
# feed image to model, 'x.1` is the input param name
results = model.run({'x.1': input_image})
# get the results back
result_np = np.array(results[0])
# print the inferred class of the input image
print(np.argmax(result_np))
```
Find additional examples of Python API in the `/examples` directory of the MIGraphX repository.
## MIGraphX C++ API
Follow these steps:
1. The following is a minimalist example that shows the usage of MIGraphX C++ API to load ONNX file, compile it for the GPU, and run inference on it. To use MIGraphX C++ API, you only need to load the `migraphx.hpp` file. This example runs inference on the Inception v3 model.
```c++
#include <vector>
#include <string>
#include <algorithm>
#include <ctime>
#include <random>
#include <migraphx/migraphx.hpp>
int main(int argc, char** argv)
{
migraphx::program prog;
migraphx::onnx_options onnx_opts;
// import and parse onnx file into migraphx::program
prog = parse_onnx("inceptioni1.onnx", onnx_opts);
// print imported model
prog.print();
migraphx::target targ = migraphx::target("gpu");
migraphx::compile_options comp_opts;
comp_opts.set_offload_copy();
// compile for the GPU
prog.compile(targ, comp_opts);
// print the compiled program
prog.print();
// randomly generate input image
// of shape (1, 3, 299, 299)
std::srand(unsigned(std::time(nullptr)));
std::vector<float> input_image(1*299*299*3);
std::generate(input_image.begin(), input_image.end(), std::rand);
// users need to provide data for the input
// parameters in order to run inference
// you can query into migraph program for the parameters
migraphx::program_parameters prog_params;
auto param_shapes = prog.get_parameter_shapes();
auto input = param_shapes.names().front();
// create argument for the parameter
prog_params.add(input, migraphx::argument(param_shapes[input], input_image.data()));
// run inference
auto outputs = prog.eval(prog_params);
// read back the output
float* results = reinterpret_cast<float*>(outputs[0].data());
float* max = std::max_element(results, results + 1000);
int answer = max - results;
std::cout << "answer: " << answer << std::endl;
}
```
2. To compile this program, you can use CMake and you only need to link the `migraphx::c` library to use MIGraphX's C++ API. The following is the `CMakeLists.txt` file that can build the earlier example:
```cmake
cmake_minimum_required(VERSION 3.5)
project (CAI)
set (CMAKE_CXX_STANDARD 14)
set (EXAMPLE inception_inference)
list (APPEND CMAKE_PREFIX_PATH /opt/rocm/hip /opt/rocm)
find_package (migraphx)
message("source file: " ${EXAMPLE}.cpp " ---> bin: " ${EXAMPLE})
add_executable(${EXAMPLE} ${EXAMPLE}.cpp)
target_link_libraries(${EXAMPLE} migraphx::c)
```
3. To build the executable file, run the following from the directory containing the `inception_inference.cpp` file:
```bash
mkdir build
cd build
cmake ..
make -j$(nproc)
./inception_inference
```
:::{note}
Set `LD_LIBRARY_PATH` to `/opt/rocm/lib` if required during the build. Additional examples can be found in the MIGraphX repository under the `/examples/` directory.
:::
## Tuning MIGraphX
MIGraphX uses MIOpen kernels to target AMD GPU. For the model compiled with MIGraphX, tune MIOpen to pick the best possible kernel implementation. The MIOpen tuning results in a significant performance boost. Tuning can be done by setting the environment variable `MIOPEN_FIND_ENFORCE=3`.
:::{note}
The tuning process can take a long time to finish.
:::
**Example:** The average inference time of the inception model example shown previously over 100 iterations using untuned kernels is 0.01383ms. After tuning, it reduces to 0.00459ms, which is a 3x improvement. This result is from ROCm v4.5 on a MI100 GPU.
:::{note}
The results may vary depending on the system configurations.
:::
For reference, the following code snippet shows inference runs for only the first 10 iterations for both tuned and untuned kernels:
```console
### UNTUNED ###
iterator : 0
Inference complete
Inference time: 0.063ms
iterator : 1
Inference complete
Inference time: 0.008ms
iterator : 2
Inference complete
Inference time: 0.007ms
iterator : 3
Inference complete
Inference time: 0.007ms
iterator : 4
Inference complete
Inference time: 0.007ms
iterator : 5
Inference complete
Inference time: 0.008ms
iterator : 6
Inference complete
Inference time: 0.007ms
iterator : 7
Inference complete
Inference time: 0.028ms
iterator : 8
Inference complete
Inference time: 0.029ms
iterator : 9
Inference complete
Inference time: 0.029ms
### TUNED ###
iterator : 0
Inference complete
Inference time: 0.063ms
iterator : 1
Inference complete
Inference time: 0.004ms
iterator : 2
Inference complete
Inference time: 0.004ms
iterator : 3
Inference complete
Inference time: 0.004ms
iterator : 4
Inference complete
Inference time: 0.004ms
iterator : 5
Inference complete
Inference time: 0.004ms
iterator : 6
Inference complete
Inference time: 0.004ms
iterator : 7
Inference complete
Inference time: 0.004ms
iterator : 8
Inference complete
Inference time: 0.004ms
iterator : 9
Inference complete
Inference time: 0.004ms
```
### YModel
The best inference performance through MIGraphX is conditioned upon having tuned kernel configurations stored in a `/home` local User Database (DB). If a user were to move their model to a different server or allow a different user to use it, they would have to run through the MIOpen tuning process again to populate the next User DB with the best kernel configurations and corresponding solvers.
Tuning is time consuming, and if the users have not performed tuning, they would see discrepancies between expected or claimed inference performance and actual inference performance. This has led to repetitive and time-consuming tuning tasks for each user.
MIGraphX introduces a feature, known as YModel, that stores the kernel config parameters found during tuning into a `.mxr` file. This ensures the same level of expected performance, even when a model is copied to a different user/system.
The YModel feature is available starting from ROCm 5.4.1 and UIF 1.1.
#### YModel example
Through the `migraphx-driver` functionality, you can generate `.mxr` files with tuning information stored inside it by passing additional `--binary --output model.mxr` to `migraphx-driver` along with the rest of the necessary flags.
For example, to generate `.mxr` file from the ONNX model, use the following:
```bash
./path/to/migraphx-driver compile --onnx resnet50.onnx --enable-offload-copy --binary --output resnet50.mxr
```
To run generated `.mxr` files through `migraphx-driver`, use the following:
```bash
./path/to/migraphx-driver run --migraphx resnet50.mxr --enable-offload-copy
```
Alternatively, you can use the MIGraphX C++ or Python API to generate `.mxr` files.
![Generating an MXR file](../data/conceptual/image018.png "Generating an MXR file")

34
docs/conceptual/oversubscription.rst Normal file
View File

@@ -0,0 +1,34 @@
.. meta::
:description: Learn what causes oversubscription.
:keywords: warning, log, gpu, performance penalty, help
*******************************************************************
Oversubscription of hardware resources in AMD Instinct accelerators
*******************************************************************
When an AMD Instinct™ MI series accelerator enters an oversubscribed state, the ``amdgpu`` driver outputs the following
message.
``amdgpu: Runlist is getting oversubscribed. Expect reduced ROCm performance.``
Oversubscription occurs when application demands exceed the available hardware resources. In an oversubscribed
state, the hardware scheduler tries to manage resource usage in a round-robin fashion. However,
this can result in reduced performance, as resources might be occupied by applications or queues not actively
submitting work. The granularity of hardware resources occupied by an inactive queue can be in the order of
milliseconds, during which the accelerator or GPU is effectively blocked and unable to process work submitted by other
queues.
What triggers oversubscription?
===============================
The system enters an oversubscribed state when one of the following conditions is met:
* **Hardware queue limit exceeded**: The number of user-mode compute queues requested by applications exceeds the
hardware limit of 24 queues for current Instinct accelerators.
* **Virtual memory context slots exceeded**: The number of user processes exceeds the number of available virtual memory
context slots, which is 11 for current Instinct accelerators.
* **Multiple processes using cooperative workgroups**: More than one process attempts to use the cooperative workgroup
feature, leading to resource contention.

13
docs/contribute/building.md
View File

@@ -7,17 +7,14 @@
# Building documentation
You can build our documentation via GitHub (in a pull request) or locally (using the command line or
Visual Studio (VS) Code.
## GitHub
If you open a pull request on the `develop` branch of a ROCm repository and scroll to the bottom of
the page, there is a summary panel. Next to the line
`docs/readthedocs.com:advanced-micro-devices-demo`, there is a `Details` link. If you click this, it takes
you to the Read the Docs build for your pull request.
If you open a pull request and scroll down to the summary panel,
there is a commit status section. Next to the line
`docs/readthedocs.com:advanced-micro-devices-demo`, there is a `Details` link.
If you click this, it takes you to the Read the Docs build for your pull request.
![Screenshot of the GitHub documentation build link](../data/contribute/github-docs-build.png)
![GitHub PR commit status](../data/contribute/commit-status.png)
If you don't see this line, click `Show all checks` to get an itemized view.

123
docs/contribute/contributing.md
View File

@@ -4,113 +4,68 @@
<meta name="keywords" content="ROCm, contributing, contribute, maintainer, contributor">
</head>
# Contribute to ROCm documentation
# Contributing to the ROCm documentation
All ROCm projects are GitHub-based, so if you want to contribute, you can do so by:
The ROCm documentation, like all of ROCm, is open source and available on GitHub. You can contribute to the ROCm documentation by forking the appropriate repository, making your changes, and opening a pull request.
* [Submitting a pull request in the appropriate GitHub repository](#submit-a-pull-request)
* [Creating an issue in the appropriate GitHub repository](#create-an-issue)
* [Suggesting a new feature](#suggest-a-new-feature)
To provide feedback on the ROCm documentation, including submitting an issue or suggesting a feature, see [Providing feedback about the ROCm documentation](./feedback.md).
```{important}
By creating a pull request (PR), you agree to allow your contribution to be licensed under the terms of the
LICENSE.txt file in the corresponding repository. Different repositories may use different licenses.
```
## The ROCm repositories
## Submit a pull request
The repositories for ROCm and all ROCm components are available on GitHub.
To make edits to our documentation via PR, follow these steps:
| Module | Documentation location |
| --- | --- |
| ROCm framework | [https://github.com/ROCm/ROCm/tree/develop/docs](https://github.com/ROCm/ROCm/tree/develop/docs) |
| ROCm installation for Linux | [https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs](https://github.com/ROCm/rocm-install-on-linux/tree/develop/docs) |
| ROCm HIP SDK installation for Windows | [https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs](https://github.com/ROCm/rocm-install-on-windows/tree/develop/docs) |
1. Identify the repository and the file you want to update. For example, to update this page, you would
need to modify content located in this file:
`https://github.com/ROCm/ROCm/blob/develop/docs/contribute/contributing.md`
Individual components have their own repositories with their own documentation in their own `docs` folders.
2. (optional, but recommended) Fork the repository.
The sub-folders within the `docs` folders across ROCm are typically structured as follows:
3. Clone the repository locally and (optionally) add your fork. Select the green 'Code' button and copy
the URL (e.g., `git@github.com:ROCm/ROCm.git`).
| Sub-folder name | Documentation type |
|-------|----------|
| `install` | Installation instructions, build instructions, and prerequisites |
| `conceptual` | Important concepts |
| `how-to` | How to implement specific use cases |
| `tutorials` | Tutorials |
| `reference` | API references and other reference resources |
* From your terminal, run:
## Editing and adding to the documentation
```bash
git clone git@github.com:ROCm/ROCm.git
```
The ROCm documentation is written in [reStructuredText (rst)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html) and [Github-flavoured Markdown](https://github.github.com/gfm/), and follows the [Google developer documentation style guide](https://developers.google.com/style/highlights). reStructuredText is preferred when adding content to the documentation.
* Optionally add your fork to this local copy of the repository by running:
To edit or add to the documentation:
```bash
git add remote <name-of-my-fork> <git@github.com:my-username/ROCm.git>
```
1. Fork the repository you want to add to or edit.
2. Clone your fork locally.
3. Create a new local branch cut from the `develop` branch of the repository.
4. Make your changes to the documentation.
To get the URL of your fork, go to your GitHub profile, select the fork and click the green 'Code'
button (the same process you followed to get the main GitHub repository URL).
4. Change directory into your local copy of the repository, and run ``git pull`` (or ``git pull origin develop``) to ensure your local copy has the most recent content.
5. Create and checkout a new branch using the following command:
5. Optionally, build the documentation locally before creating a pull request by running the following commands from within the `docs` folder:
```bash
git checkout -b <branch_name>
```
6. Change directory into the `./docs` folder and make any documentation changes locally using your preferred code editor. Follow the guidelines listed on the
[documentation structure](./doc-structure.md) page.
```{note}
Spell checking is performed for pull requests by {doc}`ROCm Docs Core<rocm-docs-core:index>`. To ensure your PR passes spell checking you might need at add new words or acronyms to the `.wordlist.txt` file as described in {doc}`Spell Check<rocm-docs-core:user_guide/spellcheck>`.
```
7. Optionally run a local test build of the documentation to ensure the content builds and looks as expected. In your terminal, run the following commands from within the `./docs` folder of your cloned repository:
```bash
pip3 install -r sphinx/requirements.txt # You only need to run this command once
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
```
The build output files are located in the `docs/_build` folder. To preview your build, open the index file
(`docs/_build/html/index.html`) file. For more information, see [Building documentation](building.md). To learn
more about our build tools, see [Documentation toolchain](toolchain.md).
The output files will be located in the `docs/_build` folder. Open `docs/_build/html/index.html` to view the documentation.
8. Commit your changes and push them to GitHub by running:
For more information on ROCm build tools, see [Documentation toolchain](toolchain.md).
6. Push your changes. A GitHub link will be returned in the output of the `git push` command. Open this link in a browser to create the pull request.
```bash
git add <path-to-my-modified-file> # To add all modified files, you can use: git add .
git commit -m "my-updates"
git push <name-of-my-fork>
```
The documentation is built as part of the checks on pull request, along with spell checking and linting. Scroll to the bottom of your pull request to view all the checks.
After pushing, you will get a GitHub link in the terminal output. Copy this link and paste it into a
browser to create your PR.
Verify that the linking and spell checking have passed, and that the documentation was built successfully. New words or acronyms can be added to the [wordlist file](https://github.com/ROCm/rocm-docs-core/blob/develop/.wordlist.txt) as needed.
## Create an issue
The Read The Docs build of your pull request can be accessed by clicking on the Details link next to the Read The Docs build check. Verify that your changes are in the build and look as expected.
1. To create a new GitHub issue, select the 'Issues' tab in the appropriate repository
(e.g., https://github.com/ROCm/ROCm/issues).
2. Use the search bar to make sure the issue doesn't already exist.
3. If your issue is not already listed, select the green 'New issue' button to the right of the page. Select
the type of issue and fill in the resulting template.
Your pull request will be reviewed by a member of the ROCm documentation team.
### General issue guidelines
See the [GitHub documentation](https://docs.github.com/en) for information on how to fork and clone a repository, and how to create and push a local branch.
* Use your best judgement for issue creation. If your issue is already listed, upvote the issue and
comment or post to provide additional details, such as how you reproduced this issue.
* If you're not sure if your issue is the same, err on the side of caution and file your issue.
You can add a comment to include the issue number (and link) for the similar issue. If we evaluate
your issue as being the same as the existing issue, we'll close the duplicate.
* If your issue doesn't exist, use the issue template to file a new issue.
* When filing an issue, be sure to provide as much information as possible, including script output so
we can collect information about your configuration. This helps reduce the time required to
reproduce your issue.
* Check your issue regularly, as we may require additional information to successfully reproduce the
issue.
## Suggest a new feature
Use the [GitHub Discussion forum](https://github.com/ROCm/ROCm/discussions)
(Ideas category) to propose new features. Our maintainers are happy to provide direction and
feedback on feature development.
## Future development workflow
The current ROCm development workflow is GitHub-based. If, in the future, we change this platform,
the tools and links may change. In this instance, we will update contribution guidelines accordingly.
```{important}
By creating a pull request (PR), you agree to allow your contribution to be licensed under the terms of the
LICENSE.txt file in the corresponding repository. Different repositories can use different licenses.
```

219
docs/contribute/doc-structure.md
View File

@@ -1,219 +0,0 @@
# Documentation structure
Our documentation follows the Pitchfork folder structure. Most documentation files are stored in the
`/docs` folder. Some special files (such as release, contributing, and changelog) are stored in the root
(`/`) folder.
All images are stored in the `/docs/data` folder. An image's file path mirrors that of the documentation
file where it is used.
Our naming structure uses kebab case; for example, `my-file-name.rst`.
## Supported formats and syntax
Our documentation includes both Markdown and RST files. We are gradually transitioning existing
Markdown to RST in order to more effectively meet our documentation needs. When contributing,
RST is preferred; if you must use Markdown, use GitHub-flavored Markdown.
We use [Sphinx Design](https://sphinx-design.readthedocs.io/en/latest/index.html) syntax and compile
our API references using [Doxygen](https://www.doxygen.nl/).
The following table shows some common documentation components and the syntax convention we
use for each:
<table>
<tr>
<th>Component</th>
<th>RST syntax</th>
</tr>
<tr>
<td>Code blocks</td>
<td>
```rst
.. code-block:: language-name
My code block.
```
</td>
</tr>
<tr>
<td>Cross-referencing internal files</td>
<td>
```rst
:doc:`Title <../path/to/file/filename>`
```
</td>
</tr>
<tr>
<td>External links</td>
<td>
```rst
`link name <URL>`_
```
</td>
</tr>
<tr>
<tr>
<td>Headings</td>
<td>
```rst
******************
Chapter title (H1)
******************
Section title (H2)
===============
Subsection title (H3)
---------------------
Sub-subsection title (H4)
^^^^^^^^^^^^^^^^^^^^
```
</td>
</tr>
<tr>
<td>Images</td>
<td>
```rst
.. image:: image1.png
```
</td>
</tr>
<tr>
<td>Internal links</td>
<td>
```rst
1. Add a tag to the section you want to reference:
.. _my-section-tag: section-1
Section 1
==========
2. Link to your tag:
As shown in :ref:`section-1`.
```
</td>
</tr>
<tr>
<tr>
<td>Lists</td>
<td>
```rst
# Ordered (numbered) list item
* Unordered (bulleted) list item
```
</td>
</tr>
<tr>
<tr>
<td>Math (block)</td>
<td>
```rst
.. math::
A = \begin{pmatrix}
0.0 & 1.0 & 1.0 & 3.0 \\
4.0 & 5.0 & 6.0 & 7.0 \\
\end{pmatrix}
```
</td>
</tr>
<tr>
<td>Math (inline)</td>
<td>
```rst
:math:`2 \times 2 `
```
</td>
</tr>
<tr>
<td>Notes</td>
<td>
```rst
.. note::
My note here.
```
</td>
</tr>
<tr>
<td>Tables</td>
<td>
```rst
.. csv-table:: Optional title here
:widths: 30, 70 #optional column widths
:header: "entry1 header", "entry2 header"
"entry1", "entry2"
```
</td>
</tr>
</table>
## Language and style
We use the
[Google developer documentation style guide](https://developers.google.com/style/highlights) to
guide our content.
Font size and type, page layout, white space control, and other formatting
details are controlled via
[rocm-docs-core](https://github.com/ROCm/rocm-docs-core). If you want to notify us
of any formatting issues, create a pull request in our
[rocm-docs-core](https://github.com/ROCm/rocm-docs-core) GitHub repository.
## Building our documentation
<!-- % TODO: Fix the link to be able to work at every files -->
To learn how to build our documentation, refer to
[Building documentation](./building.md).

22
docs/contribute/feedback.md
View File

@@ -4,12 +4,24 @@
<meta name="keywords" content="documentation, pull request, GitHub, AMD, ROCm">
</head>
# Providing feedback
# Providing feedback about the ROCm documentation
Your feedback is welcome. You can provide feedback either through GitHub Discussions or GitHub Issues.
Feedback about the ROCm documentation is welcome. You can provide feedback about the ROCm documentation either through GitHub Discussions or GitHub Issues.
Use [GitHub Discussions](https://github.com/ROCm/ROCm/discussions) to ask questions, view announcements, and communicate with other members of the community.
## Participating in discussions through GitHub Discussions
Use [GitHub Issues](https://github.com/ROCm/ROCm/issues) to submit issues you find with ROCm or with the ROCm documentation.
You can ask questions, view announcements, suggest new features, and communicate with other members of the community through [GitHub Discussions](https://github.com/ROCm/ROCm/discussions).
For information about contributing to the ROCm repository and creating a pull request (PR), see [Contributing](./contributing.md).
## Submitting issues through GitHub Issues
You can submit issues through [GitHub Issues](https://github.com/ROCm/ROCm/issues).
When creating a new issue, follow the following guidelines:
1. Always do a search to see if the same issue already exists. If the issue already exists, upvote it, and comment or post to provide any additional details you might have.
2. If you find an issue that is similar to your issue, log your issue, then add a comment that includes a link to the similar issue, as well as its issue number.
3. Always provide as much information as possible. This helps reduce the time required to reproduce the issue.
After creating your issue, make sure to check it regularly for any requests for additional information.
For information about contributing content to the ROCm documentation, see [Contributing to the ROCm documentation](./contributing.md).

39
docs/contribute/toolchain.md
View File

@@ -6,60 +6,47 @@
# ROCm documentation toolchain
Our documentation relies on several open source toolchains and sites.
The ROCm documentation relies on several open source toolchains and sites.
## `rocm-docs-core`
## rocm-docs-core
[rocm-docs-core](https://github.com/ROCm/rocm-docs-core) is an AMD-maintained
project that applies customization for our documentation. This project is the tool most ROCm
repositories use as part of the documentation build. It is also available as a
[pip package on PyPI](https://pypi.org/project/rocm-docs-core/).
project that applies customizations for the ROCm documentation. This project is the tool most ROCm repositories use as part of their documentation build pipeline. It is available as a [pip package on PyPI](https://pypi.org/project/rocm-docs-core/).
See the user and developer guides for rocm-docs-core at
{doc}`rocm-docs-core documentation<rocm-docs-core:index>`.
## Sphinx
[Sphinx](https://www.sphinx-doc.org/en/master/) is a documentation generator originally used for
Python. It is now widely used in the open source community.
[Sphinx](https://www.sphinx-doc.org/en/master/) is a documentation generator originally used for Python. It is now widely used in the open source community.
### Sphinx External ToC
[Sphinx External ToC](https://sphinx-external-toc.readthedocs.io/en/latest/intro.html) is a Sphinx
extension used for ROCm documentation navigation. This tool generates a navigation menu on the left
[Sphinx External ToC](https://sphinx-external-toc.readthedocs.io/en/latest/intro.html) is a Sphinx extension used for ROCm documentation navigation. This tool generates a navigation menu on the left
based on a YAML file (`_toc.yml.in`) that contains the table of contents.
### Sphinx-book-theme
[Sphinx-book-theme](https://sphinx-book-theme.readthedocs.io/en/latest/) is a Sphinx theme that
defines the base appearance for ROCm documentation. ROCm documentation applies some
customization, such as a custom header and footer on top of the Sphinx Book Theme.
[Sphinx-book-theme](https://sphinx-book-theme.readthedocs.io/en/latest/) is a Sphinx theme that defines the base appearance for ROCm documentation. ROCm documentation applies some customization, such as a custom header and footer, on top of the Sphinx Book Theme.
### Sphinx Design
[Sphinx design](https://sphinx-design.readthedocs.io/en/latest/index.html) is a Sphinx extension that
adds design functionality. ROCm documentation uses Sphinx Design for grids, cards, and synchronized
tabs.
[Sphinx design](https://sphinx-design.readthedocs.io/en/latest/index.html) is a Sphinx extension that adds design functionality. ROCm documentation uses Sphinx Design for grids, cards, and synchronized tabs.
## Doxygen
[Doxygen](https://www.doxygen.nl/) is a documentation generator that extracts information from inline
code. ROCm projects typically use Doxygen for public API documentation (unless the upstream project
uses a different tool).
[Doxygen](https://www.doxygen.nl/) is a documentation generator that extracts information from in-code comments. It is used for API documentation.
## Breathe
[Breathe](https://www.breathe-doc.org/) is a Sphinx plugin to integrate Doxygen content.
[Breathe](https://www.breathe-doc.org/) is a Sphinx plugin for integrating Doxygen content.
## MyST
[Markedly Structured Text (MyST)](https://myst-tools.org/docs/spec) is an extended flavor of
Markdown ([CommonMark](https://commonmark.org/)) influenced by reStructuredText (RST) and
Sphinx. It's integrated into ROCm documentation by the Sphinx extension
[`myst-parser`](https://myst-parser.readthedocs.io/en/latest/).
A MyST syntax cheat sheet is available on the [Jupyter reference](https://jupyterbook.org/en/stable/reference/cheatsheet.html) site.
[Markedly Structured Text (MyST)](https://myst-tools.org/docs/spec) is an extended flavor of Markdown ([CommonMark](https://commonmark.org/)) influenced by reStructuredText (rst) and Sphinx. It is integrated into the ROCm documentation with the [`myst-parser`](https://myst-parser.readthedocs.io/en/latest/) Sphinx extension.
See the [MyST syntax cheat sheet](https://jupyterbook.org/en/stable/reference/cheatsheet.html) at the Jupyter Book site.
## Read the Docs
[Read the Docs](https://docs.readthedocs.io/en/stable/) is the service that builds and hosts the HTML
documentation generated using Sphinx to our end users.
[Read the Docs](https://docs.readthedocs.io/en/stable/) is the service that builds and hosts the HTML version of the ROCm documentation.

BIN
docs/data/contribute/commit-status.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 64 KiB

2
docs/how-to/llm-fine-tuning-optimization/model-quantization.rst
View File

@@ -181,7 +181,7 @@ Installing bitsandbytes
# Clone the github repo
git clone --recurse https://github.com/ROCm/bitsandbytes.git
cd bitsandbytes
git checkout rocm_enabled
git checkout rocm_enabled_multi_backend
# Install dependencies
pip install -r requirements-dev.txt

2
docs/how-to/llm-fine-tuning-optimization/single-gpu-fine-tuning-and-inference.rst
View File

@@ -91,7 +91,7 @@ Setting up the base implementation environment
# Use -DBNB_ROCM_ARCH to target a specific GPU architecture.
git clone --recurse https://github.com/ROCm/bitsandbytes.git
cd bitsandbytes
git checkout rocm_enabled
git checkout rocm_enabled_multi_backend
pip install -r requirements-dev.txt
cmake -DBNB_ROCM_ARCH="gfx942" -DCOMPUTE_BACKEND=hip -S .
python setup.py install

2
docs/how-to/rocm-for-ai/install.rst
View File

@@ -22,7 +22,7 @@ If you’re new to ROCm, refer to the :doc:`ROCm quick start install guide for L
<rocm-install-on-linux:install/quick-start>`.
If you’re using a Radeon GPU for graphics-accelerated applications, refer to the
:doc:`Radeon installation instructions <radeon:docs/install/install-radeon>`.
`Radeon installation instructions <https://rocm.docs.amd.com/projects/radeon/en/docs-6.1.3/docs/install/native_linux/install-radeon.html>`_.
ROCm supports multiple :doc:`installation methods <rocm-install-on-linux:install/install-overview>`:

6
docs/how-to/system-optimization/mi100.md
View File

@@ -342,8 +342,8 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to
(logical) cores of the system:
* In the server BIOS, set IOMMU to "Enabled".
* When configuring the Grub boot loader, add the following arguments for the
Linux kernel: `amd_iommu=on iommu=pt`
* When configuring the Grub boot loader, add the following argument for the
Linux kernel: `iommu=pt`
* Update Grub to use the modified configuration:
```shell
@@ -355,7 +355,7 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to
```none
[...]
[ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt
[ 0.000000] Kernel command line: [...] iommu=pt
[...]
```

6
docs/how-to/system-optimization/mi200.md
View File

@@ -327,8 +327,8 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to
(logical) cores of the system:
* In the server BIOS, set IOMMU to "Enabled".
* When configuring the Grub boot loader, add the following arguments for the
Linux kernel: `amd_iommu=on iommu=pt`
* When configuring the Grub boot loader, add the following argument for the
Linux kernel: `iommu=pt`
* Update Grub to use the modified configuration:
```shell
@@ -340,7 +340,7 @@ If SMT is enabled by setting "CCD/Core/Thread Enablement > SMT Control" to
```none
[...]
[ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt
[ 0.000000] Kernel command line: [...] iommu=pt
[...]
```

6
docs/how-to/system-optimization/mi300x.rst
View File

@@ -299,7 +299,7 @@ For a system that has AMD host CPUs add this to ``GRUB_CMDLINE_LINUX``:
.. code-block:: text
amd_iommu=on iommu=pt
iommu=pt
Otherwise, if the system has Intel host CPUs add this instead to
``GRUB_CMDLINE_LINUX``:
@@ -500,7 +500,7 @@ If SMT is enabled by setting ``CCD/Core/Thread Enablement > SMT Control`` to
#. In the server BIOS, set IOMMU to ``Enabled``.
#. When configuring the GRUB boot loader, add the following arguments for the Linux kernel: ``amd_iommu=on iommu=pt``.
#. When configuring the GRUB boot loader, add the following argument for the Linux kernel: ``iommu=pt``.
#. Update GRUB.
@@ -515,7 +515,7 @@ If SMT is enabled by setting ``CCD/Core/Thread Enablement > SMT Control`` to
.. code-block:: shell
[...]
[ 0.000000] Kernel command line: [...] amd_iommu=on iommu=pt
[ 0.000000] Kernel command line: [...] iommu=pt
[...]
Once the system is properly configured, ROCm software can be

2
docs/how-to/system-optimization/w6000-v620.md
View File

@@ -111,7 +111,7 @@ sudo virsh net-start default /*to enable Virtual network by default
Enable input-output memory management unit (IOMMU) in GRUB settings by adding the following line to `/etc/default/grub`:
```none
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on" for AMD CPU
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" for AMD CPU
```
Update grub and reboot

3
docs/index.md
View File

@@ -46,6 +46,7 @@ ROCm documentation is organized into the following categories:
* [Fine-tuning LLMs and inference optimization](./how-to/llm-fine-tuning-optimization/index.rst)
* [System optimization](./how-to/system-optimization/index.rst)
* [AMD Instinct MI300X tuning guides](./how-to/tuning-guides/mi300x/index.rst)
* [GPU cluster networking](https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/latest/index.html)
* [System debugging](./how-to/system-debugging.md)
* [Using MPI](./how-to/gpu-enabled-mpi.rst)
* [Using advanced compiler features](./conceptual/compiler-topics.md)
@@ -63,7 +64,7 @@ ROCm documentation is organized into the following categories:
* [Using CMake](./conceptual/cmake-packages.rst)
* [ROCm & PCIe atomics](./conceptual/More-about-how-ROCm-uses-PCIe-Atomics.rst)
* [Inception v3 with PyTorch](./conceptual/ai-pytorch-inception.md)
* [Inference optimization with MIGraphX](./conceptual/ai-migraphx-optimization.md)
* [Oversubscription of hardware resources](./conceptual/oversubscription.rst)
:::
<!-- markdownlint-disable MD051 -->

2
docs/reference/gpu-arch-specs.rst
View File

@@ -7,6 +7,8 @@ Accelerator and GPU hardware specifications
The following tables provide an overview of the hardware specifications for AMD Instinct™ accelerators, and AMD Radeon™ PRO and Radeon™ GPUs.
For more information about ROCm hardware compatibility, see the ROCm `Compatibility matrix <https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html>`_.
.. tab-set::
.. tab-item:: AMD Instinct accelerators

2
docs/reference/precision-support.rst
View File

@@ -41,6 +41,8 @@ together with their corresponding HIP type and a short description.
- ``int64_t``, ``uint64_t``
- A signed or unsigned 64-bit integer
.. _precision_support_floating_point_types:
Floating-point types
==========================================

19
docs/sphinx/_toc.yml.in
View File

@@ -77,9 +77,11 @@ subtrees:
title: System tuning
- file: how-to/tuning-guides/mi300x/workload.rst
title: Workload tuning
- file: how-to/system-debugging.md
- url: https://rocm.docs.amd.com/projects/gpu-cluster-networking/en/${branch}/index.html
title: GPU cluster networking
- file: how-to/gpu-enabled-mpi.rst
title: Using MPI
- file: how-to/system-debugging.md
- file: conceptual/compiler-topics.md
title: Using advanced compiler features
subtrees:
@@ -156,8 +158,8 @@ subtrees:
title: ROCm & PCIe atomics
- file: conceptual/ai-pytorch-inception.md
title: Inception v3 with PyTorch
- file: conceptual/ai-migraphx-optimization.md
title: Inference optimization with MIGraphX
- file: conceptual/oversubscription.rst
title: Oversubscription of hardware resources
- caption: Reference
entries:
@@ -173,16 +175,13 @@ subtrees:
- caption: Contribute
entries:
- file: contribute/contributing.md
title: Contribute to ROCm docs
title: Contributing to the ROCm docmentation
subtrees:
- entries:
- file: contribute/doc-structure.md
title: Documentation structure
- file: contribute/toolchain.md
title: Documentation toolchain
title: ROCm documentation toolchain
- file: contribute/building.md
title: Build our documentation
- file: contribute/feedback.md
title: Provide feedback
title: Providing feedback about the ROCm documentation
- file: about/license.md
title: ROCm license
title: ROCm licenses

2
docs/sphinx/requirements.in
View File

@@ -1,2 +1,2 @@
rocm-docs-core==1.8.1
rocm-docs-core==1.8.3
sphinx-reredirects

2
docs/sphinx/requirements.txt
View File

@@ -92,7 +92,7 @@ requests==2.32.3
# via
# pygithub
# sphinx
rocm-docs-core==1.8.1
rocm-docs-core==1.8.3
# via -r requirements.in
smmap==5.0.1
# via gitdb

2
tools/autotag/components.xml
View File

@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="rocm-org" fetch="https://github.com/ROCm/" />
<default revision="refs/tags/rocm-6.1.1"
<default revision="refs/tags/rocm-6.2.2"
remote="rocm-org"
sync-c="true"
sync-j="4" />

Some files were not shown because too many files have changed in this diff Show More